`mineproductivity.connectors`¶

Auto-generated API reference for the Connectors package, rendered from the package's own docstrings. For the narrative overview, dependency rules, and extension guide, see Packages -> Connectors.

connectors ¶

mineproductivity.connectors -- how MineProductivity meets the real world.

Defines the single, small contract every data source -- a fleet- management system, a CSV export, a REST API, a Kafka topic -- must satisfy to feed the platform, and is the only place in the codebase permitted to know that a specific vendor or file format exists (the Cookbook's Vendor-Neutrality principle).

Implements docs/architecture/04_Connector_Framework_Design_Specification.md exactly. connectors depends on core, ontology, events, and registry -- see README.md for the full set of architectural rules this package must satisfy.

Everything documented here is part of the public API and can be imported directly from mineproductivity.connectors, e.g.::

from mineproductivity.connectors import get_connector, CONNECTORS

AuthProvider ¶

Bases: ABC

Isolates credential acquisition/refresh from connector I/O logic.

:meth:refresh MUST be safe to call concurrently without acquiring duplicate tokens or corrupting the cached :class:Credentials (design spec §24) -- this is the one thread-safety guarantee this contract makes mandatory rather than merely documented, since concurrent 401-triggered refreshes are a realistic, common failure mode for any multi-threaded ingestion pipeline.

credentials `abstractmethod` ¶

credentials()

Return the current, possibly cached, credentials.

refresh `abstractmethod` ¶

refresh()

Force acquisition of a fresh :class:Credentials, replacing whatever is cached. Safe to call concurrently (see class docstring).

Credentials `dataclass` ¶

Credentials(token, *, expires_at_utc=None)

Bases: BaseValueObject

A bearer token and its (optional) expiry.

FMSConnector ¶

Bases: ABC

Adapter from a fleet-management (or any other) source to canonical events. This is the entire contract every source-specific adapter implements -- deliberately small (Cookbook Part I, Ch. 7: "Every connector implements one small abstract base class").

Only :meth:get_cycle_data and :meth:get_delay_data are abstract (design spec AD-CN-01); the remaining four get_*_data methods have no-op defaults so a source that only produces cycles is not forced to implement methods it has nothing to yield for.

Every get_*_data implementation MUST be a generator or otherwise lazy Iterable -- never a materialized list (Cookbook Part I, Ch. 7's Python Insight: "the same connector handles a 200-row test file or a 50-million-row shift export"). This is mechanically checked by :func:~mineproductivity.connectors.contract_tests.run_fms_contract_suite.

get_cycle_data `abstractmethod` ¶

get_cycle_data(since, until)

Yield :class:CycleEvent\ s whose source timestamp falls in [since, until).

get_delay_data `abstractmethod` ¶

get_delay_data(since, until)

Yield :class:DelayEvent\ s whose source timestamp falls in [since, until).

get_maintenance_data ¶

get_maintenance_data(since, until)

Optional: default implementation yields nothing. Override for sources that carry maintenance telemetry.

get_production_data ¶

get_production_data(since, until)

Optional: default implementation yields nothing.

get_consumption_data ¶

get_consumption_data(since, until)

Optional: default implementation yields nothing.

get_safety_data ¶

get_safety_data(since, until)

Optional: default implementation yields nothing.

health_check ¶

health_check()

Default: assumes healthy state is unknown until a real pull happens. Override for sources with a real liveness/auth check.

provided_event_types `classmethod` ¶

provided_event_types()

Which of the six get_*_data methods this class actually overrides (as opposed to inheriting :class:FMSConnector's no-op default), so discovery/documentation tooling can report what a connector is good for without instantiating it (design spec §18).

IngestionMode ¶

Bases: Enum

The three ways a caller can drive an :class:FMSConnector.

AuthenticationError ¶

AuthenticationError(message, *, details=None)

Bases: ConnectorError

An :class:~mineproductivity.connectors.auth.AuthProvider could not obtain or refresh valid credentials.

ConnectorError ¶

ConnectorError(message, *, details=None)

Bases: MineProductivityError

Root of connector-specific errors.

ContractViolationError ¶

ContractViolationError(message, *, details=None)

Bases: ConnectorError

A connector implementation failed the shared contract test suite -- e.g. it returned a list instead of a lazy Iterable, or yielded an event outside the requested [since, until) window.

MappingError ¶

MappingError(message, *, details=None)

Bases: ConnectorError

A raw record could not be normalized -- e.g. an unrecognized vendor reason code with no :class:~mineproductivity.connectors.normalization.ReasonCodeMap entry, or a malformed numeric field.

SourceUnavailableError ¶

SourceUnavailableError(message, *, details=None)

Bases: ConnectorError

The source is unreachable (network, file-not-found, ...) -- the default retryable exception type.

CSVConnector ¶

CSVConnector(path, shift_id, *, delay_path=None, source_timezone='UTC', normalizer=None)

Bases: FMSConnector

Reads a haul-cycle (and, optionally, delay) CSV export -- the reference implementation of the minimal FMSConnector contract. See Cookbook Part I, Ch. 7 for the full worked version this class's shape is drawn from.

Two source files, not one, because a real FMS export typically ships cycles and delays as separate homogeneous-schema files rather than interleaved rows of different shapes; delay_path is optional since some sources (like a cycle-only export) have no delay data.

Expected columns -- cycles: event_time, equipment_id, queue_min, spot_min, load_min, haul_min, dump_min, return_min, payload_t, plus optional route_id, operator_id, material_type. Delays: event_time, equipment_id, delay_category, delay_reason, duration_min, where delay_category is a :class:~mineproductivity.ontology.DelayCategory member name.

Not thread-safe for concurrent calls against the same instance -- each call re-opens and re-reads its source file independently; construct one instance per worker for concurrent ingestion (design spec §24).

ExcelConnector ¶

ExcelConnector(path, shift_id, *, delay_path=None, source_timezone='UTC', normalizer=None)

Bases: FMSConnector

Reads a haul-cycle (and, optionally, delay) .xlsx export, using the identical row shape and normalization logic as :class:~mineproductivity.connectors.file.csv_connector.CSVConnector (first sheet only; first row is the header row).

Requires the optional openpyxl dependency (the connectors extra) -- imported lazily inside :meth:_read, never at module import time, so importing this module (or mineproductivity.connectors itself) never requires openpyxl to be installed unless an ExcelConnector is actually instantiated and read from.

Not thread-safe for concurrent calls against the same instance, for the same reason as CSVConnector (design spec §24).

ConnectorHealth `dataclass` ¶

ConnectorHealth(status, *, last_successful_pull_utc=None, detail='')

Bases: BaseValueObject

A point-in-time snapshot of one connector's health.

HealthStatus ¶

Bases: Enum

The four health states a connector can report, updated after every pull attempt (design spec §12).

GraphQLConnector ¶

GraphQLConnector(endpoint, auth, retry, shift_id, *, normalizer=None, timeout_s=10.0)

Bases: FMSConnector

Queries a GraphQL endpoint over HTTP POST (stdlib :mod:urllib only), yielding one event per record returned under the queried root field (cycles/delays).

A reference-shape implementation: one request per pull (no cursor- based pagination), matching the design specification's brief __init__-only sketch for this connector (§10.6, "follow the identical shape"). Real GraphQL sources with cursor pagination are expected to override :meth:_query in a subclass or plugin.

Not thread-safe for concurrent calls against the same instance (design spec §24).

RestConnector ¶

RestConnector(base_url, auth, retry, shift_id, *, normalizer=None, timeout_s=10.0)

Bases: FMSConnector

Pages through an HTTP API using the standard library's :mod:urllib -- no third-party HTTP client dependency.

Expects each page's JSON response shaped as {"records": [...], "has_more": bool}, where each record has the same field shape :class:~mineproductivity.connectors.file._common.FileRowNormalizer already knows how to translate (reused here rather than duplicated). A 401 response triggers exactly one :meth:AuthProvider.refresh and retry before raising; a network or 5xx failure is wrapped as :class:~mineproductivity.connectors.exceptions.SourceUnavailableError and handled by the injected :class:RetryPolicy.

Not thread-safe for concurrent calls against the same instance; construct one instance per worker for concurrent ingestion (design spec §24). The injected auth's own :meth:AuthProvider.refresh remains safe to call concurrently across different connector instances sharing the same provider.

FieldMapper `dataclass` ¶

FieldMapper(mapping)

Bases: BaseValueObject

Declarative field-name mapping: vendor field name -> canonical field name. Fields absent from mapping pass through unchanged.

Examples:

>>> mapper = FieldMapper(mapping={"TruckID": "equipment_id"})
>>> mapper.apply({"TruckID": "HT-214", "PayloadTonnes": 220.0})
{'equipment_id': 'HT-214', 'PayloadTonnes': 220.0}

apply ¶

apply(raw)

Return a new dict with every key in raw renamed per :attr:mapping (keys absent from the mapping are copied as-is).

Normalizer ¶

Bases: ABC

Applies :class:FieldMapper + :class:ReasonCodeMap to translate one raw source record into a canonical :class:~mineproductivity.events.base_event.BaseEvent.

Separated from :class:~mineproductivity.connectors.base.FMSConnector itself so mapping logic is independently unit-testable without a live/fixture source connection.

normalize_cycle `abstractmethod` ¶

normalize_cycle(raw)

Translate one raw cycle record into a :class:CycleEvent.

Raises:

Type	Description
`MappingError`	If `raw` cannot be translated (a missing required field, a non-numeric value where a number is required, ...).

normalize_delay `abstractmethod` ¶

normalize_delay(raw)

Translate one raw delay record into a :class:DelayEvent.

Raises:

Type	Description
`MappingError`	If `raw` cannot be translated.

ReasonCodeMap `dataclass` ¶

ReasonCodeMap(vendor_name, mapping)

Bases: BaseValueObject

Vendor delay-reason code -> canonical (DelayCategory, reason).

Cookbook Part I, Ch. 7: "The hardest part of a real connector is the reason-code map... Get this mapping right and every downstream delay analysis becomes comparable."

Examples:

>>> reason_map = ReasonCodeMap(
...     vendor_name="minestar",
...     mapping={"MECH_FAIL": (DelayCategory.EQUIPMENT, "mechanical failure")},
... )
>>> resolved = reason_map.resolve("MECH_FAIL")
>>> resolved.unwrap()
(<DelayCategory.EQUIPMENT: 'Equipment'>, 'mechanical failure')
>>> reason_map.resolve("UNKNOWN_CODE").is_nothing
True

resolve ¶

resolve(vendor_code)

Non-raising lookup of vendor_code.

BackoffStrategy ¶

Bases: Enum

The three backoff shapes a :class:RetryPolicy can compute.

RetryPolicy `dataclass` ¶

RetryPolicy(*, max_attempts=3, backoff=BackoffStrategy.EXPONENTIAL_JITTER, base_delay_s=1.0, retryable_exceptions=(SourceUnavailableError,))

Bases: BaseConfiguration

Retry/backoff configuration for a network-connected :class:~mineproductivity.connectors.base.FMSConnector.

compute_delay ¶

compute_delay(attempt, *, jitter_fn=random.random)

The backoff delay (seconds) before retry number attempt (1-indexed: the delay before the second overall attempt is compute_delay(1)).

is_retryable ¶

is_retryable(exc)

Whether exc is one of :attr:retryable_exceptions.

KafkaConnector ¶

KafkaConnector(topic, shift_id, *, cycle_source=None, delay_source=None, source_timezone='UTC', normalizer=None)

Bases: FMSConnector

Subscribes to a topic; yields one event per message. Naturally STREAMING-mode; :meth:get_cycle_data/:meth:get_delay_data iterate over the live subscription rather than performing a bounded pull.

Does not depend on a specific Kafka client library -- see this package's streaming/_common.py module docstring. cycle_source/ delay_source stand in for a real client's consumer iterator (a real plugin wraps kafka-python/confluent-kafka to produce this shape); either may be omitted for a topic that only carries one message kind.

Not thread-safe for concurrent calls against the same instance -- a message source (a real Kafka consumer) is typically not safe for concurrent iteration either (design spec §24).

MqttConnector ¶

MqttConnector(topic_filter, shift_id, *, cycle_source=None, delay_source=None, source_timezone='UTC', normalizer=None)

Bases: FMSConnector

Subscribes to an MQTT topic filter; yields one event per published message. Follows the identical shape as :class:~mineproductivity.connectors.streaming.kafka_connector.KafkaConnector (see its docstring and this package's streaming/_common.py module docstring for why no broker client library is a dependency of this package).

Not thread-safe for concurrent calls against the same instance (design spec §24).

get_connector ¶

get_connector(name)

Raising lookup of a registered connector class by its name.

Raises:

Type	Description
`UnregisteredLookupError`	If no connector is registered under `name`.

mineproductivity.connectors¶

connectors ¶

AuthProvider ¶

credentials abstractmethod ¶

refresh abstractmethod ¶

Credentials dataclass ¶

FMSConnector ¶

get_cycle_data abstractmethod ¶

get_delay_data abstractmethod ¶

get_maintenance_data ¶

get_production_data ¶

get_consumption_data ¶

get_safety_data ¶

health_check ¶

provided_event_types classmethod ¶

IngestionMode ¶

AuthenticationError ¶

ConnectorError ¶

ContractViolationError ¶

MappingError ¶

SourceUnavailableError ¶

CSVConnector ¶

ExcelConnector ¶

ConnectorHealth dataclass ¶

HealthStatus ¶

GraphQLConnector ¶

RestConnector ¶

FieldMapper dataclass ¶

apply ¶

Normalizer ¶

normalize_cycle abstractmethod ¶

normalize_delay abstractmethod ¶

ReasonCodeMap dataclass ¶

resolve ¶

BackoffStrategy ¶

RetryPolicy dataclass ¶

compute_delay ¶

is_retryable ¶

KafkaConnector ¶

MqttConnector ¶

get_connector ¶

`mineproductivity.connectors`¶

credentials `abstractmethod` ¶

refresh `abstractmethod` ¶

Credentials `dataclass` ¶

get_cycle_data `abstractmethod` ¶

get_delay_data `abstractmethod` ¶

provided_event_types `classmethod` ¶

ConnectorHealth `dataclass` ¶

FieldMapper `dataclass` ¶

normalize_cycle `abstractmethod` ¶

normalize_delay `abstractmethod` ¶

ReasonCodeMap `dataclass` ¶

RetryPolicy `dataclass` ¶