Penstock

Sources decode input into columnar RecordBatch buffers: primitive integer/float arrays and variable-length byte regions. Batch reuse reduces per-record object allocation; passing references within one process avoids serialization.

An FNV-1a key hash or round-robin routing selects a partition. Per-partition SPSC rings use LMAX Disruptor, with transforms inline on the same path. Backpressure and partition/core budgets regulate ingress and processing resources.

BatchBuffer flushes by row count or time. A separate I/O executor passes batches to DrainTarget, with outbox and circuit-breaker behavior supporting recovery. High throughput is a design objective; actual rates depend on protocols, transforms, storage, and hardware.

The gateway connects to a device’s TCP server and receives monitoring ASDUs. Spontaneous messages and general interrogation carry addresses, timestamps, and quality into collected records.

Control ASDUs use the same session. Select/Operate commands pass through the command extension’s policy and approval checks, with device-specific addresses and supported commands configured for each target.

Plain Java and ServiceLoader compose modules without a DI framework. Connectors, transforms, and extensions register through SPIs, with Typesafe HOCON configuration. The runtime handles daemon, one-shot, and scheduled execution.

Disruptor implements partition rings and Netty handles network I/O. Apache Ignite supports distributed state and analysis windows. Agrona, DataSketches, and fastdoubleparser support processing; expressions compile ahead of batch evaluation.

Resilience4j supplies reconnect backoff. The sink-path circuit breaker is a separate heartbeat-based implementation, rather than the library’s standard circuit breaker.