L3 columnar windowing at sub-millisecond latency, built on Apache Arrow.
Portfolio VaR, expected shortfall and risk contributions recomputed every window, with rolling covariance, factor exposure and limit breaches routed to alerts and controls.
Toxic-flow, imbalance and adverse-selection metrics over normalised L3 order-book data.
Windowed momentum, mean-reversion and other signals running live on the same engine used for historical validation.
Slippage, fill quality and benchmark performance calculated from live orders and market data.
Continuous analytics and risk across 24-hour exchange feeds, without splitting the workload across separate systems.
Keep your notebooks and models. Liquid runs them in the live pipeline at about 3 μs round trip in-process, or about 30 μs with full process-level isolation.
When research runs in Python
And production runs in C++ or Rust
Alpha decays during the rewrite.
In financial markets, the value of a signal depends on how quickly it can be tested, trusted and acted on. SpaceCell Lightning lets teams test a signal on history and run it live with the same code, alongside their existing infrastructure.
A historical dataset is treated as a bounded stream. The analytical logic stays the same, and only the data source changes.
Rolling covariance and correlation refreshed in milliseconds, factor exposure through in-pipeline regression, and multivariate breach detection.
Thresholds route breaches immediately to alerts, dashboards or trading-halt hooks, in the same process that computed them.
A live console shows the pipeline as a flowing DAG, with per-block metrics, latency distributions and searchable traces. Metrics are also exposed for Prometheus and Grafana.
The standard desk workflow maintains two versions of every model: the Python research implementation and the production port in C++ or Rust.
Every change must be reproduced, divergence becomes an ongoing audit risk, and the time spent porting is time lost from a validated signal. Each tool adds its own opinion, which makes reconciliation harder than it needs to be.
Lightning removes the duplicate implementation. The pipeline tested on historical data runs against the live feed on the same engine and code path, reducing the gap between validation and deployment.
if window.f64() > var_limit { reduce_exposure(&weights)?; }SimulatedLightning runs risk calculations on the live stream: rolling covariance and correlation refreshed in milliseconds, factor exposure through in-pipeline regression, and multivariate breach detection using measures such as Mahalanobis distance and Hotelling's T².
Thresholds can route breaches immediately to alerts, dashboards or trading-halt hooks. High-precision statistical implementations reduce cancellation error and preserve numerical accuracy over long-running streams.
if toxicity > threshold { widen_quotes(&book, 2.0)?; }SimulatedA market-making desk monitors flow toxicity against its resting quotes. Order-flow imbalance, toxic-flow and adverse-selection measures are computed in a Lightning pipeline over full-depth L3 order-book data, and a threshold breach signals the desk's quoting logic to widen spreads or reduce size until conditions normalise.
Thresholds are calibrated on historical L3 data through the same pipeline before it runs against the live feed.
let scored = rt.with_python(&window, |py, table| { # Python ML model over shared memory model.predict(table.to_polars()) })?;
Researchers who would rather stay in Python can do so. On top of Lightning Python, SpaceCell's Liquid bridge also embeds ML inline, scores the live window, and returns the result over Apache Arrow in the same pipeline. Python is confined to the model call, and the rest of the pipeline stays in Rust.
Your statistical models and machine-learning code run inside the live pipeline unchanged.
The statistical suite is validated against high-precision reference results, with documented comparisons to SciPy, R and statsmodels. Results typically agree within 1e-15 and often exceed the numerical accuracy of standard library implementations.
Online algorithms are comprehensively tested to ensure that long-running, persistent metrics update incrementally without catastrophic floating-point precision loss.
The Rust API is high-level and intuitive, while retaining the full performance stack: SIMD-accelerated kernels, adaptive parallelism, and extensive memory, cache and operator-fusion optimisations.
Python and supercharged SQL wrappers of the core engine provide flexibility for quantitative developers who would like to direct their focus to modelling the task at hand, with compilation to Rust available from SpaceQl.
let var: f64 = returns .ewma_covariance_matrix(n_symbols, 0.94) .value_at_risk(&weights, 2.33) .collect()?;
let var: LiveStream<Scalar> = returns .dam("100ms") .ewma_covariance_matrix(n_symbols, 0.94) .value_at_risk(&weights, 2.33) .live()?; for window in var { // every 100ms, across the whole book }
Lightning operates the analytical path between data infrastructure and the systems that receive or act on its output.
| Layer | Lightning | Remains in your stack |
|---|---|---|
| Market-data feeds and normalisation | - | Your feed handlers |
| Transport, durability and replay | - | Kafka, Aeron or your existing bus |
| Historical and tick storage | - | kdb+, ClickHouse, Parquet or your lakehouse |
| Research and backtesting | The same engine used for live execution | Your notebooks and Python tools, embeddable in Lightning through its Python bindings |
| Signal and feature computation | One pipeline across historical and live data | - |
| Statistical modelling | Regression, hypothesis tests, distributions and time series | - |
| ML training | - | Your Python and ML stack |
| ML inference in the live path | Through SpaceCell's Liquid bridge at about 3 μs round trip in-process, or about 30 μs with process isolation | - |
| Live risk, TCA and surveillance | Computed on the stream | - |
| Order and execution management | Does not replace OMS and EMS | Your OMS and EMS |
| Tick-to-trade order placement | Not tick-to-trade | Your latency-critical C++ or FPGA path |
| Pipeline observability | Native console, plus Prometheus and Grafana integration | Your existing Prometheus and Grafana stack |
Lightning consolidates live analytics, statistical computing, model inference and observability without conflicting with the infrastructure you already rely on.
Keep them. Lightning reads history from Parquet and Arrow IPC and runs beside your store as the live compute and statistics layer. You can adopt one open layer or the whole engine, alongside what you already operate.
No. Lightning is the analytical and statistical plane beside your execution stack. Order placement and the microstructure hot path stay on your own fast path.
Yes. Your statistical models and machine-learning code run inside the live pipeline through the Liquid bridge, unchanged. Researchers stay in Python, and the engine runs it live.
No. Demos run on synthetic data, and evaluation stays within your walls. We do not ask for production data, proprietary detail or trading strategies at any point.
Yes. Lightning is a single-process executable binary with a small, auditable set of dependencies, largely built in-house. It runs in your colo, your cloud or on a workstation, entirely within your environment.
A live engine with full-precision statistical modelling, parallel execution, streaming, adaptive engine design, windowing semantics and observability is a large undertaking. Leading firms in this space have capable engineering teams, many of which have already invested in similar capabilities. Lightning offers fast live data processing that holds up under intense market load, at a lower cost than building in-house, with an accelerated path to outcome and a company whose full-time focus is the engine and your support. Where many engines specialise in one workload, Lightning's adaptive design performs well across both general and specialised contexts. It is also composable: you can adopt it at the level that suits you, from a more focused yet powerful open-source Arrow implementation to the full engine, alongside your existing architecture, whilst your own investments continue to compound.
Request a confidential technical walkthrough.