Examples
Runnable examples live in the volley-examples crate. Six examples, each
one DAG shape.
Running
# Memory-only examples (no external deps)
cargo run --example in_memory_pipeline -p volley-examples
cargo run --example windowed_pipeline -p volley-examples
cargo run --example fanout_multisink -p volley-examples
cargo run --example observable_pipeline -p volley-examples
# Kafka (requires a live broker + two topics)
cargo run --example kafka_pipeline -p volley-examples --features kafka
# ML (first run downloads a HuggingFace model; cached thereafter)
cargo run --example ml_inference_pipeline -p volley-examples --features ml --release
What each example shows
| Example | What it teaches | Features |
|---|---|---|
in_memory_pipeline | Minimum DAG shape: from_source → filter_expr → to_sink, plus the sink.handle() pattern for reading output after execute(). | (none) |
windowed_pipeline | Tumbling / sliding / session windows over key_by → aggregate_expr, event-time semantics, RocksDB state backend. | (none) |
fanout_multisink | DAG 1.0 marquee: tee() broadcasts to two branches, each filters differently, both sinks commit atomically in one 2PC epoch. | (none) |
observable_pipeline | with_observability(...) + HealthReporter + spawn_health_server → /healthz / /readyz / /startupz; Prometheus / OTLP hooks are one uncomment away. | (none) |
kafka_pipeline | Kafka-to-Kafka exactly-once: from_kafka_exactly_once returns (stream, ConsumerGroupMetadata); to_kafka_exactly_once threads the CGM so source offsets commit inside the sink’s Kafka transaction. | kafka |
ml_inference_pipeline | Streaming ONNX classification, backend auto-select (CoreML / CUDA / CPU), HuggingFace Hub auto-download. | ml |
NEXMark benchmarks
Throughput and windowed-join benchmarks live in a sibling crate:
cargo run --example nexmark_q1 -p volley-benchmark --release
cargo run --example nexmark_q5 -p volley-benchmark --release
cargo run --example nexmark_q8 -p volley-benchmark --release
See volley-benchmark/README.md
for methodology and baselines.
Walkthrough
See Your first pipeline for a step-by-step of
in_memory_pipeline.