Streamdal Core Components

We're updating our documentation, so the presented info might not be the most recent.

1. Streamdal SDK

The SDK, designed for efficiency, tackles metrics reporting, pipeline processing, and validations using WebAssembly (Wasm). This approach assures sub-millisecond overhead, immense scalability, and cross-language compatibility.

Key Features:

  • Lightweight Design: Tailored for efficiency, avoiding unnecessary bulk.
  • Real-time Rule Processing: Employs Wasm, achieving remarkable overheads.
  • Console UI-Defined Rules: Rules are directly defined via the Console UI and fetched by the SDK for processing.

For more details, explore the Streamdal SDK documentation.

2. Streamdal Server

Acting as the pivotal hub in Streamdal’s framework, the Server ensures fluent transformation of Console UI-defined rules into live operations. It integrates rules into the WebAssembly (Wasm) format, which the SDK impeccably deciphers and executes.

Core Features:

  • Wasm Integration: Translates rules for swift, real-time operations.
  • High Availability: Crafted for uninterrupted and consistent data operations.
  • Multi-Component Communication: Vital in interfacing between different components for smooth data operations.

For more details, explore the Streamdal Server documentation.

3. Streamdal Console UI

The Console UI stands as the orchestrator, steering the intricate nuances of data operations. Rooted in a robust GRPC framework, it offers a communicative bridge with the Console Server, granting users an interactive platform for data rule and pipeline management.

Dashboard View

  • Data Graph: Visual representation of consumers, producers, and topics relationships.
  • Real-time Stats: Insights into performance metrics and data flow rates.
  • Tail View: Live observation of pipeline data, granting immediate operational feedback.
  • Schema Discovery: Explore, define, and manage schemas generated from your data events.

Beta Pipeline View

  • Detective Rules: Define guidelines for handling Personal Identifiable Information (PII).
  • Encode & Decode Rules: Manage data encoding and decoding.
  • Transformation Rules: Set dynamic data transformation parameters.

For more details, explore the Streamdal Console UI documentation.

Combining the three components Streamdal’s SDK, Server, and Console UI forms the essence of its observability paradigm.

Each component plays a distinct role, yet their synergy is what makes Streamdal a holistic solution for data operations. Dive into each, mastering their capabilities, and redefine your data’s journey.