Node Client Support for Streamdal

We're updating our documentation, so the presented info might not be the most recent.

Welcome to the Node Client Guide for Streamdal! If you’re harnessing the power of Node.js and aim to supercharge your data operations, you’re on the right track. Integrating Streamdal into your Node applications is both efficient and straightforward with our Node SDK.

For detailed information on the Node SDK and its functionalities, refer to the Node SDK README.

Instrumentation

Start with the instrumentation guide if you haven’t already. It will walk you through the process of setting up Streamdal and its components.

Integrate the SDK:

Install the SDK package:

npm install @streamdal/node-sdk

Dive Into an Example

Here’s a basic example to help you get started with Streamdal Node SDK:

import { registerStreamdal, StreamdalConfigs, Audience, OperationType } from "@streamdal/node-sdk";

const config: StreamdalConfigs = {
  streamdalUrl: "localhost:8082",
  streamdalToken: "1234",
  serviceName: "test-service-name",
  pipelineTimeout: "100",
  stepTimeout: "10",
  dryRun: "false",
};

const audience: Audience = {
  serviceName: "test-service-name",
  componentName: "kafka",
  operationType: OperationType.CONSUMER,
  operationName: "test-kafka-consumer",
};

export const example = async () => {
  const streamdal = await registerStreamdal(config);
  const result = await streamdal.processPipeline({
    audience,
    data: new TextEncoder().encode(JSON.stringify(exampleData)),
  });

  console.log("Streamdal Response:");
  console.dir(result, {depth: 20});
};

This example demonstrates how to initialize a new instance of the Streamdal SDK, and how to process data through a pipeline. You simply need to create a configuration object, an audience object, and then call the processPipeline method on a new Streamdal instance. The result will be logged to the console.

Wasm Execution

To run pipelines with minimal overhead, the Streamdal Node SDK ships and executes pipeline rules as Wasm. If you are using a Node version lower than 20.*, you’ll need to enable Wasm functionality in your Node app by supplying the flag as shown below:

node --experimental-wasi-unstable-preview1 ./build/sandbox/index.js

Libraries

Here are the current list of Go libraries/wrappers/shims: