Audience: All Levels
Technical level: Introductory
Lightbend Pipelines is a unique, powerful system for building scalable, streaming-data pipelines composed of multi-component streamlets written in one or more streaming engines.
Most streaming data solutions comprise multiple components, implemented in different technologies, which you have to integrate yourself. What you use for streaming ETL is different than what you use for aggregations or model serving. Yet these various components must act as a single service to the external services.
Lightbend Pipelines supports streamlets written in Akka Streams and Spark Structured Streaming, transparently integrated together.
- Developer productivity is accelerated by eliminating much of the boilerplate required when writing such applications, especially when composing domain-logic streamlets into complete applications.
- The Pipelines runtime environment automates many tasks, such as deployment, routing data between streamlets, and ensuring data safety.
- A powerful, intuitive GUI visualizes the running pipelines with easy access to important health and performance metrics.
- Management is simplified by automatic handling of many failure scenarios and restarts.
This brief provides context for Lightbend Pipelines helps you understand how it works with microservices to accelerate your journey toward the real-time enterprise.
Grab your copy
Please enter your information to receive your E-book chapter(s) of Lightbend Pipelines: Accelerate Your Real-Time Streaming Journey and be signed up for the Lightbend Newsletter. Once you've entered your information and submitted the form, the PDF will be emailed to your address.