WEBINAR REPLAY Reactive, fast-data, akka
Lessons Learned From PayPal: Implementing Back-Pressure With Akka Streams And Kafka
With Akara Sucharitakul, Principal MTS, Global Platform Frameworks, PayPal, Inc.
Audience: Architects, Management
Technical level: Intermediate
Akka Streams and its amazing handling of streaming with back-pressure should be no surprise to anyone. But it takes a couple of use cases to really see it in action - especially in use cases where the amount of work continues to increase as you’re processing it. This is where back-pressure really shines.
In this talk for Architects and Dev Managers by Akara Sucharitakul, Principal MTS for Global Platform Frameworks at PayPal, Inc., we look at how back-pressure based on Akka Streams and Kafka is being used at PayPal to handle very bursty workloads.
In addition, Akara will also share experiences in creating a platform based on Akka and Akka Streams that currently processes over 1 billion transactions per day (on just 8 VMs), with the aim of helping teams adopt these technologies. In this webinar, you will:
- Start with a sample web crawler use case to examine what happens when each processing pass expands to a larger and larger workload to process.
- Review how we use the buffering capabilities in Kafka and the back-pressure with asynchronous processing in Akka Streams to handle such bursts.
- Look at lessons learned, plus some constructive “rants” about the architectural components, the maturity, or immaturity you’ll expect, and tidbits and open source goodies like memory-mapped stream buffers that can be helpful in other Akka Streams and/or Kafka use cases.
WEBINAR REPLAY Reactive, fast-data, akka
Lessons Learned From PayPal: Implementing Back-Pressure With Akka Streams And Kafka
With Akara Sucharitakul, Principal MTS, Global Platform Frameworks, PayPal, Inc. and Justin Pihony, Developer Support Engineer at Lightbend, Inc.
Audience: Architects, Management
Technical level: Intermediate
Akka Streams and its amazing handling of streaming with back-pressure should be no surprise to anyone. But it takes a couple of use cases to really see it in action - especially in use cases where the amount of work continues to increase as you’re processing it. This is where back-pressure really shines.
In this talk for Architects and Dev Managers by Akara Sucharitakul, Principal MTS for Global Platform Frameworks at PayPal, Inc., we look at how back-pressure based on Akka Streams and Kafka is being used at PayPal to handle very bursty workloads.
In addition, Akara will also share experiences in creating a platform based on Akka and Akka Streams that currently processes over 1 billion transactions per day (on just 8 VMs), with the aim of helping teams adopt these technologies. In this webinar, you will:
- Start with a sample web crawler use case to examine what happens when each processing pass expands to a larger and larger workload to process.
- Review how we use the buffering capabilities in Kafka and the back-pressure with asynchronous processing in Akka Streams to handle such bursts.
- Look at lessons learned, plus some constructive “rants” about the architectural components, the maturity, or immaturity you’ll expect, and tidbits and open source goodies like memory-mapped stream buffers that can be helpful in other Akka Streams and/or Kafka use cases.
