ncclementi
commented
6 months ago Closing since it was delivered
Closed chloeh13q closed 6 months ago
ncclementi
commented
6 months ago Closing since it was delivered
We're giving a talk at the LinkedIn stream processing meetup.
Full agenda
5:30 - 6:00: Networking [in-person only + catered food]
6:00 - 6:05: Welcome
6:05 - 6:40: Unified Streaming Framework - A Declarative Real-Time Streaming Framework for Flink & Spark Basar Onat & Chen Yang, DoorDash In DoorDash, we have various streaming event processing frameworks that are used. For the end users, this split slows down their velocity and they have to learn about nuances of each framework. This caused us to unify the frameworks under one architecture with a declarative framework that can be applicable to both frameworks and pick the right one based on the user requirements and SLAs.
6:40 - 7:15: Streaming Queries without Compromise Mihai Budiu & Leonid Ryzhyk, Feldera Modern databases excel at the task of data analysis when data changes infrequently. However, for rapidly changing data many custom systems have been built under the guise of streaming systems. To offer near-real time answers, streaming systems compromise on the expressivity of computations they can perform. We argue that this compromise is unnecessary. Based on a new theoretical foundation we built Feldera, a streaming query engine which can execute any traditional database computation in streaming mode. The core of our system is an algorithm that converts an arbitrary query on data tables into a query that computes on change streams. We describe the core ideas behind this technology and give a demo of the system.
7:15 - 7:50: Ibis: The Portable Python Dataframe API Phillip Cloud & Chloe He, Voltron Data Ibis is a lightweight Python library that helps you rapidly develop analytics, from development to production, using a dataframe API. You can pull a sample of a dataset from the production system, work with it locally and then swap out connection information to run that same code in production. Ibis is also great as part of a larger pipelining system: think Kedro, dbt and sqlmesh. Recently, Ibis gained awesome streaming backends that we're excited to present. We'll give a technical overview, show it in action, and time permitting we can talk about what the future looks like for Ibis.
Link for registraion
https://www.linkedin.com/events/in-person-online-streamprocessi7174522514277040128/about/