search

pentium3 / sys_reading

system paper reading notes
235 stars 12 forks source link

Sonata: Query-Driven Streaming Network Telemetry #127

Closed pentium3 closed 3 years ago

pentium3 commented 3 years ago

https://mcanini.github.io/papers/sonata.sigcomm18.pdf

https://github.com/Sonata-Princeton/SONATA-DEV/blob/tutorial/sonata/tutorials/background.md

https://conferences.sigcomm.org/sigcomm/2018/program.html

pentium3 commented 3 years ago

do dataflow stream processing on programmable switch, for network monitoring

idea: use both switch(high ops but less resource) and CPUs(more resource but low ops)

map dataflow to data plane:

query planner: decide which part of the tasks/queries to be run on CPU/switch

resource constrains on switch: header size/stateful memory/total stages/num of actions (limited by available resources)

use ILP(an optimizer running on CPU) to generate query plan that could minimize tuples sent to stream processor(CPU. eg, spark) [so maximize usage of switch], while not exceeding the resource limitation on switch

ILP only run one time to generate the plan. (which type of packet will be sent to where?)

when a new packet arrived at switch, switch will determine whether it will be processed on CPU or switch, based on packet headers and the generated plan.

special optimization for network monitoring workload: ch4