numaproj / numaflow-python

Numaflow Python SDK
Apache License 2.0
53 stars 18 forks source link

feat: examples for asyncio with ThreadPoolExecutor and ProcessPoolExecutor #54

Closed kohlisid closed 1 year ago

kohlisid commented 1 year ago

Closes #52

Adds examples for using ThreadPool executor and ProcessPool executor with asyncio. Additionally has an example for just writing an async handler as well.

When executing operations that dont support asyncio and can be blocking in nature such as network requests or any CPU heavy computations, its good to delegate them to ThreadPoolExecutor or a ProcessPoolExecutor.

codecov[bot] commented 1 year ago

Codecov Report

Merging #54 (d76f366) into asyncio-examples (5268fd2) will not change coverage. The diff coverage is n/a.

:exclamation: Current head d76f366 differs from pull request most recent head 6fb6b57. Consider uploading reports for the commit 6fb6b57 to get more accurate results

@@                Coverage Diff                @@
##           asyncio-examples      #54   +/-   ##
=================================================
  Coverage             96.20%   96.20%           
=================================================
  Files                    10       10           
  Lines                   343      343           
  Branches                 26       26           
=================================================
  Hits                    330      330           
  Misses                    8        8           
  Partials                  5        5           

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

vigith commented 1 year ago

can you add the performance details etc in the README.md? what to use, when to use etc?

vigith commented 1 year ago

moved to #60