Story discovery engine for the Counterdata Network. Grabs relevant stories from various APIs, runs them against bespoke classifier models, post results to a central server.
We are using Prefect for parallelizing and managing the dataflow pipeline (with Dask under the hood). They have launched a re-architected v2.x version, which requires some migration work to update our codebase, which uses the v1.x line of code. This migration guide will probably be useful for this task.
rahulbot
commented
1 year ago
We are using Prefect for parallelizing and managing the dataflow pipeline (with Dask under the hood). They have launched a re-architected v2.x version, which requires some migration work to update our codebase, which uses the v1.x line of code. This migration guide will probably be useful for this task.