Switching `extract` and `most_likely_path` to software-defined assets

observatoire-mobilite / odmkraken

The **kraken** is the orchestration layer responsible for gathering and postprocessing ODM's mobility data

MIT License

0 stars 0 forks source link

Switching `extract` and `most_likely_path` to software-defined assets #73

Open gilgeorges opened 1 year ago

gilgeorges commented 1 year ago

Instead of describing the pipeline in terms of sequential ops, this switches to assets - at least for the directly ping-related datasets (i.e. excluding the network). This allows using partitioning throughout the system.

gilgeorges commented 1 year ago

After a lot of tinkering, the live tests seem to run through - although the most_likely_path step is only seen starting, not completing. Not sure what is going on...