Open Joe-Heffer-Shef opened 3 years ago
Enable multi-threading to sync in parallel and increase throughput?
For Awesome data bridge: run AQI and data in separate processes? so they both sync in near real-time
Use an ETL framework (e.g. Apache Airflow) so all harvesters can use concurrency
Enable multi-threading to sync in parallel and increase throughput?
For Awesome data bridge: run AQI and data in separate processes? so they both sync in near real-time