Open abij opened 5 years ago
Now there is a download-category.py which downloads the files locally. And an put-catagories.py to upload in ElasticSearch.
download-category.py
put-catagories.py
These can be combined in a nice Airflow DAG that is responsible for a certain time-period. For example: images from 20-sept for a given category.
This could run for multiple days in the past.
Now there is a
download-category.pywhich downloads the files locally. And anput-catagories.pyto upload in ElasticSearch.These can be combined in a nice Airflow DAG that is responsible for a certain time-period. For example: images from 20-sept for a given category.
This could run for multiple days in the past.