WHOIGit / ifcbdb

IFCB dashboard
MIT License
8 stars 10 forks source link

Starting sync automatically? #419

Open jonatanmindroad opened 2 weeks ago

jonatanmindroad commented 2 weeks ago

Is there a preferred way to sync datasets automatically? The use case would be for the dashboard to handle data that is continuously being uploaded.

I am considering extending the django container with a cronjob that calls python manage.py syncdataset periodically. But perhaps there is another obvious way to do this.

Any ideas?

joefutrelle commented 2 weeks ago

The sync_bin endpoint is used in this example script for periodically transferring data from an IFCB and syncing the bins to the dashboard.

https://github.com/WHOIGit/ifcbdb/blob/master/utilities/auto_transfer.py

jonatanmindroad commented 1 day ago

I looked into the included script, but in the end I went with my original idea.

If anyone else want to do the same I can give some notes. It turned out that cronjobs inside containers has its own challenges.

TomAkaTma commented 14 hours ago

Hi @jonatanmindroad , any notes would be greatly appreciated - I'm about to try and do exactly the same thing. Thanks!