search

sbgrid / data-capture-module

5 stars 4 forks source link

eliminate cron dependencies #4

Closed pameyer closed 7 years ago

pameyer commented 8 years ago

dataverse has impedance mismatch when interacting with systems that depend on cron. needs re-factoring to eliminate this dependency, at least at the generation step. cron dependency for checksum validation less of a UI issue (there's already an async out-of-browser step); would also require more backend and transfer script changes.

pdurbin commented 8 years ago

As a workaround, I ran count=0; while true; echo $((++count)); do /usr/local/dcm/gen/usg-wrap.bash; sleep 2; done on the DCM so it would cook me an rsync script fast enough that it wasn't painful for me to work on integration (API) tests as I worked on https://github.com/IQSS/dataverse/issues/3352

Perhaps it's fine to use cron in a variety of places in the system but one place where I would really prefer for the user not to have to wait is when they are creating a dataset. I believe it leads to a much better user experience if they are able to download the rsync script right away after creating a dataset rather than waiting for cron to run.

pameyer commented 7 years ago

There's still a glitch in the ansible/rq setup (https://github.com/sbgrid/data-capture-module/issues/13); but implementation for removing this worked w\ manual install.

Doesn't require API endpoint changes in the DCM.