DigitalSlideArchive / import-tracker

A Girder plugin for data import tracking in HistomicsUI
Other
2 stars 2 forks source link

Add Job and ProgressContext tracking to /folder/{id}/move route #27

Closed willdunklin closed 8 months ago

manthey commented 9 months ago

Two points:

As long as we are updating the job, with information, we should do something like Job().updateJob(job, log='Moving %s in folder %s\n' % (file['name'], folder['name']), updateMessage=<same as log message>) for each file (and each attached file with possibly a different message). Otherwise on a big move, the job shows that it is running but there is no way to tell how much has been done.

Looking at this, I originally missed seeing that we are only transferring the attached files on one item per folder (and not attached files on the folders). We should do this for each item in a folder AND for the folder itself.

willdunklin commented 8 months ago

@manthey I just added a commit that should hopefully resolve most of the job formatting issues you mentioned.

The one thing I'm still working on is the nuance with moving the attached files in the way you'd mentioned. It's been a while since we spoke about the exact problem and my memory has yet to be fully jogged after reading your written feedback here. Let me know if this is the gist of what you were getting at:

In this section of the code I'm moving all files attached to the given folder we're examining. You mentioned this doesn't catch certain patterns of attached files(?). If so, what is the pattern I should be looking to cover? Was it that other items could have attached files and that I needed to check for those? If there's any documentation you know of regarding these rules for attached files, please let me know as well! I've searched across the docs but they didn't yield the info quickly enough to warrant blindly looking much deeper.

willdunklin commented 8 months ago

@manthey I think those issues should be resolved now. One odd thing I've noticed about this code is that in the job tracker, the logs for each job seem to not persist on refresh. For example I'll run a folder_move and see the job log output while the task is executing image

However, after the job is complete & I refresh the page, the log is no longer present image

Have you seen this pattern before? I'm kind of perplexed why this isn't being saved persistently.

willdunklin commented 8 months ago

@manthey I think everything is resolved here. Unless you have any further comments, are we good to merge?