search

dirac-institute / kbmod

KBMOD (Kernel-Based Moving Object Detection)
BSD 2-Clause "Simplified" License
46 stars 14 forks source link

Need feedback/progress updates to user during reproject, etc. #641

Closed ColinOrionChandler closed 3 months ago

ColinOrionChandler commented 5 months ago

Ideally, an indicator of the time remaining too, but at the very least letting the user see what is happening during the course of multi-hour operations like constructing work units.

vlnistor commented 4 months ago

I'd like to take this, but wanted to clarify:

  1. When you say constructing work units is a multi-hour operation, do you mean it takes hours to construct work units from fits files ? If so, I think we can provide feedback on the number of images loaded from the fits file during the loading phase.
  2. In terms of feedback for reprojection, an easy implementation can just display Image i / total done. We can use \r character to overwrite the previous line so we avoid having total number of rows printed. I can't think of a good way to estimate the time, but if my proposed solution is good enough we can start with that.
maxwest-uw commented 4 months ago
  1. I think this refers to the creation of a WorkUnit from an ImageCollection, although it might be good to updates to all of the phases of reprojection including loading in a WorkUnit from disk/FITs files (which takes a couple of minutes).
  2. In general for other lincc-frameworks projects we tend to use tqdm for tracking progress because it works pretty well out of the box. There are ways to configure it with multiprocessing futures as well for the parallel version of reproject.
vlnistor commented 4 months ago

Thank you for those clarifications - I'm looking at discrete_piles_e2e.ipynb to see an example of creating a WorkUnit from an ImageCollection. I installed the lsst package from here (pip install git+https://github.com/lsst/daf_butler.git). However, I am missing the test data specified in REPO path in the notebook. Is there some test data you can share ? Otherwise, I can try to use the MockButler

maxwest-uw commented 4 months ago

I think you shouldn't have to worry about installing the LSST stack to do development on this portion of KBMOD! If you just want to test against a basic WorkUnit we have one we use for testing included in the test directory (this one, it's also the one we run against for the reprojection unit tests). If we wanted to run tests against actual data we could try doing some runs with the data we have our HPC clusters, but I think the unit test one would be a good place to start !

maxwest-uw commented 4 months ago

^ this is just in relation to reprojection, thinking about it you might need the stack to work with ImageCollection generation, but I might need to defer to another dev about working with that

vlnistor commented 4 months ago

@maxwest-uw Made a PR with a start on this. Some points to confirm but let's address them there

maxwest-uw commented 4 months ago

Need to add the TQDM loop to a few of the ImageCollection and WorkUnit functions so that we can track the progress of various large tasks in the reproject/KBMOD search pipeline.

Important requirement: needs to interact nicely with output files on HPCs like klone.

Required Changes

tqdm should be added to the following places in the code:

Testing