search

desihub / desimeter

DESI coordinates and transformations
BSD 3-Clause "New" or "Revised" License
2 stars 4 forks source link

make posparam fits more atomic #68

Closed joesilber closed 4 years ago

joesilber commented 4 years ago

This branch addresses issue #64

The positioner parameter fitting is handled for each positioner in a new module fithandler.py.

The multiprocessing batches are much simpler to manage: for each positioner, shoot a data table over to the fithandler module and let it run.

Long jobs are more robust as well. For each positioner, a separate data table is written as soon as it has been processed. This is vastly more memory efficient than before (running all 5020 tables in memory at once) and a crash doesn't lose all the work.

julienguy commented 4 years ago

Working for me. Except the script plot_posparams is now deprecated, I think.

sbailey commented 4 years ago

Structurally this looks good. Thanks for these updates. It would be nice if @julienguy could confirm that it still works for him prior to merging to catch numpy version etc. stuff, but if merging this is a blocking factor for @joesilber it's fine with me to merge and patch up any compatibility issues post-facto.

joesilber commented 4 years ago

Except the script plot_posparams is now deprecated, I think.

That script should work

joesilber commented 4 years ago

Structurally this looks good. Thanks for these updates. It would be nice if @julienguy could confirm that it still works for him prior to merging to catch numpy version etc. stuff, but if merging this is a blocking factor for @joesilber it's fine with me to merge and patch up any compatibility issues post-facto.

It's not blocking me, I can start another branch as needed.