rachel3834
commented
5 months ago To evaluate fit_need_events_PSPL's current code, I added checkpoints throughout the management command which measured the time taken and the number of DB connections open at any time.
The code starts by making a query for all Alive targets which have a Last_fit date earlier than a given threshold. Checkpoints before (1) and after (2) this query record: N DB connectioned 1: 1 N DB connectioned 2: 1 Time for targets query: 0:00:00.004373 Target(s) remaining in processing queue: 4118
This executes a multi-criteria query on the TargetExtra table but it seems to be efficient in both DB connections and time.
Next, the code selects just the first target, and queries the DB for associated ReducedDatums. This query is actually run twice - once for photometry and once for the model lightcurve. After which the checkpoint reports: N DB connectioned 3: 6 Time for ReducedDatums query for 1 target: 0:00:00.190659
So still efficient in time, but now we have multiple DB connections open.
- [x] This could be improved by using the output from the first query to extract the model lightcurve as well. The target in question (Gaia23aiy) has both Gaia ReducedDatums and those for uploaded photometry and the model lightcurve.
The code then updates the Last_fit timestamp, generating more connections:
N DB connectioned 4: 12 Time for storing Last_fit timestamp: 0:00:00.010926
At this point, the code calls the run_fit function for the selected single target. Step 1 is to check whether it is a Gaia target, in which case it pulls the ReducedDatums from the DB again, updates the photometric uncertainties and saves them again.
Then it checks whether the target is classified as microlensing; if it isn't the Alive status is set to False and stored as an extra_field. After all this, checkpoint output is:
N DB connections 1: 20
- [x] It's not clear to me why the Gaia uncertainties are evaluated here rather than during ingest at the Gaia broker - so this call can be moved from the fitting code. I suspect that this may once have been a necessary update but is legacy code at this point.
- [x] Checking for microlensing as the classification should be a filter in the original query for targets, rather than a second query.
As the code loops through the list of targets, it seems to accumulate DB connections, even if no further processing of the target is done. In the case of Gaia23aiy, it appears to go into an infinite loop, because (as it happens), it had a non-microlensing classification, so the run_fit function returned to main without further output and returning no result.
The main function does not check the output, or remove the target from the list of targets to consider. This appears to cause it to remain in the target list, because the next set of output refers to the same object at the top of the list:
Target(s) remaining in processing queue: 4118 N DB connectioned 3: 25 Time for targets query: 0:00:00.166363 Target post save hook: Gaia23aiy created: False N DB connectioned 4: 31
With more DB connections accumulating.
- [x] run_fit should return a definitive result in all cases.
- [x] fit_need_events_PSPL should evaluate the result of run_fit, and remove a target from the list of events to process.
KKruszynska
From the LCO Slack discussion focusing around fit_all_events_PSPL.py (actually disabled but similar in structure to fit_need_events_PSPL.py):
Proposed solutions after discussions with the TOM Team: