search

EranOfek / AstroPack

Astronomy & Astrophysics Software Pacakge
Other
17 stars 4 forks source link

Merged source catalogs missing in 1-3 visits each night in marvin; SMS.populateAllConsecutive fails #427

Closed param-rekhi closed 6 months ago

param-rekhi commented 6 months ago

The visit level merged source catalog files (LAST*MergedMat*.hdf5) are missing in 1-3 visits almost every night. This causes the SearchMatchedSources.populateAllConsecutive function to fail.

param-rekhi commented 6 months ago

Mount 1 has this issue on most nights. Other mounts don't, but even 1 problematic visit in the entire directory structure causes it to fail. Furthermore, I can't restrict the input directory to a specific night or month - it has to be LAST.01.xx.0x

EranOfek commented 6 months ago

please provide a full example - what command you run, from which dir, full error message, and the name of the visit on which it failed.

param-rekhi commented 6 months ago

Examples: 1.

>> SMS.populateAllConsecutive('BasePath','/marvin/LAST.01.01.01')
Index exceeds the number of array elements. Index must not exceed 0.

Error in pipeline.DemonLAST/prepListOfProcVisits (line 1200)
                            List(Ind).FieldID = FN.FieldID{1};

Error in pipeline.last.SearchMatchedSources/populateAllConsecutive (line 67)
                List = D.prepListOfProcVisits;`

Failed at /marvin/LAST.01.01.01/2023/12/15/proc/183744v0

2.

>> SMS.populateAllConsecutive('BasePath','/marvin/LAST.01.04.03')
Index exceeds the number of array elements. Index must not exceed 0.

Error in pipeline.DemonLAST/prepListOfProcVisits (line 1200)
                            List(Ind).FieldID = FN.FieldID{1};

Error in pipeline.last.SearchMatchedSources/populateAllConsecutive (line 67)
                List = D.prepListOfProcVisits;

Failed at /marvin/LAST.01.04.03/2024/02/26/proc/012452v0

  1. Doesn't accept sub-directories as the BasePath argument: 
    
>> SMS.populateAllConsecutive('BasePath','/marvin/LAST.01.04.03/2024/03')
Output argument "List" (and possibly others) not assigned a value in the execution with "pipeline/DemonLAST/prepListOfProcVisits" function.

Error in pipeline.last.SearchMatchedSources/populateAllConsecutive (line 67) List = D.prepListOfProcVisits;

EranOfek commented 6 months ago

Can not reproduce the error - what version are you using?

param-rekhi commented 6 months ago

v2.0-147-g20cd1988

EranOfek commented 6 months ago

You should work with dev1

param-rekhi commented 6 months ago

I was working on dev1, but I think it was early last weeks commit. I git pulled the latest version (v2.0-187-g70b5fff8) and it works.