extraction beamtimes issue

[twshimwell]

Hey,

Where running the extract pipeline with the new software I get the error below.

• 11:44:02 - ClassJones [4.7/7.2 7.9/10.5 39.8Gb] Loading solution file /net/lofar8/data2/shimwell/extraction/workingdir/ACT-CLJ2333.1+0349/P353+06/SOLSDIR/L765101_SB001_uv_12F24A405t_121MHz.pre-cal.ms/killMS.DDS3_full_smoothed.sols.npz
• 11:44:07 - ClassJones [5.7/7.2 8.9/10.5 40.0Gb] Getting Jones frequency domains from solutions file
• 11:44:08 - AsyncProcessPool [5.7/7.2 8.9/10.5 40.0Gb] process io00: exception raised processing job DATA:0:0: Traceback (most recent call last): File "/usr/local/src/DDFacet/DDFacet/Other/AsyncProcessPool.py", line 873, in _dispatch_job result = call(*args, **kwargs) File "/usr/local/src/DDFacet/DDFacet/Data/ClassVisServer.py", line 615, in _handler_LoadVisChunk JonesMachine.InitDDESols(DATA) File "/usr/local/src/DDFacet/DDFacet/Data/ClassJones.py", line 180, in InitDDESols DicoSols, TimeMapping, DicoClusterDirs = self.MakeSols("killMS", DATA, quiet=quiet) File "/usr/local/src/DDFacet/DDFacet/Data/ClassJones.py", line 322, in MakeSols DicoClusterDirs_killMS, DicoSols = self.GiveKillMSSols() File "/usr/local/src/DDFacet/DDFacet/Data/ClassJones.py", line 506, in GiveKillMSSols DicoClusterDirs, DicoSols, VisToJonesChanMapping = self.GiveKillMSSols_SingleFile( File "/usr/local/src/DDFacet/DDFacet/Data/ClassJones.py", line 843, in GiveKillMSSols_SingleFile VisToJonesChanMapping,DicoClusterDirs,DicoSols,G=self.ReadNPZ(SolsFile) File "/usr/local/src/DDFacet/DDFacet/Data/ClassJones.py", line 585, in ReadNPZ self.BeamTimes_kMS = (DicoSolsFile["BeamTimes"]["t0"]+DicoSolsFile["BeamTimes"]["t1"])/2 IndexError: only integers, slices (:), ellipsis (...), numpy.newaxis (None) and integer or boolean arrays are valid indices

[mhardcastle]

Yes, we discovered this problem a little while ago. The old software still works. I assume it's incompatibility in the solutions files....

[twshimwell]

Yeh indeed, old software and new pipeline does the trick nicely. Those changes you put in to deal with both new and old software are very nice!

[mhardcastle]

I'm leaving this open with the 'new version' flag because when we start using new DDF versions we're going to have to put a fix in. (This is I think the same thing that Cyril had to fix with the DynSpecMS reprocessing.)