bennahugo
commented
2 years ago @Athanaseus #fd3af5d passes tests locally - the CI server should confirm. The only issue that I can currently see is that /usr/bin/python needs to be symlinked (as it is in a virtualenv) to python3. Everything installs and works inside a venv, but not quite on system level without additional trickery. Unfortunately I don't know where the issue stems from at the moment -- I have tried to eliminate 'python' from the Meqtrees universe for the past while and updated the hashbangs but still can't track this one down.
@o-smirnov perhaps you can provide some further thoughts here as this is originally your codebase: This is what I get without the additional symlink
Using all interferometers
Making dirty image: lwimager ms=/usr/local/lib/python3.8/dist-packages/Pyxis/recipes/meqtrees-batch-test/WSRT.MS data=DATA operation=image stokes=I mode=channel weight=natural wprojplanes=0 npix=512 cellsize=1arcsec spwid=0 field=0 padding=1.2 cachesize=512 prefervelocity=False robust=0 image=tmp.img
lwimager: Version 1.4.0
2022-01-26 20:41:47 INFO Imager::open() Opening MeasurementSet /usr/local/lib/python3.8/dist-packages/Pyxis/recipes/meqtrees-batch-test/WSRT.MS
2022-01-26 20:41:47 INFO imager::data selection Selecting data
2022-01-26 20:41:47 INFO imager::data selection Selected 6552 out of 7560 visibilities.
2022-01-26 20:41:47 INFO Imager::selectDataChannel() Selecting channel 0 for spw 0
2022-01-26 20:41:47 INFO imager::defineimage() Defining image properties:nx=512 ny=512 cellx='1arcsec' celly='1arcsec' stokes=I' mode=channel nchan=1 start=0 step=1 spwids=[0] fieldid=0 facets=1 frame=1 distance='0m'
2022-01-26 20:41:47 INFO imager::defineimage() phaseCenter='field-0 ' mStart='Radialvelocity: 0' qStep='1 km/s'' mFreqStart='Frequency: 0
2022-01-26 20:41:47 INFO imager::weight() Weighting MS: Imaging weights will be changed
2022-01-26 20:41:47 INFO imager::weight() Natural weighting
2022-01-26 20:41:47 INFO imager::setoptions() Setting processing options
2022-01-26 20:41:47 INFO imager::makeimage() Calculating image (without full skyequation)
2022-01-26 20:41:47 INFO imager::makeimage() Making dirty image from observed data
2022-01-26 20:41:47 INFO imager::makeimage() Image is : tmp.img
2022-01-26 20:41:48 INFO imager::createFTMachine() Performing interferometric gridding...
lwimager normally ended
real 0m0.489s
user 0m0.201s
sys 0m0.051s
Success, all *.img files will be removed after conversion to FITS
image2fits: Version 20090915GvD
The input image is: tmp.img*2
2022-01-26 20:41:48 INFO ImageFitsConverter::ImageToFITS Copying 'Expression: tmp.img*2' to file All pixels fit in memory (262144 pixels).
image2fits normally ended
E
======================================================================
ERROR: batch_test.testMeqtreesBatchJob
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/local/lib/python3.8/dist-packages/nose/case.py", line 198, in runTest
self.test(*self.arg)
File "/src/pyxis-1.7.4.3/Pyxis/recipes/meqtrees-batch-test/batch_test.py", line 161, in testMeqtreesBatchJob
if not os.path.exists(os.path.join(PACKAGE_TEST_DIR, "WSRT.MS.MODEL_DATA.channel.1ch.fits")): raise RuntimeError("Output FITS file does not exist")
RuntimeError: Output FITS file does not existand this is with the symlink
The input image is: tmp.img*2
2022-01-26 20:47:21 INFO ImageFitsConverter::ImageToFITS Copying 'Expression: tmp.img*2' to file All pixels fit in memory (262144 pixels).
image2fits normally ended
lwimager: Version 1.4.0
2022-01-26 20:47:23 INFO Imager::open() Opening MeasurementSet /usr/local/lib/python3.8/dist-packages/Pyxis/recipes/meqtrees-batch-test/WSRT.MS
2022-01-26 20:47:23 INFO imager::data selection Selecting data
2022-01-26 20:47:23 INFO imager::data selection Selected 6552 out of 7560 visibilities.
2022-01-26 20:47:23 INFO Imager::selectDataChannel() Selecting 64 channels, starting at 0, stepped by 1, for spw 0
2022-01-26 20:47:23 INFO imager::defineimage() Defining image properties:nx=1024 ny=1024 cellx='14.0625arcsec' celly='14.0625arcsec' stokes=IQUV' mode=channel nchan=1 start=0 step=64 spwids=[0] fieldid=0 facets=1 frame=1 distance='0m'
2022-01-26 20:47:23 INFO imager::defineimage() phaseCenter='field-0 ' mStart='Radialvelocity: 0' qStep='1 km/s'' mFreqStart='Frequency: 0
2022-01-26 20:47:23 INFO imager::weight() Weighting MS: Imaging weights will be changed
2022-01-26 20:47:23 INFO imager::weight() Radial weighting
2022-01-26 20:47:23 INFO imager::filter() Imaging weights will be tapered
2022-01-26 20:47:23 INFO VisImagingWeight::setFilter() Filtering for Gaussian of shape: 12 by 12 (arcsec) at p.a. 0 (degrees)
2022-01-26 20:47:23 INFO imager::setoptions() Setting processing options
2022-01-26 20:47:23 INFO imager::makeimage() Calculating image (without full skyequation)
2022-01-26 20:47:23 INFO imager::makeimage() Making dirty image from model data
2022-01-26 20:47:23 INFO imager::makeimage() Image is : /usr/local/lib/python3.8/dist-packages/Pyxis/recipes/meqtrees-batch-test/WSRT.MS.MODEL_DATA.channel.1ch.img
2022-01-26 20:47:23 INFO imager::createFTMachine() Performing interferometric gridding...
2022-01-26 20:47:24 INFO ImageFitsConverter::ImageToFITS Copying '/usr/local/lib/python3.8/dist-packages/Pyxis/recipes/meqtrees-batch-test/WSRT.MS.MODEL_DATA.channel.1ch.img' to file Copying plane by plane (1048576 pixels).
lwimager normally endedSo the second lwimager call does not execute... I'm a bit stumped to tracking this one down
@Athanaseus this builds against the current casacore, but we are missing those executables I mentioned on the KERN ticket. I'm committing these as a WIP pending passing the test. Hopefully we can simplify the docker to test the kern 7 debs once we found the issue with the kern casarest package