Closed landmanbester closed 3 years ago
Not sure this is an issue with shadems per se but I have gotten myself into a situation where I get the following error
2021-01-20 17:08:03 - shadems - INFO - using colourmap colorcet.bkr 2021-01-20 17:08:03 - shadems - INFO - using colourmap cmasher.pride 2021-01-20 17:08:03 - shadems - INFO - using colourmap colorcet.glasbey_dark 2021-01-20 17:08:03 - shadems - INFO - /home/bester/venvs/shadems/bin/shadems --xaxis FREQ --yaxis UNCORRECTED_DATA:amp:XX --dir udatacol 1557347448_sdp_l0-cal.ms/ 2021-01-20 17:08:03 - shadems - INFO - ------------------------------------------------------ 2021-01-20 17:08:03 - shadems - INFO - : MS 1557347448_sdp_l0-cal.ms contains 708354 rows 2021-01-20 17:08:03 - shadems - INFO - : (1, 4096) spectral windows and channels 2021-01-20 17:08:03 - shadems - INFO - : 1 fields: J1939-6342 2021-01-20 17:08:03 - shadems - INFO - : 8 scans: 1 3 5 7 9 11 13 15 2021-01-20 17:08:04 - shadems - INFO - : 58/58 antennas: 0:m000 1:m001 2:m002 3:m003 4:m004 5:m005 6:m006 7:m007 8:m008 9:m009 10:m010 11:m011 12:m012 13:m013 14:m015 15:m016 16:m018 17:m019 18:m020 19:m021 20:m022 21:m023 22:m024 23:m025 24:m026 25:m027 26:m028 27:m029 28:m030 29:m031 30:m033 31:m034 32:m035 33:m036 34:m037 35:m038 36:m039 37:m040 38:m041 39:m042 40:m043 41:m044 42:m047 43:m048 44:m049 45:m050 46:m051 47:m052 48:m053 49:m054 50:m055 51:m056 52:m057 53:m058 54:m060 55:m061 56:m062 57:m063 2021-01-20 17:08:04 - shadems - INFO - : 1711/1711 baselines present 2021-01-20 17:08:04 - shadems - INFO - : corrs/Stokes XX XY YX YY I Q U V 2021-01-20 17:08:04 - shadems - INFO - ------------------------------------------------------ 2021-01-20 17:08:04 - shadems - INFO - : Data selected for plotting: 2021-01-20 17:08:04 - shadems - INFO - Antenna(s) : all 2021-01-20 17:08:04 - shadems - INFO - Baseline(s) : all except autocorrelations 2021-01-20 17:08:04 - shadems - INFO - Field(s) : all 2021-01-20 17:08:04 - shadems - INFO - SPW(s) : all 2021-01-20 17:08:04 - shadems - INFO - Scan(s) : all 2021-01-20 17:08:04 - shadems - INFO - Channels : all 2021-01-20 17:08:04 - shadems - INFO - Corr/Stokes : XX XY YX YY 2021-01-20 17:08:04 - shadems - INFO - ------------------------------------------------------ 2021-01-20 17:08:04 - shadems - INFO - loading minmax cache from udatacol/1557347448_sdp_l0-cal-minmax-cache.json 2021-01-20 17:08:04 - shadems - INFO - axis: FREQ, range (None, None), discretization None 2021-01-20 17:08:04 - shadems - INFO - axis: amp(UNCORRECTED_DATA), corr 0, range (None, None), discretization None 2021-01-20 17:08:04 - shadems - INFO - : you have asked for 1 plots employing 2 unique datums 2021-01-20 17:08:04 - shadems - INFO - : Indexing MS and building dataframes (684342 rows, chunk size is 5000) Traceback (most recent call last): File "/home/bester/venvs/shadems/bin/shadems", line 7, in <module> exec(compile(f.read(), __file__, 'exec')) File "/home/bester/shadeMS/bin/shadems", line 8, in <module> main.main([a for a in sys.argv[1:]]) File "/home/bester/shadeMS/shade_ms/main.py", line 418, in main row_chunk_size=options.row_chunk_size) File "/home/bester/shadeMS/shade_ms/data_plots.py", line 191, in get_plot_data value = axis.get_value(group, corr, extras, flag=flag, flag_row=flag_row, chanslice=chanslice) File "/home/bester/shadeMS/shade_ms/data_mappers.py", line 383, in get_value return dama.masked_array(coldata, da.logical_or(flag, bad_data)) File "/home/bester/venvs/shadems/lib/python3.6/site-packages/dask/array/ma.py", line 229, in masked_array "%s." % (repr(data.shape), repr(mask.shape)) numpy.ma.core.MaskError: Mask and data not compatible: data shape is (684342, 4096), and mask shape is (684342, 4096, 4096).
UNCORRECTED_DATA is a column I have written to the measurement set myself using
https://github.com/ratt-ru/pfb-clean/blob/add_gains/scripts/kgb2col.py
Strangely, it seems to have the same shape as DATA and CORRECTED_DATA both of which plot just fine. Any idea what coul dbe going wrong here?
I think this is the same as https://github.com/ratt-ru/shadeMS/issues/64
Ah, thanks @IanHeywood. This is a duplicate
Not sure this is an issue with shadems per se but I have gotten myself into a situation where I get the following error
UNCORRECTED_DATA is a column I have written to the measurement set myself using
https://github.com/ratt-ru/pfb-clean/blob/add_gains/scripts/kgb2col.py
Strangely, it seems to have the same shape as DATA and CORRECTED_DATA both of which plot just fine. Any idea what coul dbe going wrong here?