ratt-ru / pfb-imaging

Preconditioned forward/backward clean algorithm
MIT License
6 stars 5 forks source link

MS file with one channel #65

Closed masoumehghn closed 1 year ago

masoumehghn commented 1 year ago

Hi, I produced MS empty file with ONE channel for some checking purpose. I have following error when I do 'init':

INFO 11:19:24 - INIT | Using weights from WEIGHT column Traceback (most recent call last): File "/Users/masoumeh/opt/anaconda3/envs/landman/bin/pfb", line 33, in sys.exit(load_entry_point('pfb-clean', 'console_scripts', 'pfb')()) File "/Users/masoumeh/opt/anaconda3/envs/landman/lib/python3.8/site-packages/click/core.py", line 1130, in call return self.main(args, kwargs) File "/Users/masoumeh/opt/anaconda3/envs/landman/lib/python3.8/site-packages/click/core.py", line 1055, in main rv = self.invoke(ctx) File "/Users/masoumeh/opt/anaconda3/envs/landman/lib/python3.8/site-packages/click/core.py", line 1657, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/Users/masoumeh/opt/anaconda3/envs/landman/lib/python3.8/site-packages/click/core.py", line 1404, in invoke return ctx.invoke(self.callback, ctx.params) File "/Users/masoumeh/opt/anaconda3/envs/landman/lib/python3.8/site-packages/click/core.py", line 760, in invoke return __callback(args, kwargs) File "/Users/masoumeh/Desktop/AGN_simulation/landman/git/pfb-clean/pfb/workers/init.py", line 96, in init return _init(opts) File "/Users/masoumeh/Desktop/AGN_simulation/landman/git/pfb-clean/pfb/workers/init.py", line 378, in _init chan_width=chan_width[Inu], File "/Users/masoumeh/opt/anaconda3/envs/landman/lib/python3.8/site-packages/dask/array/core.py", line 1962, in getitem index2 = normalize_index(index, self.shape) File "/Users/masoumeh/opt/anaconda3/envs/landman/lib/python3.8/site-packages/dask/array/slicing.py", line 904, in normalize_index raise IndexError("Too many indices for array") IndexError: Too many indices for array

landmanbester commented 1 year ago

It is very likely that monochromatic imaging does not currently work because I have never tried it myself. I'll see if I can add a fix later today

landmanbester commented 1 year ago

@masoumehghn I have merged a number of changes (in particular support to predict from fits and for single channel measurement sets) into the master branch. It should also work with the latest stimela master branch in case you want to get started building recipes. Please let me know if it works for you so we can close this one

masoumehghn commented 1 year ago

@landmanbester Thanks. Unfortunately, I have the same error again: IndexError: Too many indices for array

landmanbester commented 1 year ago

That is strange. I simulated a single channel MS and it works for me. Have you pulled the latest master branch and reinstalled the package? If so, please can you post the exact command that is failing for you and point me at the full log file?

masoumehghn commented 1 year ago

@landmanbester I think I have pulled the latest master branch and reinstalled the package.

Traceback (most recent call last): File "/Users/masoumeh/opt/anaconda3/envs/landman/bin/pfb", line 33, in sys.exit(load_entry_point('pfb-clean', 'console_scripts', 'pfb')()) File "/Users/masoumeh/opt/anaconda3/envs/landman/lib/python3.8/site-packages/click/core.py", line 1130, in call return self.main(args, kwargs) File "/Users/masoumeh/opt/anaconda3/envs/landman/lib/python3.8/site-packages/click/core.py", line 1055, in main rv = self.invoke(ctx) File "/Users/masoumeh/opt/anaconda3/envs/landman/lib/python3.8/site-packages/click/core.py", line 1657, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/Users/masoumeh/opt/anaconda3/envs/landman/lib/python3.8/site-packages/click/core.py", line 1404, in invoke return ctx.invoke(self.callback, ctx.params) File "/Users/masoumeh/opt/anaconda3/envs/landman/lib/python3.8/site-packages/click/core.py", line 760, in invoke return __callback(args, kwargs) File "/Users/masoumeh/Desktop/AGN_simulation/landman/git/pfb-clean/pfb/workers/init.py", line 96, in init return _init(opts) File "/Users/masoumeh/Desktop/AGN_simulation/landman/git/pfb-clean/pfb/workers/init.py", line 378, in _init chan_width=chan_width[Inu], File "/Users/masoumeh/opt/anaconda3/envs/landman/lib/python3.8/site-packages/dask/array/core.py", line 1962, in getitem index2 = normalize_index(index, self.shape) File "/Users/masoumeh/opt/anaconda3/envs/landman/lib/python3.8/site-packages/dask/array/slicing.py", line 904, in normalize_index raise IndexError("Too many indices for array") IndexError: Too many indices for array init_20221103-164306.log

landmanbester commented 1 year ago

Oh, I see the error does not end up in the log file. I should probably change that.

Did you get test_ascii_1h60.0s.MS using packratt? If so, how did you convert it to a single channel MS? Did you use casa's split? I did just that and I can't reproduce the error. I got a different error related to using WEIGHT because I forgot to migrate that functionality from the degrid_from_fits branch. It has been fixed in the issue65 branch. If I were you, I would delete the old virtual environment and start from scratch. Otherwise, since your MS is small, I suggest just zip it and email it to me so I can try to reproduce the error

landmanbester commented 1 year ago

Something that is very telling is that the parameters printed in the log file are no longer the same as the parameters on the master branch. I think something went wrong when you reinstalled so please create a new venv and try again

landmanbester commented 1 year ago

Maybe conda related, I never use conda only pip. Something like

$ python3.8 -m venv pfb
$ source pfb/bin/activate
$ pip install -U pip setuptools wheel
$ cd pfb-clean
$ git fetch origin
$ git checkout issue65
$ pip install -e .

should do the trick

masoumehghn commented 1 year ago

Maybe conda related, I never use conda only pip. Something like

$ python3.8 -m venv pfb
$ source pfb/bin/activate
$ pip install -U pip setuptools wheel
$ cd pfb-clean
$ git fetch origin
$ git checkout issue65
$ pip install -e .

should do the trick

Thanks, I followed this instruction and also installed casacore, I have following error for degrid now: pfb degrid --ms MS/test_ascii_8h60.0s.MS -o out/test8 --model-fits fits/cube.fits -nb 1 -nbo 1 --scheduler threads

Traceback (most recent call last): File "/Users/masoumeh/Desktop/AGN_simulation/landman/git/pfb/bin/pfb", line 33, in sys.exit(load_entry_point('pfb-clean', 'console_scripts', 'pfb')()) File "/Users/masoumeh/Desktop/AGN_simulation/landman/git/pfb/lib/python3.8/site-packages/click/core.py", line 1130, in call return self.main(args, kwargs) File "/Users/masoumeh/Desktop/AGN_simulation/landman/git/pfb/lib/python3.8/site-packages/click/core.py", line 1055, in main rv = self.invoke(ctx) File "/Users/masoumeh/Desktop/AGN_simulation/landman/git/pfb/lib/python3.8/site-packages/click/core.py", line 1657, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/Users/masoumeh/Desktop/AGN_simulation/landman/git/pfb/lib/python3.8/site-packages/click/core.py", line 1404, in invoke return ctx.invoke(self.callback, ctx.params) File "/Users/masoumeh/Desktop/AGN_simulation/landman/git/pfb/lib/python3.8/site-packages/click/core.py", line 760, in invoke return __callback(args, *kwargs) File "/Users/masoumeh/Desktop/AGN_simulation/landman/git/pfb-clean/pfb/workers/degrid.py", line 31, in degrid from daskms.fsspec_store import DaskMSStore File "/Users/masoumeh/Desktop/AGN_simulation/landman/git/pfb/lib/python3.8/site-packages/daskms/init.py", line 11, in from daskms.dask_ms import ( File "/Users/masoumeh/Desktop/AGN_simulation/landman/git/pfb/lib/python3.8/site-packages/daskms/dask_ms.py", line 6, in from daskms.table_proxy import TableProxy File "/Users/masoumeh/Desktop/AGN_simulation/landman/git/pfb/lib/python3.8/site-packages/daskms/table_proxy.py", line 9, in import pyrap.tables as pt File "/Users/masoumeh/Desktop/AGN_simulation/landman/git/pfb/lib/python3.8/site-packages/pyrap/tables.py", line 1, in from casacore.tables import File "/Users/masoumeh/Desktop/AGN_simulation/landman/git/pfb/lib/python3.8/site-packages/casacore/tables/init.py", line 60, in from .msutil import * File "/Users/masoumeh/Desktop/AGN_simulation/landman/git/pfb/lib/python3.8/site-packages/casacore/tables/msutil.py", line 29, in from casacore.tables.table import (table, taql, File "/Users/masoumeh/Desktop/AGN_simulation/landman/git/pfb/lib/python3.8/site-packages/casacore/tables/table.py", line 40, in from ._tables import (Table, ImportError: dlopen(/Users/masoumeh/Desktop/AGN_simulation/landman/git/pfb/lib/python3.8/site-packages/casacore/tables/_tables.cpython-38-darwin.so, 2): Library not loaded: @rpath/libcasa_derivedmscal.7.dylib Referenced from: /Users/masoumeh/Desktop/AGN_simulation/landman/git/pfb/lib/python3.8/site-packages/casacore/tables/_tables.cpython-38-darwin.so Reason: image not found

masoumehghn commented 1 year ago

Oh, I see the error does not end up in the log file. I should probably change that.

Did you get test_ascii_1h60.0s.MS using packratt? If so, how did you convert it to a single channel MS? Did you use casa's split? I did just that and I can't reproduce the error. I got a different error related to using WEIGHT because I forgot to migrate that functionality from the degrid_from_fits branch. It has been fixed in the issue65 branch. If I were you, I would delete the old virtual environment and start from scratch. Otherwise, since your MS is small, I suggest just zip it and email it to me so I can try to reproduce the error

I simply used simms ASCII command with nc 1:

simms -T vla -t ascii -cs itrf -l test_ascii -dec 30d0m0s -ra 10h0m0s -st 8 -dt 60 -f0 1.0GHz -nc 1 -df 10MHz Simms/observatories/vlac.itrf.txt

Is it wrong?

landmanbester commented 1 year ago

The problem seems to be that casa can't find a specific shared object so I suspect this is an issue with your casacore installation. You should be able to do the following in a python session

$ ipython
Python 3.8.10 (default, Mar 15 2022, 12:22:08)
Type 'copyright', 'credits' or 'license' for more information
IPython 8.6.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: from casacore.tables import *

In [2]:

If not, you need to fix that first.

landmanbester commented 1 year ago

This should work now. Please reopen if the issue persists