search

caracal-pipeline / caracal

Containerized Automated Radio Astronomy Calibration (CARACal) pipeline
GNU General Public License v2.0
29 stars 6 forks source link

Set model Parameter #1612

Open mangla-sarvesh opened 4 days ago

mangla-sarvesh commented 4 days ago

I want to use my own model of Pictor A as calibrator

So i used no_verify: True

no_verify
bool, optional, default = False
Enables setting standard manually [???].

But how should I give my model file.im ?

Athanaseus commented 3 days ago

Hi @mangla-sarvesh thank you for reporting this.

The no_verify needs some updates as it currently doesn't support user external model files. Currently, we internally support CASA model components, lsm.html models and WSClean clean component model. We are happy to provide an option for users to specify model files. Please let us know what format your sky model is in. Feel free to share an example.

Best regards

mangla-sarvesh commented 3 days ago

Hello @Athanaseus

I thought so too. So I followed this

  1. I have created a model of PICTOR A using MFS image from here
  2. Then through CASA (outside caracal), I ran setjy on the data which ran successfully and

  3. Then ran crosscal again but set_model = False , and in the first step of calibration this error came

    2024-11-19 15:16:37 INFO gaincal::::+ ########################################## 2024-11-19 15:16:37 INFO gaincal::::+ ##### Begin Task: gaincal ##### 2024-11-19 15:16:37 INFO gaincal:::: gaincal(vis="/stimela_mount/msdir/data_first-cal.ms",caltable="/stimela_mount/output/crosscal-data_first-1gc1_primary.K0",field="PictorA",spw="",intent="", 2024-11-19 15:16:37 INFO gaincal::::+ selectdata=True,timerange="",uvrange="",antenna="",scan="", 2024-11-19 15:16:37 INFO gaincal::::+ observation="",msselect="",solint="inf",combine="",preavg=-1.0, 2024-11-19 15:16:37 INFO gaincal::::+ refant="m009",refantmode="flex",minblperant=4,minsnr=3.0,solnorm=False, 2024-11-19 15:16:37 INFO gaincal::::+ normtype="mean",gaintype="K",smodel=[],calmode="ap",solmode="", 2024-11-19 15:16:37 INFO gaincal::::+ rmsthresh=[],corrdepflags=False,append=False,splinetime=3600.0,npointaver=3, 2024-11-19 15:16:37 INFO gaincal::::+ phasewrap=180.0,docallib=False,callib="",gaintable=[],gainfield=[], 2024-11-19 15:16:37 INFO gaincal::::+ interp=[],spwmap=[],parang=False) 2024-11-19 15:16:37 INFO gaincal::calibrater::open Using NEW VI2-driven calibrater tool 2024-11-19 15:16:37 INFO gaincal::calibrater::open Opening MS: /stimela_mount/msdir/data_first-cal.ms for calibration. 2024-11-19 15:16:38 INFO gaincal::Calibrater:: Initializing nominal selection to the whole MS. 2024-11-19 15:16:38 INFO gaincal:::: NB: gaincal automatically excludes auto-correlations. 2024-11-19 15:16:38 INFO calibrater::setdata Beginning selectvis--(MSSelection version)------- 2024-11-19 15:16:38 INFO calibrater::reset Reseting solve/apply state 2024-11-19 15:16:38 INFO Calibrater::selectvis Performing selection on MeasurementSet 2024-11-19 15:16:38 INFO Calibrater::selectvis+ Selecting on field: 'PictorA' 2024-11-19 15:16:38 INFO Calibrater::selectvis+ Selecting with TaQL: 'ANTENNA1!=ANTENNA2' 2024-11-19 15:16:38 INFO Calibrater::selectvis By selection 350436 rows are reduced to 338352 2024-11-19 15:16:38 INFO Calibrater::selectvis Frequency selection: Selecting all channels in all spws. 2024-11-19 15:16:38 INFO calibrater::setdata chanmode=none nchan=1 start=0 step=1 mStart='0km/s' mStep='0km/s' msSelect='ANTENNA1!=ANTENNA2' 2024-11-19 15:16:38 INFO calibrater::setsolve Beginning setsolve--(MSSelection version)------- 2024-11-19 15:16:38 INFO Calibrater::setsolve Arranging to SOLVE: 2024-11-19 15:16:38 INFO Calibrater::setsolve . K Jones: table=/stimela_mount/output/crosscal-data_first-1gc1_primary.K0 append=false solint=inf refantmode='flex' refant='m009' minsnr=3 apmode=AP solnorm=false 2024-11-19 15:16:38 INFO calibrater::solve Beginning solve----------------------------- 2024-11-19 15:16:38 INFO Calibrater::solve The following calibration terms are arranged for apply: 2024-11-19 15:16:38 INFO Calibrater::solve . (None) 2024-11-19 15:16:38 INFO Calibrater::solve The following calibration term is arranged for solve: 2024-11-19 15:16:38 INFO Calibrater::solve . K Jones: table=/stimela_mount/output/crosscal-data_first-1gc1_primary.K0 append=false solint=inf refantmode='flex' refant='m009' minsnr=3 apmode=AP solnorm=false 2024-11-19 15:16:39 INFO Calibrater::solve For solint = inf, found 7 solution intervals. 2024-11-19 15:16:40 SEVERE Calibrater::solve Caught exception: Failure to load /scratch3/users/mangla1991/SetBlo_first/crosscal/MS/data_first-cal.ms/SOURCE/FT_MODEL2415240_16406 image from disk 2024-11-19 15:16:40 INFO Calibrater::solve Reseting entire solve/apply state. 2024-11-19 15:16:40 SEVERE Exception Reported: Error in Calibrater::solve. 2024-11-19 15:16:40 SEVERE gaincal:::: Task gaincal raised an exception of class RuntimeError with the following message: Error in Calibrater::solve. Running CASA task 'gaincal' Traceback (most recent call last): File "/stimela_mount/code/run.py", line 13, in task.run() File "/usr/local/lib/python3.8/dist-packages/Crasa/Crasa.py", line 122, in run raise CasaException("CASA raised a SEVERE exception while running task {0:s}".format(self.task)) Crasa.Crasa.CasaException: CASA raised a SEVERE exception while running task gaincal 2024-11-19 17:16:41 CARACal.Stimela.delay_cal-1gc1-0-0-primary ERROR: /software/common/singularity/4.1.3/bin/singularity run --workdir /scratch3/users/mangla1991/SetBlo_first/crosscal/.stimela_workdir-17320293264945986 --containall --userns returns error code 1 2024-11-19 17:16:41 CARACal.Stimela.delay_cal-1gc1-0-0-primary ERROR: job failed at 2024-11-19 17:16:41.009206 after 0:00:16.951714 2024-11-19 17:16:41 CARACal ERROR: Job 'delay_cal-1gc1-0-0-primary:: K calibration' failed: /software/common/singularity/4.1.3/bin/singularity run --workdir /scratch3/users/mangla1991/SetBlo_first/crosscal/.stimela_workdir-17320293264945986 --containall --userns returns error code 1 [PipelineException] 2024-11-19 17:16:41 CARACal INFO: More information can be found in the logfile at /scratch3/users/mangla1991/SetBlo_first/crosscal/output/logs-20241119-171438/log-caracal.txt 2024-11-19 17:16:41 CARACal INFO: exiting with error code 1

How can i solve this? Should I make some.html file and somehow add this somewhere? I don't know how to do that or Use crystalball somehow?

Athanaseus commented 1 day ago

Hi @mangla-sarvesh,

Using the corresponding model fits images, one could use the wsclean predict operation, which could possibly be an option for caracal. Could you request access to the per-band model.fits images?

Regarding the error: 2024-11-19 15:16:40 SEVERE Calibrater::solve Caught exception: Failure to load /scratch3/users/mangla1991/SetBlo_first/crosscal/MS/data_first-cal.ms/SOURCE/FT_MODEL2415240_16406 image from disk

I haven’t encountered this issue before. Could you try running the same task in CASA to check if the error is reproducible?

gaincal(vis="/stimela_mount/msdir/data_first-cal.ms",caltable="/stimela_mount/output/crosscal-data_first-1gc1_primary.K0",field="PictorA",spw="",intent="",
2024-11-19 15:16:37 INFO gaincal::::+ selectdata=True,timerange="",uvrange="",antenna="",scan="",
2024-11-19 15:16:37 INFO gaincal::::+ observation="",msselect="",solint="inf",combine="",preavg=-1.0,
2024-11-19 15:16:37 INFO gaincal::::+ refant="m009",refantmode="flex",minblperant=4,minsnr=3.0,solnorm=False,
2024-11-19 15:16:37 INFO gaincal::::+ normtype="mean",gaintype="K",smodel=[],calmode="ap",solmode="",
2024-11-19 15:16:37 INFO gaincal::::+ rmsthresh=[],corrdepflags=False,append=False,splinetime=3600.0,npointaver=3,
2024-11-19 15:16:37 INFO gaincal::::+ phasewrap=180.0,docallib=False,callib="",gaintable=[],gainfield=[],
2024-11-19 15:16:37 INFO gaincal::::+ interp=[],spwmap=[],parang=False)

PS: Please also share the full log files.