error in prep accessing the calibrator ?

[abhisek176]

I see the below error after the pipeline finishes successfully. The Calibrator used is J0408-6545. Could you please let me know what's the issue?

Appended below the log: 2024-05-16 01:00:59 CARACal INFO: prep: initializing 2024-05-16 01:00:59 CARACal INFO: Obtaining divine knowledge from /home/amohapatra/soft/caracal/caracal/data/southern_calibrators.txt 2024-05-16 01:01:00 CARACal INFO: Obtaining divine knowledge from /home/amohapatra/soft/caracal/caracal/data/casa_calibrators.txt 2024-05-16 01:01:00 CARACal INFO: Checking for crossmatch --- Logging error --- Traceback (most recent call last): File "/home/amohapatra/soft/anaconda3.9.4/lib/python3.9/logging/init.py", line 1083, in emit msg = self.format(record) File "/home/amohapatra/soft/anaconda3.9.4/lib/python3.9/logging/init.py", line 927, in format return fmt.format(record) File "/home/amohapatra/soft/abhi-caracal/lib/python3.9/site-packages/stimela/utils/logger.py", line 88, in format return self._default_formatter.format(record) File "/home/amohapatra/soft/anaconda3.9.4/lib/python3.9/logging/init.py", line 663, in format record.message = record.getMessage() File "/home/amohapatra/soft/anaconda3.9.4/lib/python3.9/logging/init.py", line 367, in getMessage msg = msg % self.args TypeError: not all arguments converted during string formatting Call stack: File "/home/amohapatra/soft/abhi-caracal/bin/caracal", line 8, in sys.exit(driver()) File "/home/amohapatra/soft/caracal/caracal/main.py", line 222, in driver main(sys.argv[1:]) File "/home/amohapatra/soft/caracal/caracal/main.py", line 318, in main execute_pipeline(options, config, block=True) File "/home/amohapatra/soft/caracal/caracal/main.py", line 214, in execute_pipeline return run(debug=options.debug) File "/home/amohapatra/soft/caracal/caracal/main.py", line 189, in run pipeline.run_workers() File "/home/amohapatra/soft/caracal/caracal/workers/worker_administrator.py", line 436, in run_workers worker.worker(self, recipe, config) File "/home/amohapatra/soft/caracal/caracal/workers/prep_worker.py", line 78, in worker fielddb, ra_corr, dec_corr = getfield_coords(msdict, f, db, tol=tol, tol_diff=tol_diff) File "/home/amohapatra/soft/caracal/caracal/workers/prep_worker.py", line 38, in getfield_coords caracal.log.info("Database keys:", dbcp.keys()) Message: 'Database keys:' Arguments: (dict_keys(['0056-001', '0201-440', '0237-233', '0252-712', 'CTA21', '0320+053', '0403-132', '0408-65', 'PKS1934-63', '0410-752', '0420-625', '0531+194', '0823-500', '0906-682', '3C237', '1127-145', '1139-285', '1151-348', '1240-209', '3C283', '1421-490', '1740-517', '1827-360', '1830-210', '1921-293', '1938-155', '2203-188', '3C446', 'CTA102']),) --- Logging --- Traceback (most recent call last): File "/home/amohapatra/soft/anaconda3.9.4/lib/python3.9/logging/init.py", line 1083, in emit msg = self.format(record) File "/home/amohapatra/soft/anaconda3.9.4/lib/python3.9/logging/init.py", line 927, in format return fmt.format(record) File "/home/amohapatra/soft/abhi-caracal/lib/python3.9/site-packages/stimela/utils/logger.py", line 88, in format return self._default_formatter.format(record) File "/home/amohapatra/soft/abhi-caracal/lib/python3.9/site-packages/stimela/utils/logger.py", line 74, in format return ConsoleColors.colorize(super(ColorizingFormatter, self).format(record), style or self._default_color) File "/home/amohapatra/soft/anaconda3.9.4/lib/python3.9/logging/init.py", line 663, in format record.message = record.getMessage() File "/home/amohapatra/soft/anaconda3.9.4/lib/python3.9/logging/init.py", line 367, in getMessage msg = msg % self.args TypeError: not all arguments converted during string formatting Call stack: File "/home/amohapatra/soft/abhi-caracal/bin/caracal", line 8, in sys.exit(driver()) File "/home/amohapatra/soft/caracal/caracal/main.py", line 222, in driver main(sys.argv[1:]) File "/home/amohapatra/soft/caracal/caracal/main.py", line 318, in main execute_pipeline(options, config, block=True) File "/home/amohapatra/soft/caracal/caracal/main.py", line 214, in execute_pipeline return run(debug=options.debug) File "/home/amohapatra/soft/caracal/caracal/main.py", line 189, in run pipeline.run_workers() File "/home/amohapatra/soft/caracal/caracal/workers/worker_administrator.py", line 436, in run_workers worker.worker(self, recipe, config) File "/home/amohapatra/soft/caracal/caracal/workers/prep_worker.py", line 78, in worker fielddb, ra_corr, dec_corr = getfield_coords(msdict, f, db, tol=tol, tol_diff=tol_diff) File "/home/amohapatra/soft/caracal/caracal/workers/prep_worker.py", line 38, in getfield_coords caracal.log.info("Database keys:", dbcp.keys()) Message: 'Database keys:' Arguments: (dict_keys(['0056-001', '0201-440', '0237-233', '0252-712', 'CTA21', '0320+053', '0403-132', '0408-65', 'PKS1934-63', '0410-752', '0420-625', '0531+194', '0823-500', '0906-682', '3C237', '1127-145', '1139-285', '1151-348', '1240-209', '3C283', '1421-490', '1740-517', '1827-360', '1830-210', '1921-293', '1938-155', '2203-188', '3C446', 'CTA102']),) 2024-05-16 01:01:00 CARACal INFO: Calibrator coordinates match within the specified tolerance. 2024-05-16 01:01:00 CARACal INFO: Checking the CASA database of calibrators. 2024-05-16 01:01:00 CARACal INFO: Checking for crossmatch --- Logging error --- Traceback (most recent call last): File "/home/amohapatra/soft/anaconda3.9.4/lib/python3.9/logging/init.py", line 1083, in emit msg = self.format(record) File "/home/amohapatra/soft/anaconda3.9.4/lib/python3.9/logging/init.py", line 927, in format return fmt.format(record) File "/home/amohapatra/soft/abhi-caracal/lib/python3.9/site-packages/stimela/utils/logger.py", line 88, in format return self._default_formatter.format(record) File "/home/amohapatra/soft/anaconda3.9.4/lib/python3.9/logging/init.py", line 663, in format record.message = record.getMessage() File "/home/amohapatra/soft/anaconda3.9.4/lib/python3.9/logging/init.py", line 367, in getMessage msg = msg % self.args TypeError: not all arguments converted during string formatting Call stack: File "/home/amohapatra/soft/abhi-caracal/bin/caracal", line 8, in sys.exit(driver()) File "/home/amohapatra/soft/caracal/caracal/main.py", line 222, in driver main(sys.argv[1:]) File "/home/amohapatra/soft/caracal/caracal/main.py", line 318, in main execute_pipeline(options, config, block=True) File "/home/amohapatra/soft/caracal/caracal/main.py", line 214, in execute_pipeline return run(debug=options.debug) File "/home/amohapatra/soft/caracal/caracal/main.py", line 189, in run pipeline.run_workers() File "/home/amohapatra/soft/caracal/caracal/workers/worker_administrator.py", line 436, in run_workers worker.worker(self, recipe, config) File "/home/amohapatra/soft/caracal/caracal/workers/prep_worker.py", line 81, in worker fielddb, ra_corr, dec_corr = getfield_coords(msdict, f, dbc, tol=tol, tol_diff=tol_diff) File "/home/amohapatra/soft/caracal/caracal/workers/prep_worker.py", line 38, in getfield_coords caracal.log.info("Database keys:", dbcp.keys()) Message: 'Database keys:' Arguments: (dict_keys(['3C48', '3C123', '3C138', '3C147', '3C196', '3C286', '3C380']),) --- Logging error --- Traceback (most recent call last): File "/home/amohapatra/soft/anaconda3.9.4/lib/python3.9/logging/init.py", line 1083, in emit msg = self.format(record) File "/home/amohapatra/soft/anaconda3.9.4/lib/python3.9/logging/init.py", line 927, in format return fmt.format(record) File "/home/amohapatra/soft/abhi-caracal/lib/python3.9/site-packages/stimela/utils/logger.py", line 88, in format return self._default_formatter.format(record) File "/home/amohapatra/soft/abhi-caracal/lib/python3.9/site-packages/stimela/utils/logger.py", line 74, in format return ConsoleColors.colorize(super(ColorizingFormatter, self).format(record), style or self._default_color) File "/home/amohapatra/soft/anaconda3.9.4/lib/python3.9/logging/init.py", line 663, in format record.message = record.getMessage() File "/home/amohapatra/soft/anaconda3.9.4/lib/python3.9/logging/init.py", line 367, in getMessage msg = msg % self.args TypeError: not all arguments converted during string formatting Call stack: File "/home/amohapatra/soft/abhi-caracal/bin/caracal", line 8, in sys.exit(driver()) File "/home/amohapatra/soft/caracal/caracal/main.py", line 222, in driver main(sys.argv[1:]) File "/home/amohapatra/soft/caracal/caracal/main.py", line 318, in main execute_pipeline(options, config, block=True) File "/home/amohapatra/soft/caracal/caracal/main.py", line 214, in execute_pipeline return run(debug=options.debug) File "/home/amohapatra/soft/caracal/caracal/main.py", line 189, in run pipeline.run_workers() File "/home/amohapatra/soft/caracal/caracal/workers/worker_administrator.py", line 436, in run_workers worker.worker(self, recipe, config) File "/home/amohapatra/soft/caracal/caracal/workers/prep_worker.py", line 81, in worker fielddb, ra_corr, dec_corr = getfield_coords(msdict, f, dbc, tol=tol, tol_diff=tol_diff) File "/home/amohapatra/soft/caracal/caracal/workers/prep_worker.py", line 38, in getfield_coords caracal.log.info("Database keys:", dbcp.keys()) Message: 'Database keys:' Arguments: (dict_keys(['3C48', '3C123', '3C138', '3C147', '3C196', '3C286', '3C380']),) 2024-05-16 01:01:00 CARACal INFO: The file MKAT_sdp_l0.8k-cal.ms already has a flag version called "caracal_legacy". Restoring it. 2024-05-16 01:01:00 CARACal INFO: prep: running 2024-05-16 01:01:00 CARACal.Stimela.restore-flags-prep-ms0 INFO: job started at 2024-05-16 01:01:00.519952

[KshitijT]

@abhisek176 could you please append your settings file?

[abhisek176]

Thanks @KshitijT, here is the file. [config.txt]

[KshitijT]

@abhisek176 could you please check what's the field name for the calibrator in the measurement set?

[abhisek176]

Hi @KshitijT, the field name is J0408-6545.

[abhisek176]

Hi @KshitijT: it seems like the issue is in this line caracal.log.info("Database keys:", dbcp.keys()) (line 38). TypeError due to dbcp.keys(). After commenting out this line, it works fine. May be adding str(dbcp.keys()) would help?