casangi / xradio

Xarray Radio Astronomy Data IO
Other
9 stars 5 forks source link

Bug in time range in TaQL of read_expanded_main_table() #133

Closed FedeMPouzols closed 3 months ago

FedeMPouzols commented 6 months ago

Read_expanded_main_table, used for lazy/delayed reads, is broken, I think after PR #115 https://github.com/casangi/xradio/pull/115/files. The fix is simple:

--- a/src/xradio/vis/_vis_utils/_ms/_tables/read_main_table.py
+++ b/src/xradio/vis/_vis_utils/_ms/_tables/read_main_table.py
@@ -195,7 +195,7 @@ def read_main_table_chunks(
     n_baseline_chunks = chunks[1]
     # loop over time chunks
     for time_chunk in range(0, n_unique_times, n_time_chunks):
-        time_start = unique_times[time_chunk] - tol,
+        time_start = unique_times[time_chunk] - tol
         time_end = unique_times[min(n_unique_times, time_chunk + n_time_chunks) - 1] + tol

         # chunk time length

This can produce errors like:

  File "/home/fedemp/ws_xradio_pointing/venv_xradio_python_38/lib/python3.8/site-packages/xradio/vis/_vis_utils/_ms/_tables/read_main_table.py", line 160, in read_expanded_main_table
    xds, attrs = read_main_table_chunks(
  File "/home/fedemp/ws_xradio_pointing/venv_xradio_python_38/lib/python3.8/site-packages/xradio/vis/_vis_utils/_ms/_tables/read_main_table.py", line 220, in read_main_table_chunks
    with open_query(None, ts_taql) as query_times_ants:
  File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__
    return next(self.gen)
  File "/home/fedemp/ws_xradio_pointing/venv_xradio_python_38/lib/python3.8/site-packages/xradio/vis/_vis_utils/_ms/_tables/table_query.py", line 20, in open_query
    ttq = tables.taql(query)
  File "/home/fedemp/ws_xradio_pointing/venv_xradio_python_38/lib/python3.8/site-packages/casacore/tables/table.py", line 154, in taql
    tab = table(cmd, tabs, _oper=2)
  File "/home/fedemp/ws_xradio_pointing/venv_xradio_python_38/lib/python3.8/site-packages/casacore/tables/table.py", line 329, in __init__
    Table.__init__(self, tablename, tabledesc)
RuntimeError: Error in TaQL command: using style Python select * from $1 where DATA_DESC_ID = 0 AND (STATE_ID = 0 OR STATE_ID = 1 OR STATE_ID = 2 OR STATE_ID = 3 OR STATE_ID = 4 OR STATE_ID = 5 OR STATE_ID = 6 OR STATE_ID = 7 OR STATE_ID = 8 OR STATE_ID = 9 OR STATE_ID = 10 OR STATE_ID = 11 OR STATE_ID = 12 OR STATE_ID = 13 OR STATE_ID = 14 OR STATE_ID = 15 OR STATE_ID = 16 OR STATE_ID = 17 OR STATE_ID = 18 OR STATE_ID = 19 OR STATE_ID = 20 OR STATE_ID = 21 OR STATE_ID = 22 OR STATE_ID = 23) AND TIME BETWEEN (4819721894.076001,) and 4822405643.844 AND ANTENNA1 BETWEEN 1 and 25
  parse error at or near position 497 ')'

The code changes from https://github.com/casangi/xradio/commit/84ab864524376d834ad40590a0fbf8360795ca03 left a comma at the end of that line that turns a float value into a tuple with that value, which in turn adds parentheses to the TaQL query string, confusing the parser.