search

cta-observatory / cta-lstchain

LST prototype testbench chain
https://cta-observatory.github.io/cta-lstchain/
BSD 3-Clause "New" or "Revised" License
24 stars 77 forks source link

Merging muon files fails sometimes #1191

Open morcuended opened 10 months ago

morcuended commented 10 months ago 
Traceback (most recent call last):
  File "/fefs/aswg/software/conda/envs/osa-v0.10/lib/python3.11/site-packages/astropy/table/operations.py", line 1481, in _vstack
    col = col_cls.info.new_like(cols, n_rows, metadata_conflicts, out_name)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/fefs/aswg/software/conda/envs/osa-v0.10/lib/python3.11/site-packages/astropy/table/column.py", line 482, in new_like
    attrs = self.merge_cols_attributes(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/fefs/aswg/software/conda/envs/osa-v0.10/lib/python3.11/site-packages/astropy/utils/data_info.py", line 752, in merge_cols_attributes
    out["dtype"] = metadata.common_dtype(cols)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/fefs/aswg/software/conda/envs/osa-v0.10/lib/python3.11/site-packages/astropy/utils/metadata.py", line 75, in common_dtype
    raise tme
astropy.utils.metadata.MergeConflictError: Arrays have incompatible types ['bool', 'float64']

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/var/spool/slurm//job30431945/slurm_script", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/fefs/aswg/software/conda/envs/osa-v0.10/lib/python3.11/site-packages/lstchain/scripts/lstchain_merge_muon_files.py", line 67, in main
    tab = vstack([tab, tab2])
          ^^^^^^^^^^^^^^^^^^^
  File "/fefs/aswg/software/conda/envs/osa-v0.10/lib/python3.11/site-packages/astropy/table/operations.py", line 682, in vstack
    out = _vstack(tables, join_type, col_name_map, metadata_conflicts)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/fefs/aswg/software/conda/envs/osa-v0.10/lib/python3.11/site-packages/astropy/table/operations.py", line 1485, in _vstack
    raise TableMergeError(
astropy.table.np_utils.TableMergeError: The 'good_ring' columns have incompatible types: ['bool', 'float64']
moralejo commented 9 months ago

I think this is related to (rare) files with no entries, in which some columns get a wrong default type. Can you check?

rlopezcoto commented 9 months ago

there used to be a check for files with no entries in lstchain, do not recall exactly where...