Catalog is mangrove got error

[parkma99]

When I run command python -m gwemopt --event S230609u --catalog mangrove, I got error, message is below

Found voevent S230609u-5-Update.xml
Latest skymap URL: https://gracedb.ligo.org/api/superevents/S230609u/files/Bilby.multiorder.fits,0
File /home/ma/Data/gwemopt/skymaps/S230609u_5_Bilby.multiorder.fits,0 already exists. Using this.
natural_nside = 256
nside = 256
Output directory: /home/ma/Data/gwemopt/S230609u_5_Bilby.multiorder/ATLAS
Loading skymap...
Generating catalog...
Loading from saved catalog: /home/ma/Data/gwemopt/catalogs/mangrove.hdf5
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/home/ma/code/gwemopt/gwemopt/__main__.py", line 55, in <module>
    run(sys.argv[1:])
  File "/home/ma/code/gwemopt/gwemopt/run.py", line 59, in run
    map_struct, catalog_struct = get_catalog(params, map_struct)
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ma/code/gwemopt/gwemopt/catalogs/get.py", line 46, in get_catalog
    cat_df = cat.get_catalog()
             ^^^^^^^^^^^^^^^^^
  File "/home/ma/code/gwemopt/gwemopt/catalogs/base_catalog.py", line 38, in get_catalog
    return self.load_catalog()
           ^^^^^^^^^^^^^^^^^^^
  File "/home/ma/code/gwemopt/gwemopt/catalogs/mangrove.py", line 61, in load_catalog
    return df[mask]
           ~~^^^^^^
  File "/home/ma/miniconda3/envs/gwemopt/lib/python3.11/site-packages/pandas/core/frame.py", line 3767, in __getitem__
    indexer = self.columns._get_indexer_strict(key, "columns")[1]
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ma/miniconda3/envs/gwemopt/lib/python3.11/site-packages/pandas/core/indexes/base.py", line 5876, in _get_indexer_strict
    self._raise_if_missing(keyarr, indexer, axis_name)
  File "/home/ma/miniconda3/envs/gwemopt/lib/python3.11/site-packages/pandas/core/indexes/base.py", line 5935, in _raise_if_missing
    raise KeyError(f"None of [{key}] are in the [{axis_name}]")
KeyError: "None of [Index([     0,      1,      2,      3,      4,      5,      6,      7,      8,\n            9,\n       ...\n       800976, 800977, 800978, 800979, 800980, 800981, 800982, 800983, 800984,\n       800985],\n      dtype='int64', length=800986)] are in the [columns]"

[parkma99]

I try fix this by changing L59 to

        mask = np.where(df["distmpc"] >= 0)

        return df.iloc[mask]

But I got another error:

Found voevent S230609u-5-Update.xml
Latest skymap URL: https://gracedb.ligo.org/api/superevents/S230609u/files/Bilby.multiorder.fits,0
File /home/ma/Data/gwemopt/skymaps/S230609u_5_Bilby.multiorder.fits,0 already exists. Using this.
natural_nside = 256
nside = 256
Output directory: /home/ma/Data/gwemopt/S230609u_5_Bilby.multiorder/ATLAS
Loading skymap...
Generating catalog...
Loading from saved catalog: /home/ma/Data/gwemopt/catalogs/mangrove.hdf5
Traceback (most recent call last):
  File "/home/ma/miniconda3/envs/gwemopt/lib/python3.11/site-packages/pandas/core/indexes/base.py", line 3652, in get_loc
    return self._engine.get_loc(casted_key)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "pandas/_libs/index.pyx", line 147, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/index.pyx", line 176, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/hashtable_class_helper.pxi", line 7080, in pandas._libs.hashtable.PyObjectHashTable.get_item
  File "pandas/_libs/hashtable_class_helper.pxi", line 7088, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'r'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/home/ma/code/gwemopt/gwemopt/__main__.py", line 55, in <module>
    run(sys.argv[1:])
  File "/home/ma/code/gwemopt/gwemopt/run.py", line 59, in run
    map_struct, catalog_struct = get_catalog(params, map_struct)
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ma/code/gwemopt/gwemopt/catalogs/get.py", line 69, in get_catalog
    mask = np.zeros(len(cat_df["r"]))
                        ~~~~~~^^^^^
  File "/home/ma/miniconda3/envs/gwemopt/lib/python3.11/site-packages/pandas/core/frame.py", line 3761, in __getitem__
    indexer = self.columns.get_loc(key)
              ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ma/miniconda3/envs/gwemopt/lib/python3.11/site-packages/pandas/core/indexes/base.py", line 3654, in get_loc
    raise KeyError(key) from err
KeyError: 'r'

[mcoughlin]

@parkma99 Thanks for reporting, see PR here: https://github.com/skyportal/gwemopt/pull/146. I will get this fixed.

[parkma99]

@mcoughlin Thank you 😊

[parkma99]

By the way, How to get the clu catalog?

[mcoughlin]

@parkma99 CLU is private, the new NED catalog should be sufficient.

[parkma99]

@parkma99 CLU is private, the new NED catalog should be sufficient.

@mcoughlin Thanks