dnarayanan / caesar

Other
15 stars 10 forks source link

fof6d error in galaxy finding at high redshift #43

Open weiguangcui opened 3 years ago

weiguangcui commented 3 years ago

Here are the calculations with errors:

In [1]: import caesar, yt
In [2]: ds  = yt.load('/home2/weiguang/data7/Gizmo-Simba/NewMDCLUSTER_0037/snap_022.hdf5')
In [3]: obj = caesar.CAESAR(ds,skip_hash_check=True)
In [4]: obj.member_search(haloid='snap',fof6d_file='/tmp/fof6d',nproc=1,lowres=[2,3])
yt : [INFO     ] 2020-12-17 22:56:34,861 member_search() running on 1 cores
/home2/weiguang/caesar/caesar/data_manager.py:32: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if select is 'all': self._load_gas_data()
/home2/weiguang/caesar/caesar/data_manager.py:34: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if select is 'all': self._load_star_data()
/home2/weiguang/caesar/caesar/data_manager.py:37: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if select is 'all': self._load_bh_data()
/home2/weiguang/caesar/caesar/data_manager.py:153: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if select is 'all':
/home2/weiguang/caesar/caesar/data_manager.py:213: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if select is 'all':
/home2/weiguang/caesar/caesar/data_manager.py:245: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if select is 'all':
yt : [INFO     ] 2020-12-17 22:56:35,141 No black holes found, RAM=0.2813 GB
yt : [INFO     ] 2020-12-17 22:56:35,152 Calculated Omega_Baryon=0.0482017 and 3840^3 effective resolution
yt : [INFO     ] 2020-12-17 22:56:35,152 Using FOF Halo ID from snapshots, RAM=0.2813 GB
/home2/weiguang/.local/lib/python3.8/site-packages/numpy/core/_asarray.py:83: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray
  return array(a, dtype, copy=False, order=order)
yt : [INFO     ] 2020-12-17 22:56:35,740 Initializing member search, loading particles, RAM=0.367 GB
yt : [INFO     ] 2020-12-17 22:56:35,740 No black holes found, RAM=0.367 GB
yt : [INFO     ] 2020-12-17 22:56:37,971 Loaded particle data, RAM=0.3666 GB
yt : [INFO     ] 2020-12-17 22:56:39,087 Loaded baryon data, RAM=0.3454 GB
yt : [INFO     ] 2020-12-17 22:56:39,146 Found 1 valid halos, loaded indexes, RAM=0.3454 GB
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-4-7e59a27c3716> in <module>
----> 1 obj.member_search(haloid='snap',fof6d_file='/tmp/fof6d',nproc=1,lowres=[2,3])

~/caesar/caesar/main.py in member_search(self, *args, **kwargs)
    338         else:
    339             from caesar.fubar_halo import fubar_halo
--> 340             fubar_halo(self)
    341             assign.assign_galaxies_to_halos(self)
    342             assign.assign_clouds_to_galaxies(self)

~/caesar/caesar/fubar_halo.py in fubar_halo(obj)
     71         mylog.warning('No valid halos found! Aborting member search')
     72         return
---> 73     get_group_properties(halos,halos.obj.halo_list)  # compute halo properties
     74 
     75     # Find galaxies, or load galaxy membership info

~/caesar/caesar/group.py in get_group_properties(self, grp_list)
    774 def get_group_properties(self,grp_list):
    775 
--> 776     from caesar.group_funcs import get_group_overall_properties,get_group_gas_properties,get_group_star_properties,get_group_bh_properties
    777 
    778     get_group_overall_properties(self,grp_list)

ImportError: cannot import name 'get_group_overall_properties' from 'caesar.group_funcs' (unknown location)
### a little more details on the variable values:
---------------------------------------------------------------------------------------------------------------------------
ipdb> mygals
array([0, 1])
ipdb> len(grp_tags)
1
ipdb> len(self.obj.halo_list)
1
ipdb> grp_tags[ih]
array([-1,  1,  1,  1,  1,  1, -1,  1,  1, -1, -1,  1,  1,  1,  1, -1, -1,
       -1, -1,  1,  1,  1,  1,  1,  1,  0, -1, -1,  0,  0,  0,  0,  0,  0,
        0,  0,  0,  0, -1, -1, -1, -1, -1, -1, -1, -1,  0,  0,  0,  0, -1,
       -1, -1, -1,  0, -1,  0,  0,  0,  0,  1, -1, -1,  0,  0,  0,  0,  0,
        0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  1,  1,
        1,  1, -1,  1, -1,  1,  1,  1,  1])
ipdb> g_inds[0].size
94
ipdb> grp_tags[0].size
94
ipdb> len_hi
250
ipdb> len_gi
94

This seems to happen when there is only one halo. The tags in grp_tags don’t get removed properly.

dnarayanan commented 3 years ago

thanks for reporting this! is it possible to post an example snapshot somewhere that I could use to help debug?