AllenInstitute / bmtk

Brain Modeling Toolkit
https://alleninstitute.github.io/bmtk/
BSD 3-Clause "New" or "Revised" License
272 stars 88 forks source link

Using bionet iclamp with MPI fails due to accessing cells not on rank #331

Open tjbanks opened 1 year ago

tjbanks commented 1 year ago

When adding a current clamp to cells on multiple ranks an error occurs when the gid isn't local to that rank -

Traceback (most recent call last):
  File "generate_traces.py", line 17, in <module>
    generate_parametric_traces(config)
  File "/home/tbg28/git_stage/act/act/utils.py", line 238, in generate_parametric_traces
    sim.run()
  File "/home/tbg28/git_stage/bmtk/bmtk/simulator/bionet/biosimulator.py", line 262, in run
    mod.initialize(self)
  File "/home/tbg28/git_stage/bmtk/bmtk/simulator/bionet/modules/iclamp.py", line 136, in initialize
    cell = sim.net.get_cell_gid(gid)
  File "/home/tbg28/git_stage/bmtk/bmtk/simulator/bionet/bionetwork.py", line 124, in get_cell_gid
    return self._rank_node_gids[gid]
KeyError: 0

A solution that worked for me was to update iclamp.py

# line 130
def initialize(self, sim):
        # Get select node gids, but only for those nodes that are on the current rank (if running on multiple cores)
        select_gids = list(sim.net.get_node_set(self._node_set).gids())
+        local_gids = sim.net.get_local_cells()
+        gids_on_rank = list(set(select_gids) & set(local_gids))
-         gids_on_rank = list(set(select_gids) & set(select_gids))

Will put up a PR for consideration.