Open ClaytonBingham opened 3 years ago
JustasB
commented
3 years ago Hi @bingsome, The error does suggest the model exceeds available resources, but it may be some other issue. Would you be able to attach a zip file with minimal code to reproduce the issue?
JustasB
commented
3 years ago Hi @bingsome, I took a look at the files you sent me.
One issue that I think will be problematic in general is that the NEURON sections defined in the *_multi.py file are connected ("topology") in a daisy-chain fashion, but the 3D points define a branched, tree-like morphology.
This should visualize ok, but the simulation will be inaccurate.
If the *_multi.py file is generated using a program, is there a way to have it include the correct branching structure as part of the topology?
I'm looking for a workaround for the recursion/memory issue so you can at least visualize the cell.
ClaytonBingham
commented
3 years ago *multi.py is generated programmatically. I’ll be honest, I don’t yet understand the issue. Are you saying that you believe the morphology is connected incorrectly or out of order in NEURON? Or is there some other issue with the general approach to defining the morphology this way?
On Feb 13, 2021, at 8:22 AM, Justas Birgiolas notifications@github.com wrote:
Hi @bingsome, I took a look at the files you sent me.
One issue that I think will be problematic in general is that the NEURON sections defined in the *_multi.py file are connected ("topology") in a daisy-chain fashion, but the 3D points define a branched, tree-like morphology.
This should visualize ok, but the simulation will be inaccurate.
If the *_multi.py file is generated using a program, is there a way to have it include the correct branching structure as part of the topology?
I'm looking for a workaround for the recursion/memory issue so you can at least visualize the cell.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.
JustasB
commented
3 years ago Are you saying that you believe the morphology is connected incorrectly or out of order in NEURON?
I believe so. This should be a cell that has splitting dendrites, so there should be branch points defined in the way NEURON sections are connected.
For example, in multi.py, each section is connected to the end of the previous section.
#Building axon topology
from neuron import h
sectionList = [h.Section() for i in range(3548)]
sectionList[1].connect(sectionList[0](1),0)
sectionList[2].connect(sectionList[1](1),0)
sectionList[3].connect(sectionList[2](1),0)
...I don't see any branch points. A branch point would be defined as two sections being connected to a parent section (e.g. when a dendrite splits). For example this is how a branch would be defined:
sectionList[1].connect(sectionList[0](1),0) # Connects to parent sectionList[0]
sectionList[2].connect(sectionList[0](1),0) # also connects to same parent sectionList[0]I can't tell by glancing at the list if there are any branch points but doesn't look like it.
The program that generates the "#Building axon topology" part of the file should be creating those branch points, but it doesn't appear to be doing so.
ClaytonBingham
commented
3 years ago Grr. I'll revisit this. Thanks.
It's unclear to me how it would relate to the memory issue in Blender but I suppose it's not inconceivable either.
Should we press pause on this issue until I sort it out to be conscious of your time or are you thinking the model issue is unrelated to the memory error?
JustasB
commented
3 years ago So one workaround is to make all NEURON sections the immediate children of the first section. This will at least visualize the cell in BlenderNEURON.
I've emailed you the modified *multi.py file.
The underlying issue is that the cell sections are defined as nested child sections with 3500+ levels deep. When trying to reconstruct the tree, python recursion stack overflows.
TBH, I've never encountered NEURON models defined in this way. Usually, each non-branching segment of a dendrite gets one NEURON section, but in the multi file each non-branching segment appears to have hundreds of NEURON sections. If the non-branching segment sections could be merged, that would reduce the number of levels.
ClaytonBingham
commented
3 years ago One reason for defining the morphology in this way is to preserve micro structures (eg multiple serial myelin or bouton related sections which do not branch. These are ignored in swc's)...the approach is a programmatic pythonization (albeit, with a lurking bug) of the approach used for many popular hoc defined morphologies (eg https://senselab.med.yale.edu/ModelDB/ShowModel?model=114685&file=/JohnsonMcIntyre2008/GPe_model/GPe_uaxon_PROPS.hoc#tabs-2)
I will revisit the branching question but there are 3500 sections because there are 3500 biophysically distinct sections. Even if there were no complex branching collateral on this fiber, there would still be >1000 sections defined in NEURON.
JustasB
commented
3 years ago Ah I see, the need for biophysically distinct sections makes sense.
Unfortunately, I don't have a quick solution to address the memory issue. I'll keep looking at the code to see how such models could be handled better and will let you know if I find a solution.
Helveg
commented
3 years ago You could try tail-call optimization of the recursive functions:
def make_branches(roots):
branch_iter = iter(roots)
parent = None
branch_stack = []
while True:
try:
branch = next(branch_iter)
except StopIteration:
try:
branch_iter, parent = branch_stack.pop()
except KeyError:
break
else:
make_branch(branch, parent)
if branch.children:
branch_stack.append((branch_iter, parent))
branch_iter = iter(branch.children)
parent = branchThis should give you the same depth first iteration of the branches but without recursion.
I'm running a long corticofugal fiber in NEURON with ~3500 sections (owing to myelin,paranodes,nodes) and encountering this issue:
By changing the system recursion limit
I can limp past this issue but then encounter memory issues later:
Is my cell too complex for BlenderNEURON?