jimplank
commented
4 weeks ago There's additional subtlety, because you should only fire if the charge value exceeds your threshold at the end of the integration cycle. So if my threshold is 10 and my current potential is 0, then receiving three spikes with weights of 5 and one with a weight of -9 means at the end of the integration cycle, I shouldn't fire, but my potential should be 6.
Different neuroprocessors have addressed this in different ways:
-
DANNA: There was an explicit max potential and min potential. In the above example, if the maximum potential is 13, then the three spikes with a weight of 5 would add up to 13 and not 15, and then the negative spike would set the potential to be 4. Obviously, now the order of spikes matters and is an additional potential mismatch between simulator and neuroprocessor. In DANNA, there were 16 subcycles within each integration cycle, and when Adam wrote the simulator, he accounted for that, which removed the nondeterminism. It slowed the simulator down, of course.
-
RAVENS: If I'm not mistaken, the maximum fan-in of Ravens means that you can calculate the min and max potentials a-priori, and set min and max thresholds accordingly.
I'm not sure what I advocate here — we can add parameters for min_potential and max_potential and then pray it doesn't bite us with non-determinism too badly. If we set it high enough, it won't. We can also have the simulator identify non-determinism, and flag it to the user, so that they can modify the min/max potentials.
That can be done with graph analysis too (which is pretty much what RAVENS does), but graph analysis will be overly conservative.
Thoughts?
On Aug 14, 2024, at 12:33 AM, Keegan Dent @.***> wrote:
(best viewed on GitHub site/app for TeX formatting)
From my review of the RISP processor code, it seems that, in a scenario where there is no leak on a neuron and non_negative_charge is false, a neuron could, in theory, accumulate infinite negative charge. This doesn't happen on the positive end because there is a maximum threshold for neurons, and therefore they must eventually fire with continued application of positive charge.
Obviously this is not a behavior I can replicate in hardware with finite integer widths. At the moment I only bother to provide enough width to accommodate $[- 2 ^ {n - 1}, 2 ^ {n - 1} - 1]$ where $n = \left\lceil \log_{2}{ \left( \text{threshold}+\text{threshold_inclusive} \right)} \right\rceil$. For neurons with small thresholds, this will cause significant divergence from the Framework when they accumulate negative charge.
The way I see it, there are two ways to address this problem.
- Assert non_negative_charge==true for FPGA networks. We only support discrete networks anyway, so perhaps adding another restriction isn't overly burdensome to the user.
- The Framework RISP (and possibly other processors) should define rules for negative charge accumulation limits, specifically for discrete networks. I can use these limits to define the potential width for the FPGA neurons.
Thanks
— Reply to this email directly, view it on GitHubhttps://github.com/TENNLab-UTK/fpga/issues/17, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AGEAIREDD52V4WZ6OWGZEKTZRLM3JAVCNFSM6AAAAABMPQ2XB6VHI2DSMVQWIX3LMV43ASLTON2WKOZSGQ3DIOBXHE2DSOI. You are receiving this because you were assigned.Message ID: @.***>
BGull00
keegandent
(best viewed on GitHub site/app for TeX formatting)
From my review of the RISP processor code, it seems that, in a scenario where there is no leak on a neuron and
non_negative_chargeisfalse, a neuron could, in theory, accumulate infinite negative charge. This doesn't happen on the positive end because there is a maximum threshold for neurons, and therefore they must eventually fire with continued application of positive charge.Obviously this is not a behavior I can replicate in hardware with finite integer widths. At the moment I only bother to provide enough width to accommodate $[- 2 ^ {n - 1}, 2 ^ {n - 1} - 1]$ where $n = \left\lceil \log_{2}{ \left( \text{threshold}+\text{threshold\_inclusive} \right)} \right\rceil$. For neurons with small thresholds, this will cause significant divergence from the Framework when they accumulate negative charge.
The way I see it, there are two ways to address this problem.
non_negative_charge==truefor FPGA networks. We only support discrete networks anyway, so perhaps adding another restriction isn't overly burdensome to the user.Thanks