The weight of the Connection cannot be updated using STDP rules based on the one-step running network mode in BindsNET

[289848771]

Hello everyone, I have a question about the weight update of BindsNET, I hope someone can help me solve it.

My problem description is: During the process of running the network in a single step, I found that the synaptic weights of each synapse in the two neuron layers that have been added to STDP have always remained constant without any change.

Specifically, I built a spiking neural network model via BindsNET and ran it via single-step mode, i.e. network.run(time=1). I added a 5000-round loop outside this sentence, which means that the network must run in single-step operation mode for 5000 time steps. But I found that in this state, the STDP learning rule between a certain two layers in my model did not play any role. In 5000 steps, each of my synaptic weights always remained the same, always the same.

The design code for Connection is as follows:

`

Create the connection between A_layer and B_layer

   A_B_conn = Connection(
        source=A_layer,
        target=B_layer,
        w=1.6 * torch.ones(A_layer.n, B_layer.n).to(device),
        nu=(1e-4, 1e-2),
        wmin=torch.zeros(A_layer.n, B_layer.n).to(device),
        wmax=4.0 * torch.ones(A_layer.n, B_layer.n).to(device),
        update_rule=PostPre

`

As shown in the code above, my weights are designed with learning rules to help update. In the initial stage, I set the connection of this layer to fully connected mode, that is, the weights are all the same, all 1.6. My target layer has another different and uneven pulse stimulus. It stands to reason that the neurons in these two layers receive the same pulse. The output is different, and the activation conditions are also different. This will cause the learning rules to be updated, resulting in uneven changes in the weights between A_layer and B_layer, which should be reflected in the picture as spot patterns of different colors. But as shown in the figure below, my weight has never changed (I plotted the network weight every 500 time steps and found that it has always remained unchanged). It is exactly the same yellow throughout, which means that the weight has always been constant at 1.6.

The above is my problem. I sincerely ask you and the developers of BindNET how to solve my problem. This is really important to me. I would really appreciate it if you would discuss it with me.

[Hananel-Hazan]

Thank you for raising this issue. From your description the weights should be updated...

Can you post two another figures.

1. figure that plot the voltage of each neurons vs time.
2. figure that plots the input values / spikes train that the input layer received.

[rafaelblevin821]

What dataset are you training it on?

It looks like the network is not learning.

Can you save the weights and share them?

I use this code to save weights:

    if epoch == 0 or (epoch + 1) % 1 == 0:
        np.savetxt(
            os.path.join('network', cur_time, 'weights', f'weights_epoch_{epoch + 1}.csv'),
            network.connections[('X', 'Ae')].w.detach().cpu().numpy(), delimiter=',', fmt='%0.3f')
        print(f"Weights saved for epoch {epoch + 1}")

[289848771]

你在什么数据集上训练它？

看起来网络没有学习。

你能保存权重并分享它们吗？

我使用此代码来保存权重：

    if epoch == 0 or (epoch + 1) % 1 == 0:
        np.savetxt(
            os.path.join('network', cur_time, 'weights', f'weights_epoch_{epoch + 1}.csv'),
            network.connections[('X', 'Ae')].w.detach().cpu().numpy(), delimiter=',', fmt='%0.3f')
        print(f"Weights saved for epoch {epoch + 1}")

What dataset are you training it on?

It looks like the network is not learning.

Can you save the weights and share them?

I use this code to save weights:

    if epoch == 0 or (epoch + 1) % 1 == 0:
        np.savetxt(
            os.path.join('network', cur_time, 'weights', f'weights_epoch_{epoch + 1}.csv'),
            network.connections[('X', 'Ae')].w.detach().cpu().numpy(), delimiter=',', fmt='%0.3f')
        print(f"Weights saved for epoch {epoch + 1}")

Hello, I am trying to use some inverse kinematics data of a robotic arm to drive the robotic arm to perform circular trajectory control through torque. Among them, I encoded the joint position and speed information of the robotic arm and input it to the SNN network built by BindsNet. Then after its training and learning, the output layer outputs spikes. I converted the spikes into torque commands through a simple linear formula and output Give the robotic arm and control its movement.

It is indeed what you said. The neural network I built did not learn, and its connection weights never changed.

I tried to use a txt file to save the weights of the first 500 steps of the network run (can be seen in the attachment), weight.txt but we can see that all the weights of the connection always remain the same as the initial value (I set the connection matrix to a full connection, and each The element value is 1.6. The weights did not change during the entire run)

Since this connected layer is huge, I cannot save the weights for 5000 steps. My source layer has 70,000 LIF neurons, and my target layer has 700 LIF neurons. Their initial connection method is fully connected, and their connection matrix is very large. Here is my weight print statement:

weight = str(network.monitors['gc_pc_monitor'].get("w"))
weight_file = open("Circle_Trajectory_Save/weight.txt", "w")
weight_file.write(weight)
weight_file.close()

I could try saving the weights over different time periods if you need.

[289848771]

感谢您提出这个问题。根据您的描述，权重应该更新......

你能再贴两个数字吗？

1. 绘制每个神经元的电压与时间的关系图。
2. 绘制输入层接收的输入值/峰值训练的图。

Hello, thank you very much for your reply.

I will provide the two figures you proposed. But before providing it, I have to briefly explain the purpose of my network design and the SNN network structure. I am trying to use some inverse kinematics data of the robotic arm to drive the robotic arm to perform circular trajectory control through torque. Among them, I encode the joint position and speed information of the robotic arm and input it into the SNN network built by BindsNet. Then after training and learning, the output layer outputs spikes. I convert the spikes into torque commands via a simple linear formula and give the output to the robotic arm and control its movement.

The above briefly describes the purpose of my SNN network model. Let's take a look at the simple schematic diagram of my SNN network:

As shown in the picture, my network has two input layers: MF layer and CF layer. My network only has one output layer: DCN layer. Among them, only the STDP learning rule is used in the GC-PC layer. The GC layer is the source layer with 70,000 LIF neurons. The PC layer is the target layer and has 700 LIF neurons. There is a fully connected relationship between them. So I set the connection matrix of GC-PC to an all-one matrix, where the value of each element is 1.6.

The following is the figures you need.

The first is the relationship between voltage and time for each neuron layer, as shown below (My SNN network model has a total of five neuron layers, 2 of which are input layers and have no voltage attributes, so they can only provide the remaining The voltage-time relationship diagram of GC, PC, and DCN).

Then there is a diagram of the input layer receiving the input spikes (where MF and CF are input layers), as shown below.

[Hananel-Hazan]

Thank you for providing the figures. It doesn't seem like the GC part is getting any information from the MF. Since the PC and DCN appear to be very active, that suggests their activity is coming from the CF. Because the GC is not active, it seems that either the connection between MF and GC is too weak, or the MF has less information to transmit, or the encoding of the MF input doesn't elicit enough activity.

Suggestions:

1. Since the DCN already receives input from both MF and CF, can you route the output of CF to the GC as well (make it similar to the DCN)? It maybe that the input from MF alone may not be strong enough, or some sort of encoding might be needed to transform the given input.

2. Can you increase the weights between MF and GC by factors of 10, 100, or 1000? I would like to see some voltage changes in the GC part of the voltage figure.

[289848771]

感谢您提供这些数字。 GC 部分似乎没有从 MF 获取任何信息。由于 PC 和 DCN 看起来非常活跃，这表明它们的活动来自 CF。 因为 GC 没有处于活动状态，所以似乎 MF 和 GC 之间的连接太弱，或者 MF 要传输的信息较少，或者 MF 输入的编码没有引起足够的活动。

建议：

1. 由于 DCN 已经接收来自 MF 和 CF 的输入，您能否将 CF 的输出也路由到 GC（使其类似于 DCN）？可能仅来自 MF 的输入可能不够强大，或者可能需要某种编码来转换给定的输入。
2. 您可以将 MF 和 GC 之间的权重增加 10、100 或 1000 倍吗？我希望看到电压图的 GC 部分的一些电压变化。

Thank you very much for your detailed advice. I apologize for the delay in replying.

I would definitely also try routing the output of the CF to the GC, similar to a DCN. Additionally, I will try increasing the weight between MF and GC by a factor of 10, 100, and 1000 to observe the voltage changes in the GC portion of the voltage plot.

It is worth mentioning that due to my algorithm settings, only 7 of the 70,000 neurons in the GC at each time step can and must be activated (the algorithm requires that the sparse activation characteristics of the GC be maintained). I'm thinking about whether this setting will cause the MF-GC connection you said to be too weak. If it is necessary to maintain such a setting of sparse activation performance of GC, is there any way to improve the activity of GC so that the activity of PC is more affected by GC instead of CF.

Thank you very much for your help and I apologize again for my delayed reply.

[Hananel-Hazan]

It is worth mentioning that due to my algorithm settings, only 7 of the 70,000 neurons in the GC at each time step can and must be activated (the algorithm requires that the sparse activation characteristics of the GC be maintained). I'm thinking about whether this setting will cause the MF-GC connection you said to be too weak. If it is necessary to maintain such a setting of sparse activation performance of GC, is there any way to improve the activity of GC so that the activity of PC is more affected by GC instead of CF.

If only seven neurons in the GC can be active at each step, there might be a need for an echo chamber to enhance the activity. This echo chamber could be a liquid state machine, which is a type of recurrent network, positioned between the GC and PC.

[289848771]

It is worth mentioning that due to my algorithm settings, only 7 of the 70,000 neurons in the GC at each time step can and must be activated (the algorithm requires that the sparse activation characteristics of the GC be maintained). I'm thinking about whether this setting will cause the MF-GC connection you said to be too weak. If it is necessary to maintain such a setting of sparse activation performance of GC, is there any way to improve the activity of GC so that the activity of PC is more affected by GC instead of CF.

If only seven neurons in the GC can be active at each step, there might be a need for an echo chamber to enhance the activity. This echo chamber could be a liquid state machine, which is a type of recurrent network, positioned between the GC and PC.

Thank you very much for your suggestion, I feel very inspired by this suggestion. I will try the method you provided.

In addition, I would like to ask you: Can the echo chamber you are talking about be a recurrent network based on a spiking neural network? Does BindsNET support building recurrent networks? Or does BindsNET have a library for developing recurrent networks?

Thanks again for your help.

[Hananel-Hazan]

BindsNET support recurrent connection, see the reservoir example

[289848771]

BindsNET support recurrent connection, see the reservoir example

Thank you so much, I will try it next.