Closed jimzhou112 closed 3 years ago
If you want to enable power measurements in the snntoolbox, you can set these parameters in the [loihi]
section:
profile_performance = True
partition = nahuku32_2h
node = ncl-ext-ghrd-01
An example how the plotting functions can be used is here: https://github.com/intel-nrc-ecosystem/models/blob/master/nxsdk_modules_ncl/dnn/tutorials/c_slayer_nxtf_nmnist.py
Another script that illustrates the use of probes for energy measurements is here: https://github.com/intel-nrc-ecosystem/models/blob/master/nxsdk_modules_ncl/dnn/tutorials/b_image_classification_cifar.ipynb
Hi,
First off, apologies for the inconvienance caused by all the questions. I've made great progress in converting a CNN to SNN and measuring its metrics such as energy, power, and latency per inference. However, I find that the above solution only works if the timesteps is sufficiently large (around 200 timesteps). Otherwise, I get the following stacktrace:
Traceback (most recent call last):
File "main.py", line 220, in <module>
main(config_filepath)
File "/homes/jimxzhou/python3_venv/lib/python3.8/site-packages/snntoolbox/bin/run.py", line 31, in main
run_pipeline(config)
File "/homes/jimxzhou/python3_venv/lib/python3.8/site-packages/snntoolbox/bin/utils.py", line 145, in run_pipeline
results = run(spiking_model, **testset)
File "/homes/jimxzhou/python3_venv/lib/python3.8/site-packages/snntoolbox/bin/utils.py", line 220, in wrapper
results.append(run_single(snn, **testset))
File "/homes/jimxzhou/python3_venv/lib/python3.8/site-packages/snntoolbox/bin/utils.py", line 142, in run
return snn.run(**test_set)
File "/homes/jimxzhou/python3_venv/lib/python3.8/site-packages/snntoolbox/simulation/utils.py", line 606, in run
output_b_l_t = self.simulate(**data_batch_kwargs)
File "/homes/jimxzhou/models/nxsdk_modules_ncl/snntoolbox/nx_backend.py", line 541, in simulate
self.composed_snn.finishRun()
File "/homes/jimxzhou/python3_venv/lib/python3.8/site-packages/nxsdk/composable/model.py", line 185, in finishRun
self._board.finishRun()
File "/homes/jimxzhou/python3_venv/lib/python3.8/site-packages/nxsdk/arch/base/nxboard.py", line 313, in finishRun
return self.executor.finish()
File "/homes/jimxzhou/python3_venv/lib/python3.8/site-packages/nxsdk/driver/executor.py", line 122, in finish
self._notifyListeners(ExecutionEventEnum.POST_EXECUTION)
File "/homes/jimxzhou/python3_venv/lib/python3.8/site-packages/nxsdk/driver/executor.py", line 157, in _notifyListeners
listener.postExecution()
File "/homes/jimxzhou/python3_venv/lib/python3.8/site-packages/nxsdk/driver/listeners/composite_monitor.py", line 56, in postExecution
[m.postExecution() for m in self._collection.values()]
File "/homes/jimxzhou/python3_venv/lib/python3.8/site-packages/nxsdk/driver/listeners/composite_monitor.py", line 56, in <listcomp>
[m.postExecution() for m in self._collection.values()]
File "/homes/jimxzhou/python3_venv/lib/python3.8/site-packages/nxsdk/driver/listeners/monitors/performance_monitor.py", line 44, in postExecution
self._energyTimeMonitor.updateProbes()
File "/homes/jimxzhou/python3_venv/lib/python3.8/site-packages/nxsdk/graph/nxenergy_time.py", line 934, in updateProbes
prb._updateProbe(probeData)
File "/homes/jimxzhou/python3_venv/lib/python3.8/site-packages/nxsdk/graph/nxenergy_time.py", line 390, in _updateProbe
powerProbeData = self.probeDataFactory.getPowerDataProbe(
File "/homes/jimxzhou/python3_venv/lib/python3.8/site-packages/nxsdk/graph/nxpower.py", line 445, in getPowerDataProbe
return NahukuPowerProbeDataPerRun(*args, **kwargs)
File "/homes/jimxzhou/python3_venv/lib/python3.8/site-packages/nxsdk/graph/nxpower.py", line 383, in __init__
self._unpackPowerData()
File "/homes/jimxzhou/python3_venv/lib/python3.8/site-packages/nxsdk/graph/nxpower.py", line 431, in _unpackPowerData
raise ValueError("Insufficient number of power readings. "
ValueError: Insufficient number of power readings. Please run longer to collect more data points.
Since I would ideally like to use a timestep value that is smaller than the values permitted by the energy probe (64 timesteps is when the SNN accuracy becomes saturated), is there any way around this error to measure the energy/power/latency for smaller timesteps?
Thank you!!!
What I'd do in that case is run the profiling outside of the toolbox, after conversion, in a separate script. The steps then are something like this:
This pipeline works better because you can string all the testsamples together, so to the probe it appears as one long run. See example code below.
import os
import numpy as np
from tensorflow.keras.datasets import mnist
from nxsdk.api.enums.api_enums import ProbeParameter
from nxsdk.graph.monitor.probes import PerformanceProbeCondition
from nxsdk_modules_ncl.dnn.src.dnn_layers import loadNxModel
from nxsdk_modules_ncl.dnn.composable.composable_dnn import ComposableDNN
from nxsdk_modules.input_generator.input_generator import InputGenerator
from nxsdk.composable.model import Model
from nxsdk_modules_ncl.snntoolbox.nx_backend import print_performance, \
save_performance_stats
from snntoolbox.simulation.plotting import plot_execution_time_probe, \
plot_energy_probe, plot_power_probe
os.environ['PARTITION'] = "nahuku32_2h"
os.environ['BOARD'] = "ncl-ext-ghrd-01"
buffer_size = 1024
bin_size = 16
num_steps_per_sample = 100
num_samples = int(2 * buffer_size * bin_size / num_steps_per_sample)
(x_train, y_train), (x_test, y_test) = mnist.load_data()
log_dir = '/homes/rbodo/Repositories/snntoolbox_experiments/mnist/cnn/loihi/' \
'log/gui/04'
outdir = os.path.join(log_dir, 'performance_nxtf')
os.makedirs(outdir, exist_ok=True)
model_path = os.path.join(log_dir, 'model_dumps', 'runnables', 'nxModel.h5')
snn_model = loadNxModel(filepath=model_path, logdir=log_dir, doCompile=True)
cdnn = ComposableDNN(model=snn_model, num_steps_per_img=num_steps_per_sample)
input_generator = InputGenerator(shape=snn_model.input_shape[1:-1],
interval=num_steps_per_sample,
numSnipsPerChip=3)
cdnn.name = "dnn"
input_generator.name = "input"
input_generator.setBiasExp(6)
model = Model('dnn')
model.add(cdnn)
model.add(input_generator)
input_generator.connect(cdnn)
input_generator.processes.inputEncoder.executeAfter(cdnn.processes.reset)
model.compile()
condition = PerformanceProbeCondition(
tStart=buffer_size * bin_size + 1, tEnd=num_samples * num_steps_per_sample,
bufferSize=buffer_size, binSize=bin_size)
performance_probe = model.board.probe(ProbeParameter.ENERGY, condition)
model.start(model.board)
model.run(num_steps_per_sample * num_samples, aSync=True)
print("Sending images...")
for i, x in enumerate(x_test[:num_samples]):
print(i)
model.composables.input.encode(x)
print("Waiting for classification to finish...")
classifications = model.composables.dnn.readout_channel.read(num_samples)
print('{:.2%}'.format(np.mean(np.equal(classifications,
y_test[:num_samples]))))
model.finishRun()
model.disconnect()
stats = model.board.energyTimeMonitor.powerProfileStats
print_performance(stats, num_steps_per_sample)
save_performance_stats(stats, outdir, num_steps_per_sample)
plot_execution_time_probe(outdir, performance_probe)
plot_energy_probe(outdir, performance_probe)
plot_power_probe(outdir, performance_probe)
I am using SNN Toolbox to convert a Keras/TF ANN model to SNN to run on Intel's Loihi chip. One interesting comparison I would like to make is the energy/power consumption since neuromorphic computing is known for its low power consumption.
I noticed that in plotting.py which is the file that plots variables, there is both a function for plotting energy/power (lines 1196-1205), but they are never called.
Is there any support within SNN-Toolbox for either measuring or plotting energy/power? If not, what are some other methods to measure them?
Thanks