How to Measure Energy/Power Consumption?

[jimzhou112]

I am using SNN Toolbox to convert a Keras/TF ANN model to SNN to run on Intel's Loihi chip. One interesting comparison I would like to make is the energy/power consumption since neuromorphic computing is known for its low power consumption.

I noticed that in plotting.py which is the file that plots variables, there is both a function for plotting energy/power (lines 1196-1205), but they are never called.

Is there any support within SNN-Toolbox for either measuring or plotting energy/power? If not, what are some other methods to measure them?

Thanks

[rbodo]

If you want to enable power measurements in the snntoolbox, you can set these parameters in the [loihi] section:

profile_performance = True
partition = nahuku32_2h
node = ncl-ext-ghrd-01

An example how the plotting functions can be used is here: https://github.com/intel-nrc-ecosystem/models/blob/master/nxsdk_modules_ncl/dnn/tutorials/c_slayer_nxtf_nmnist.py

Another script that illustrates the use of probes for energy measurements is here: https://github.com/intel-nrc-ecosystem/models/blob/master/nxsdk_modules_ncl/dnn/tutorials/b_image_classification_cifar.ipynb

[jimzhou112]

Hi,

First off, apologies for the inconvienance caused by all the questions. I've made great progress in converting a CNN to SNN and measuring its metrics such as energy, power, and latency per inference. However, I find that the above solution only works if the timesteps is sufficiently large (around 200 timesteps). Otherwise, I get the following stacktrace:

Traceback (most recent call last):
  File "main.py", line 220, in <module>
    main(config_filepath)
  File "/homes/jimxzhou/python3_venv/lib/python3.8/site-packages/snntoolbox/bin/run.py", line 31, in main
    run_pipeline(config)
  File "/homes/jimxzhou/python3_venv/lib/python3.8/site-packages/snntoolbox/bin/utils.py", line 145, in run_pipeline
    results = run(spiking_model, **testset)
  File "/homes/jimxzhou/python3_venv/lib/python3.8/site-packages/snntoolbox/bin/utils.py", line 220, in wrapper
    results.append(run_single(snn, **testset))
  File "/homes/jimxzhou/python3_venv/lib/python3.8/site-packages/snntoolbox/bin/utils.py", line 142, in run
    return snn.run(**test_set)
  File "/homes/jimxzhou/python3_venv/lib/python3.8/site-packages/snntoolbox/simulation/utils.py", line 606, in run
    output_b_l_t = self.simulate(**data_batch_kwargs)
  File "/homes/jimxzhou/models/nxsdk_modules_ncl/snntoolbox/nx_backend.py", line 541, in simulate
    self.composed_snn.finishRun()
  File "/homes/jimxzhou/python3_venv/lib/python3.8/site-packages/nxsdk/composable/model.py", line 185, in finishRun
    self._board.finishRun()
  File "/homes/jimxzhou/python3_venv/lib/python3.8/site-packages/nxsdk/arch/base/nxboard.py", line 313, in finishRun
    return self.executor.finish()
  File "/homes/jimxzhou/python3_venv/lib/python3.8/site-packages/nxsdk/driver/executor.py", line 122, in finish
    self._notifyListeners(ExecutionEventEnum.POST_EXECUTION)
  File "/homes/jimxzhou/python3_venv/lib/python3.8/site-packages/nxsdk/driver/executor.py", line 157, in _notifyListeners
    listener.postExecution()
  File "/homes/jimxzhou/python3_venv/lib/python3.8/site-packages/nxsdk/driver/listeners/composite_monitor.py", line 56, in postExecution
    [m.postExecution() for m in self._collection.values()]
  File "/homes/jimxzhou/python3_venv/lib/python3.8/site-packages/nxsdk/driver/listeners/composite_monitor.py", line 56, in <listcomp>
    [m.postExecution() for m in self._collection.values()]
  File "/homes/jimxzhou/python3_venv/lib/python3.8/site-packages/nxsdk/driver/listeners/monitors/performance_monitor.py", line 44, in postExecution
    self._energyTimeMonitor.updateProbes()
  File "/homes/jimxzhou/python3_venv/lib/python3.8/site-packages/nxsdk/graph/nxenergy_time.py", line 934, in updateProbes
    prb._updateProbe(probeData)
  File "/homes/jimxzhou/python3_venv/lib/python3.8/site-packages/nxsdk/graph/nxenergy_time.py", line 390, in _updateProbe
    powerProbeData = self.probeDataFactory.getPowerDataProbe(
  File "/homes/jimxzhou/python3_venv/lib/python3.8/site-packages/nxsdk/graph/nxpower.py", line 445, in getPowerDataProbe
    return NahukuPowerProbeDataPerRun(*args, **kwargs)
  File "/homes/jimxzhou/python3_venv/lib/python3.8/site-packages/nxsdk/graph/nxpower.py", line 383, in __init__
    self._unpackPowerData()
  File "/homes/jimxzhou/python3_venv/lib/python3.8/site-packages/nxsdk/graph/nxpower.py", line 431, in _unpackPowerData
    raise ValueError("Insufficient number of power readings. "
ValueError: Insufficient number of power readings. Please run longer to collect more data points.

Since I would ideally like to use a timestep value that is smaller than the values permitted by the energy probe (64 timesteps is when the SNN accuracy becomes saturated), is there any way around this error to measure the energy/power/latency for smaller timesteps?

Thank you!!!

[rbodo]

What I'd do in that case is run the profiling outside of the toolbox, after conversion, in a separate script. The steps then are something like this:

1. Load a converted SNN from disk
2. Set up a performance probe
3. Run the network on a large batch of test samples
4. Print results

This pipeline works better because you can string all the testsamples together, so to the probe it appears as one long run. See example code below.

[rbodo]

import os
import numpy as np
from tensorflow.keras.datasets import mnist

from nxsdk.api.enums.api_enums import ProbeParameter
from nxsdk.graph.monitor.probes import PerformanceProbeCondition
from nxsdk_modules_ncl.dnn.src.dnn_layers import loadNxModel
from nxsdk_modules_ncl.dnn.composable.composable_dnn import ComposableDNN
from nxsdk_modules.input_generator.input_generator import InputGenerator
from nxsdk.composable.model import Model
from nxsdk_modules_ncl.snntoolbox.nx_backend import print_performance, \
    save_performance_stats
from snntoolbox.simulation.plotting import plot_execution_time_probe, \
    plot_energy_probe, plot_power_probe

os.environ['PARTITION'] = "nahuku32_2h"
os.environ['BOARD'] = "ncl-ext-ghrd-01"

buffer_size = 1024
bin_size = 16

num_steps_per_sample = 100
num_samples = int(2 * buffer_size * bin_size / num_steps_per_sample)

(x_train, y_train), (x_test, y_test) = mnist.load_data()

log_dir = '/homes/rbodo/Repositories/snntoolbox_experiments/mnist/cnn/loihi/' \
          'log/gui/04'
outdir = os.path.join(log_dir, 'performance_nxtf')
os.makedirs(outdir, exist_ok=True)

model_path = os.path.join(log_dir, 'model_dumps', 'runnables', 'nxModel.h5')

snn_model = loadNxModel(filepath=model_path, logdir=log_dir, doCompile=True)

cdnn = ComposableDNN(model=snn_model, num_steps_per_img=num_steps_per_sample)

input_generator = InputGenerator(shape=snn_model.input_shape[1:-1],
                                 interval=num_steps_per_sample,
                                 numSnipsPerChip=3)

cdnn.name = "dnn"
input_generator.name = "input"

input_generator.setBiasExp(6)

model = Model('dnn')
model.add(cdnn)
model.add(input_generator)
input_generator.connect(cdnn)
input_generator.processes.inputEncoder.executeAfter(cdnn.processes.reset)

model.compile()

condition = PerformanceProbeCondition(
    tStart=buffer_size * bin_size + 1, tEnd=num_samples * num_steps_per_sample,
    bufferSize=buffer_size, binSize=bin_size)
performance_probe = model.board.probe(ProbeParameter.ENERGY, condition)

model.start(model.board)
model.run(num_steps_per_sample * num_samples, aSync=True)

print("Sending images...")
for i, x in enumerate(x_test[:num_samples]):
    print(i)
    model.composables.input.encode(x)

print("Waiting for classification to finish...")
classifications = model.composables.dnn.readout_channel.read(num_samples)

print('{:.2%}'.format(np.mean(np.equal(classifications,
                                       y_test[:num_samples]))))

model.finishRun()
model.disconnect()

stats = model.board.energyTimeMonitor.powerProfileStats
print_performance(stats, num_steps_per_sample)
save_performance_stats(stats, outdir, num_steps_per_sample)
plot_execution_time_probe(outdir, performance_probe)
plot_energy_probe(outdir, performance_probe)
plot_power_probe(outdir, performance_probe)