`lava_inference.ipynb` in the official `baseline_solution` is still not working

[haoxiangsnr]

Hi, the lava_inference.ipynb in the official baseline_solution is still not working.

When I run the cell

num_samples = denoised_receiver.num_samples
num_steps = num_samples * (len(noisy) // hop_length + blank_time) + 1

print(f'{num_samples = }')
print(f'{num_steps = }')
# run_config = Loihi2SimCfg(select_tag='fixed_pt')
# run_config = AsyncSimCfg(select_tag='fixed_pt')
run_config = Loihi2SimCfg(exception_proc_model_map=exception_proc_map, select_tag='fixed_pt')
audio_source.run(condition=RunSteps(num_steps=num_steps), run_cfg=run_config)
si_snrs = denoised_receiver.si_snr.get()
base_si_snrs = noisy_receiver.si_snr.get()
audio_source.stop()

The output is:

num_samples = 5
num_steps = 18801
/home/xianghao/mambaforge/envs/lava/lib/python3.8/site-packages/lava/magma/compiler/compiler_graphs.py:868: UserWarning: Cannot import module '<module 'train_sdnn' from '/home/xianghao/proj/IntelNeuromorphicDNSChallenge/baseline_solution/sdnn_delays/train_sdnn.py'>' when searching ProcessModels for Process 'FloatPtAmp'.
  warnings.warn(
/home/xianghao/mambaforge/envs/lava/lib/python3.8/site-packages/lava/magma/compiler/compiler_graphs.py:868: UserWarning: Cannot import module '<module 'train_sdnn' from '/home/xianghao/proj/IntelNeuromorphicDNSChallenge/baseline_solution/sdnn_delays/train_sdnn.py'>' when searching ProcessModels for Process 'DelayBuffer'.
  warnings.warn(
/home/xianghao/mambaforge/envs/lava/lib/python3.8/site-packages/lava/magma/compiler/compiler_graphs.py:868: UserWarning: Cannot import module '<module 'train_sdnn' from '/home/xianghao/proj/IntelNeuromorphicDNSChallenge/baseline_solution/sdnn_delays/train_sdnn.py'>' when searching ProcessModels for Process 'FixedPtAmp'.
  warnings.warn(
/home/xianghao/mambaforge/envs/lava/lib/python3.8/site-packages/lava/magma/compiler/compiler_graphs.py:868: UserWarning: Cannot import module '<module 'train_sdnn' from '/home/xianghao/proj/IntelNeuromorphicDNSChallenge/baseline_solution/sdnn_delays/train_sdnn.py'>' when searching ProcessModels for Process 'STFT'.
  warnings.warn(
/home/xianghao/mambaforge/envs/lava/lib/python3.8/site-packages/lava/magma/compiler/compiler_graphs.py:868: UserWarning: Cannot import module '<module 'train_sdnn' from '/home/xianghao/proj/IntelNeuromorphicDNSChallenge/baseline_solution/sdnn_delays/train_sdnn.py'>' when searching ProcessModels for Process 'AudioReceiver'.
  warnings.warn(
/home/xianghao/mambaforge/envs/lava/lib/python3.8/site-packages/lava/magma/compiler/compiler_graphs.py:868: UserWarning: Cannot import module '<module 'train_sdnn' from '/home/xianghao/proj/IntelNeuromorphicDNSChallenge/baseline_solution/sdnn_delays/train_sdnn.py'>' when searching ProcessModels for Process 'AmplitudeMixer'.
  warnings.warn(
/home/xianghao/mambaforge/envs/lava/lib/python3.8/site-packages/lava/magma/compiler/compiler_graphs.py:868: UserWarning: Cannot import module '<module 'train_sdnn' from '/home/xianghao/proj/IntelNeuromorphicDNSChallenge/baseline_solution/sdnn_delays/train_sdnn.py'>' when searching ProcessModels for Process 'ISTFT'.
  warnings.warn(
/home/xianghao/mambaforge/envs/lava/lib/python3.8/site-packages/lava/magma/compiler/compiler_graphs.py:868: UserWarning: Cannot import module '<module 'train_sdnn' from '/home/xianghao/proj/IntelNeuromorphicDNSChallenge/baseline_solution/sdnn_delays/train_sdnn.py'>' when searching ProcessModels for Process 'Bias'.
  warnings.warn(
/home/xianghao/mambaforge/envs/lava/lib/python3.8/site-packages/lava/magma/compiler/compiler_graphs.py:868: UserWarning: Cannot import module '<module 'train_sdnn' from '/home/xianghao/proj/IntelNeuromorphicDNSChallenge/baseline_solution/sdnn_delays/train_sdnn.py'>' when searching ProcessModels for Process 'AudioSource'.
  warnings.warn(
Encountered Fatal Exception: DelayDense Process 'delays' expects values between 0 and 63 for Loihi, got [[22 22 22 ... 22 22 22]
 [ 0  0  0 ...  0  0  0]
 [ 4  4  4 ...  4  4  4]
 ...
 [12 12 12 ... 12 12 12]
 [17 17 17 ... 17 17 17]
 [ 0  0  0 ...  0  0  0]].
Traceback: 
Traceback (most recent call last):
  File "/home/xianghao/mambaforge/envs/lava/lib/python3.8/site-packages/lava/magma/runtime/runtime.py", line 95, in target_fn
    actor.start(*args, **kwargs)
  File "/home/xianghao/mambaforge/envs/lava/lib/python3.8/site-packages/lava/magma/core/model/py/model.py", line 93, in start
    self.run()
  File "/home/xianghao/mambaforge/envs/lava/lib/python3.8/site-packages/lava/magma/core/model/py/model.py", line 232, in run
    raise inst
  File "/home/xianghao/mambaforge/envs/lava/lib/python3.8/site-packages/lava/magma/core/model/py/model.py", line 218, in run
    self._cmd_handlers[cmd]()
  File "/home/xianghao/mambaforge/envs/lava/lib/python3.8/site-packages/lava/magma/core/model/py/model.py", line 387, in _spike
    self.run_spk()
  File "/home/xianghao/mambaforge/envs/lava/lib/python3.8/site-packages/lava/proc/dense/models.py", line 330, in run_spk
    raise ValueError("DelayDense Process 'delays' expects values "
ValueError: DelayDense Process 'delays' expects values between 0 and 63 for Loihi, got [[22 22 22 ... 22 22 22]
 [ 0  0  0 ...  0  0  0]
 [ 4  4  4 ...  4  4  4]
 ...
...
  File "/home/xianghao/mambaforge/envs/lava/lib/python3.8/site-packages/lava/proc/dense/models.py", line 346, in run_spk
    self.a_buff += (
ValueError: operands could not be broadcast together with shapes (512,63) (512,51) (512,63) 

I have tested the lava-dl==0.3.3 + lava==0.7.0 and lava-dl==0.4.0 + lava==0.8.0. However, the output is the same.

Could you help me to solve it? Thank you very much.

[bamsumit]

@haoxiangsnr thanks for reaching out to us. Please use the latest main branch of both lava and lava-dl. The issue has been resolved there. Apologies for the confusion.

[haoxiangsnr]

Hi @bamsumit I appreciate your previous response. However, the issue I've been experiencing seems to persist. I have completely reset my development environment using the following steps:

git clone git@github.com:IntelLabs/IntelNeuromorphicDNSChallenge.git IntelNeuromorphicDNSChallenge-V2
cd IntelNeuromorphicDNSChallenge-V2

conda create --name lava-main python=3.10
conda activate lava-main
conda install pytorch==1.13.1 torchvision==0.14.1 torchaudio==0.13.1 pytorch-cuda=11.7 -c pytorch -c nvidia

pip install git+https://github.com/lava-nc/lava.git
pip install git+https://github.com/lava-nc/lava-dl.git
pip install -r requirements.txt
pip install pyyaml  # missing dep

cp snr.py baseline_solution/sdnn_delays
cp audio_dataloader.py baseline_solution/sdnn_delays

# modify the value of `path` in args.txt to my data path

However, when I proceed to run this cell

num_samples = denoised_receiver.num_samples
num_steps = num_samples * (len(noisy) // hop_length + blank_time) + 1

print(f'{num_samples = }')
print(f'{num_steps = }')
# run_config = Loihi2SimCfg(select_tag='fixed_pt')
# run_config = AsyncSimCfg(select_tag='fixed_pt')
run_config = Loihi2SimCfg(exception_proc_model_map=exception_proc_map, select_tag='fixed_pt')
audio_source.run(condition=RunSteps(num_steps=num_steps), run_cfg=run_config)
si_snrs = denoised_receiver.si_snr.get()
base_si_snrs = noisy_receiver.si_snr.get()
audio_source.stop()

The cell hangs and does not proceed further, as shown in the attached screenshot.

num_samples = 5
num_steps = 18801
/home/xianghao/mambaforge/envs/lava-main/lib/python3.10/site-packages/lava/magma/compiler/compiler_graphs.py:868: UserWarning: Cannot import module '<module 'train_sdnn' from '/home/xianghao/proj/IntelNeuromorphicDNSChallenge-V2/baseline_solution/sdnn_delays/train_sdnn.py'>' when searching ProcessModels for Process 'FloatPtAmp'.
  warnings.warn(
/home/xianghao/mambaforge/envs/lava-main/lib/python3.10/site-packages/lava/magma/compiler/compiler_graphs.py:868: UserWarning: Cannot import module '<module 'train_sdnn' from '/home/xianghao/proj/IntelNeuromorphicDNSChallenge-V2/baseline_solution/sdnn_delays/train_sdnn.py'>' when searching ProcessModels for Process 'STFT'.
  warnings.warn(
/home/xianghao/mambaforge/envs/lava-main/lib/python3.10/site-packages/lava/magma/compiler/compiler_graphs.py:868: UserWarning: Cannot import module '<module 'train_sdnn' from '/home/xianghao/proj/IntelNeuromorphicDNSChallenge-V2/baseline_solution/sdnn_delays/train_sdnn.py'>' when searching ProcessModels for Process 'ISTFT'.
  warnings.warn(
/home/xianghao/mambaforge/envs/lava-main/lib/python3.10/site-packages/lava/magma/compiler/compiler_graphs.py:868: UserWarning: Cannot import module '<module 'train_sdnn' from '/home/xianghao/proj/IntelNeuromorphicDNSChallenge-V2/baseline_solution/sdnn_delays/train_sdnn.py'>' when searching ProcessModels for Process 'AudioReceiver'.
  warnings.warn(
/home/xianghao/mambaforge/envs/lava-main/lib/python3.10/site-packages/lava/magma/compiler/compiler_graphs.py:868: UserWarning: Cannot import module '<module 'train_sdnn' from '/home/xianghao/proj/IntelNeuromorphicDNSChallenge-V2/baseline_solution/sdnn_delays/train_sdnn.py'>' when searching ProcessModels for Process 'DelayBuffer'.
  warnings.warn(
/home/xianghao/mambaforge/envs/lava-main/lib/python3.10/site-packages/lava/magma/compiler/compiler_graphs.py:868: UserWarning: Cannot import module '<module 'train_sdnn' from '/home/xianghao/proj/IntelNeuromorphicDNSChallenge-V2/baseline_solution/sdnn_delays/train_sdnn.py'>' when searching ProcessModels for Process 'Bias'.
  warnings.warn(
/home/xianghao/mambaforge/envs/lava-main/lib/python3.10/site-packages/lava/magma/compiler/compiler_graphs.py:868: UserWarning: Cannot import module '<module 'train_sdnn' from '/home/xianghao/proj/IntelNeuromorphicDNSChallenge-V2/baseline_solution/sdnn_delays/train_sdnn.py'>' when searching ProcessModels for Process 'FixedPtAmp'.
  warnings.warn(
/home/xianghao/mambaforge/envs/lava-main/lib/python3.10/site-packages/lava/magma/compiler/compiler_graphs.py:868: UserWarning: Cannot import module '<module 'train_sdnn' from '/home/xianghao/proj/IntelNeuromorphicDNSChallenge-V2/baseline_solution/sdnn_delays/train_sdnn.py'>' when searching ProcessModels for Process 'AmplitudeMixer'.
  warnings.warn(
/home/xianghao/mambaforge/envs/lava-main/lib/python3.10/site-packages/lava/magma/compiler/compiler_graphs.py:868: UserWarning: Cannot import module '<module 'train_sdnn' from '/home/xianghao/proj/IntelNeuromorphicDNSChallenge-V2/baseline_solution/sdnn_delays/train_sdnn.py'>' when searching ProcessModels for Process 'AudioSource'.
  warnings.warn(

Even when I use trained_folder = 'Trained_nodelay', there is still no output.

I would greatly appreciate any further guidance or assistance you could provide to resolve this issue.

[janek123p]

Despite using the latest lava and lava-dl versions, I am facing the same issue.

[bamsumit]

@haoxiangsnr do you still get the same exception?

Encountered Fatal Exception: DelayDense Process 'delays' expects values between 0 and 63 for Loihi, got

It looks like it is just hanging now. Have you tried using Jupyter browser session instead of VSCode? The code takes a while to run and usually VSCode session times out if the computation takes a while.