sammlapp
commented
2 years ago Because ONNX will require a numeric vector input, the input will logically be either (a) audio samples or (b) a pre-processed 2d representation of the audio such as a spectrogram, potentially with multiple channels. The advantage of passing the audio sample vector (wav) is that all preprocessing parameters will be included in the model; the only thing the user has to get right is the audio sampling rate. Option (b) gives more flexibility because pre-processing does not need to be packaged into the model, but allows more opportunity for pre-processing operations and parameters to be lost or mis-implemented when the model changes hands.
[ ] We should be able to export a model to ONNX so that someone could run predictions using the model without opensoundscape.
[ ] Second, we should be able to load an ONNX model and generate predictions. If this is best done with a simple torch script rather than by implementing something in opensoundscape, that's fine - we can just add documentation of how to do this
[ ] Third, we should be able to load and ONNX model into opensoundscape such that we could re-train (eg, warm-starting) within opensoundscape
It may be necessary or at least logical to use torchaudio to incorporate preprocessing steps into the model, as mentioned in #337
Be aware of the numpy & built-in types caveats for the torch.onnx module