Open marlow77 opened 1 year ago
hyakuchiki
commented
1 year ago The problem seems to be that data.raw_dir expects a folder that wave files are in, not a wav file.
So, in your case it should probably be data.raw_dir='C:/Users/marlo/anaconda3/envs/RealtimeAudioDDSPv2/realtimeDDSP-master/data/
That was a bit unclear in the descriptions, sorry for the confusion.
Sorry about the colab as well, it seems that Colab is getting less and less generous these days...
marlow77
commented
1 year ago Unfortunately, I get the same issue, with both the standard data in the folder and in separate tests with other wav files. I've attached a folder of samples in a zip file which I am trying to make into a model.
Maybe this is a syntax issue? I am on Windows 10, using a virtual env under conda. I have tried different directions of slashes, single and double quotation marks, and many other things, but it doesn't change the output errors below.
The specific trace information is as follows:
(RealtimeAudioDDSPv2) C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\realtimeDDSP-master>python train.py name=Alto_Sax data.raw_dir='C:/Users/marlo/anaconda3/envs/RealtimeAudioDDSPv2/realtimeDDSP-master/data/' data.db_path='C:/Users/marlo/anaconda3/envs/RealtimeAudioDDSPv2/realtimeDDSP-master/temp_db_dir'
Traceback (most recent call last):
File "C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\lib\
Sax_Alto.zip
site-packages\hydra_internal\utils.py", line 213, in run_and_report
return func()
File "C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\lib\site-packages\hydra_internal\utils.py", line 453, in
(RealtimeAudioDDSPv2) C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\realtimeDDSP-master>
hyakuchiki
commented
1 year ago I don't have a windows environment to test this, but I think this error is caused by the colon in the directory (C:). Can you try escaping the colon with a backslash? Or just specifying them as a relative directory?
marlow77
commented
1 year ago I can't seem to figure out the syntax. I have tried every permutation that I can think of without any success. This relative path SHOULD work, but it doesn't, even though it is in the same directory, just one folder: python train.py name='Alto_Sax' data.raw_dir='/Sax_Alto/' data.db_path='/temp_db_dir/'
I've tried the escape character slash and reversed the direction of the slashes as well as silly complicated syntax for hydra that could even look like:
As you can see by the stack trace below, I tried an example with folders called 'typo' and 'typo2' just to determine syntax or file location errors, which is the same errors I get with the correct syntax. For this reason, I feel like it is either the files that are the problem or the syntax to their location. However, I have used the same data directory 'data' which has both wav and mp3 files in it and it gives the same errors. For this reason, I believe it is simply a syntax issue, but I can't figure out what it should be! I've attached a picture of the path to the folder that has wav files and the location of the train.py file that it should be relative to. I also checked the default values of the wav files which was 48Khz, which is the format of the wav files too.
(RealtimeAudioDDSPv2) C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\realtimeDDSP-master>python train.py name=Alto_Sax data.raw_dir=typo data.db_path=typo2
Traceback (most recent call last):
File "C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\lib\site-packages\hydra_internal\utils.py", line 213, in run_and_report
return func()
File "C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\lib\site-packages\hydra_internal\utils.py", line 453, in
(RealtimeAudioDDSPv2) C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\realtimeDDSP-master>python train.py name='Alto_Sax' data.raw_dir='/Sax_Alto/' data.db_path='/temp_db_dir/'
Traceback (most recent call last):
File "C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\lib\site-packages\hydra_internal\utils.py", line 213, in run_and_report
return func()
File "C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\lib\site-packages\hydra_internal\utils.py", line 453, in
(RealtimeAudioDDSPv2) C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\realtimeDDSP-master>
hyakuchiki
commented
1 year ago Thanks for the thorough testing. I'm not quite sure why the first example (typo, typo2) caused the issue with yaml parsing.
Can you try putting the quotation marks around the equation like 'data.raw_dir=ABS_PATH_TO_REPO/data/Sax_Alto'?
Also I forgot that relative paths don't work well with hydra. Basically, the current directory changes to logs/RUN_NAME, so relative paths might work like ../../data and ../../temp_db_dir.
As a last resort, can you also try putting in the directory in the config file by changing raw_dir: null and db_path: null in configs/data/slice.yaml?
marlow77
commented
1 year ago No luck on the following, putting quotation marks around the 'data.raw_dir' portion of the commandline as shown from the stack below:
(RealtimeAudioDDSPv2) C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\realtimeDDSP-master>python train.py name='Alto_Sax' 'data.raw_dir=C:/Users/marlo/anaconda3/envs/RealtimeAudioDDSPv2/realtimeDDSP-master/Sax_Alto/' data.db_path='C:/Users/marlo/anaconda3/envs/RealtimeAudioDDSPv2/realtimeDDSP-master/temp_db_dir/' LexerNoViableAltException: 'data.raw_dir=C:/Users/marlo/anaconda3/envs/RealtimeAudioDDSPv2/realtimeDDSP-master/Sax_Alto/' ^ See https://hydra.cc/docs/next/advanced/override_grammar/basic for details
Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.
(RealtimeAudioDDSPv2) C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\realtimeDDSP-master>
marlow77
commented
1 year ago No joy with the relative paths method
../../data and ../../temp_db_dir
as seen below:
(RealtimeAudioDDSPv2) C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\realtimeDDSP_master>python train.py name='Alto_Sax' data.raw_dir='../../Sax_Alto/' data.db_path='../../temp_db_dir/'
Traceback (most recent call last):
File "C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\lib\site-packages\hydra_internal\utils.py", line 213, in run_and_report
return func()
File "C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\lib\site-packages\hydra_internal\utils.py", line 453, in
(RealtimeAudioDDSPv2) C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\realtimeDDSP_master>
marlow77
commented
1 year ago I modified the null values in the slice.yaml file to be:
target: diffsynth.data.SliceDataset
raw_dir: C:/Users/marlo/anaconda3/envs/RealtimeAudioDDSPv2/realtimeDDSP-master/Sax_Alto db_path: C:/Users/marlo/anaconda3/envs/RealtimeAudioDDSPv2/realtimeDDSP_master/temp_db_dir sample_rate: ${sample_rate} length: 1.0 frame_rate: 50 f0_range: [32, 2000] f0_viterbi: true
But it gave the same issue in the stack as below, with no mention of issues in the slice.yaml file:
(RealtimeAudioDDSPv2) C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\realtimeDDSP_master>python train.py name=Alto_Sax
Traceback (most recent call last):
File "C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\lib\site-packages\hydra_internal\utils.py", line 213, in run_and_report
return func()
File "C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\lib\site-packages\hydra_internal\utils.py", line 453, in
(RealtimeAudioDDSPv2) C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\realtimeDDSP_master>
hyakuchiki
commented
1 year ago There might be some weird issue with hydra and yaml on some window shells. I'll try to get a windows environment tomorrow and test this. Just one last thing, for the last one with the slice.yaml file, can you try putting single quotes around the path name?
marlow77
commented
1 year ago I manually edited the slice.yaml file with relative paths first (unsuccessful) and then the following (also unsuccessful):
target: diffsynth.data.SliceDataset
raw_dir: 'C:/Users/marlo/anaconda3/envs/RealtimeAudioDDSPv2/realtimeDDSP-master/Sax_Alto' db_path: 'C:/Users/marlo/anaconda3/envs/RealtimeAudioDDSPv2/realtimeDDSP_master/temp_db_dir' sample_rate: ${sample_rate} length: 1.0 frame_rate: 50 f0_range: [32, 2000] f0_viterbi: true
Stack Trace output (the same):
(RealtimeAudioDDSPv2) C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\realtimeDDSP_master>python train.py name=Alto_Sax
Traceback (most recent call last):
File "C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\lib\site-packages\hydra_internal\utils.py", line 213, in run_and_report
return func()
File "C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\lib\site-packages\hydra_internal\utils.py", line 453, in
(RealtimeAudioDDSPv2) C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\realtimeDDSP_master>
marlow77
commented
1 year ago Ok. So, I had some success. I reinstalled everything and found out that the relative path puts the files in the first layer of the logs folder as you had mentioned. So I moved the wav folder to that directory and got further in the process. Now the problem is that it seems to require a CUDA device. I had assumed it would default to CPU in the absence of an NVIDIA card. Is there a way to set it to use the CPU vs. the GPU? I know it will take MUCH longer.
Here is the general default without hydra stack trace set to 1:
(RealtimeAudioDDSPv2) C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\realtimeDDSP_master>python train.py name='Alto_Sax' data.raw_dir='../../Sax_Alto/' data.db_path='../../temp_db_dir/' [2023-02-21 07:19:40,879][pytorch_lightning.utilities.seed][INFO] - Global seed set to 0 Starting Preprocessing. 0%| | 0/132 [00:12<?, ?it/s] Error executing job with overrides: ["name='Alto_Sax'", "data.raw_dir='../../Sax_Alto/'", "data.db_path='../../temp_db_dir/'"] Error in call to target 'diffsynth.data.SliceDataset': RuntimeError('Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from http://www.nvidia.com/Download/index.aspx') full_key: data
Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.
(RealtimeAudioDDSPv2) C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\realtimeDDSP_master>
Also, here are all of my packages that I've installed so far:
(RealtimeAudioDDSPv2) C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\realtimeDDSP_master>pip list Package Version
absl-py 1.4.0 aiohttp 3.8.4 aiosignal 1.3.1 antlr4-python3-runtime 4.9.3 appdirs 1.4.4 async-timeout 4.0.2 attrs 22.2.0 audioread 3.0.0 cachetools 5.3.0 certifi 2022.12.7 cffi 1.15.1 charset-normalizer 3.0.1 colorama 0.4.6 cycler 0.11.0 decorator 5.1.1 fonttools 4.38.0 frozenlist 1.3.3 fsspec 2023.1.0 google-auth 2.16.0 google-auth-oauthlib 0.4.6 grpcio 1.51.1 hydra-core 1.2.0 idna 3.4 importlib-metadata 6.0.0 joblib 1.2.0 jsonschema 4.17.3 kiwisolver 1.4.4 librosa 0.9.1 llvmlite 0.39.1 lmdb 1.3.0 Markdown 3.4.1 MarkupSafe 2.1.2 matplotlib 3.5.3 multidict 6.0.4 neutone-sdk 1.0.1 numba 0.56.4 numpy 1.21.6 oauthlib 3.2.2 omegaconf 2.3.0 packaging 23.0 Pillow 9.4.0 pip 22.3.1 pooch 1.6.0 protobuf 3.20.1 pyasn1 0.4.8 pyasn1-modules 0.2.8 pycparser 2.21 pyDeprecate 0.3.2 pyparsing 3.0.9 pyrsistent 0.19.3 python-dateutil 2.8.2 pytorch-lightning 1.6.5 PyYAML 6.0 requests 2.28.2 requests-oauthlib 1.3.1 resampy 0.4.2 rsa 4.9 scikit-learn 1.2.1 scipy 1.10.0 setuptools 65.6.3 six 1.16.0 soundfile 0.12.1 tensorboard 2.12.0 tensorboard-data-server 0.7.0 tensorboard-plugin-wit 1.8.1 threadpoolctl 3.1.0 torch 1.11.0+cu113 torchaudio 0.11.0+cu113 torchcrepe 0.0.17 torchmetrics 0.11.1 tqdm 4.64.1 typing_extensions 4.5.0 urllib3 1.26.14 Werkzeug 2.2.3 wheel 0.37.1 wincertstore 0.2 yarl 1.8.2 zipp 3.13.0
(RealtimeAudioDDSPv2) C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\realtimeDDSP_master>
hyakuchiki
commented
1 year ago Thanks for the details. I couldn't recreate your initial errors with the argument syntax in Windows, but I'll look into it again later.
I hadn't considered training on CPUs but it seems somewhat feasible.
I just made some commits to this repository that enables CPU training so please pull that and try it out.
Just add trainer.gpus=0 to tell the program the GPU isn't available. You may also want to set batch_size=8 or something lower than default.
marlow77
commented
1 year ago Thank you! I am excited to try it out! I will report back how it goes.
marlow77
commented
1 year ago Ok. So the checkpoint files are being written and I wanted to test out creating the model and I had issues in doing so.
I renamed the 'epoch=42-step=5110.ckpt' file to '42.ckpt' to avoid issues with hydra and then entered the following commandline input, thinking that it should work similarly to the initial files structure for building the checkpoints:
python export.py '../../Sax_Alto/42.ckpt' Alto_Sax_Model
(RealtimeAudioDDSPv2) C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\realtimeDDSP-master>python export.py '../../Sax_Alto/42.ckpt' Alto_Sax_Model
Traceback (most recent call last):
File "C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\realtimeDDSP-master\export.py", line 155, in
What am I doing wrong? Can you help me to put the checkpoint file in the right place to make the model?
Thank you in advance.
hyakuchiki
commented
1 year ago The export script doesn't use hydra, so the current working directory is "C:/Users/marlo/anaconda3/envs/RealtimeAudioDDSPv2/realtimeDDSP-master/" and you can just point to it as 'Sax_Alto/42.ckpt' I think.
marlow77
commented
1 year ago Ok. That seemed to work except it generated an .mp3 file error:
(RealtimeAudioDDSPv2) C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\realtimeDDSP-master>python export.py 42.ckpt Alto_Sax_Model
100%|███████████████████████████████████████████████████████████████████████████████████████████| 79/79 [00:23<00:00, 3.39it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████| 81/81 [00:22<00:00, 3.65it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████| 52/52 [00:17<00:00, 2.92it/s]
INFO:neutone_sdk.utils:Converting model to torchscript...
INFO:neutone_sdk.utils:Extracting metadata...
INFO:neutone_sdk.utils:Running model on audio samples...
Traceback (most recent call last):
File "C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\realtimeDDSP-master\export.py", line 175, in
(RealtimeAudioDDSPv2) C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\realtimeDDSP-master>
marlow77
commented
1 year ago And with this, I don't think it created the model.
hyakuchiki
commented
1 year ago Alright... torchaudio on windows doesn't seem to support loading of mp3s...
support on windows · Issue #1178 · pytorch/audio · GitHub
I'll make a change to use wav/flac/ogg files in the future, but in the mean time, can you use your own audio files by specifying them like --sounds path/to/wavefile.wav? wav/flac/ogg should be supported fine on windows.
marlow77
commented
1 year ago I thought the same thing, but there weren't any .mp3 files in the sample folder.
I attempted to specify the .wav file but it gives me this error:
(RealtimeAudioDDSPv2) C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\realtimeDDSP_master>python train.py name=Alto_Sax data.raw_dir='../../Alto_Sax_Sample_Folder/Alto_Sax.wav' data.db_path='../../temp_db_dir/' trainer.gpus=0
Traceback (most recent call last):
File "C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\lib\site-packages\hydra_internal\utils.py", line 213, in run_and_report
return func()
File "C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\lib\site-packages\hydra_internal\utils.py", line 453, in
So I removed the specific wav file name and only used the folder with a single wav file in it and am trying it again. I think the mp3 part is from something else. The data folder had mp3 files in it from as an example in the main directory, but I did not train on them at all nor did I reference them. Let me know if you have any other ideas if it gives me the same mp3 error after this run.
Thanks!
hyakuchiki
commented
1 year ago Sorry for the confusion, I forgot to mention that this error above is with the export script and doesn't have anything to do with the training script.
So the export script needs a example input sound so that you can see the effects of the model easily, and the export script is setup to use mp3 files for that example input (data/*.mp3) which was causing problems on windows.
I've made a commit that changes the .mp3 files to .flac files so that it won't cause any problems on windows. You can git pull origin master right now and export.py should work fine. Let me know if something goes wrong.
marlow77
commented
1 year ago Downloaded and moved the files. I identified that your files were FLAC files, but it still gives the mp3 error here:
(RealtimeAudioDDSPv2) C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\realtimeDDSP-masterv2>python export.py 42.ckpt Alto_Sax_Model
100%|██████████████████████████████████████████████████████████████████████████████████| 79/79 [00:19<00:00, 3.96it/s]
100%|██████████████████████████████████████████████████████████████████████████████████| 81/81 [00:17<00:00, 4.53it/s]
100%|██████████████████████████████████████████████████████████████████████████████████| 52/52 [00:13<00:00, 4.00it/s]
INFO:neutone_sdk.utils:Converting model to torchscript...
INFO:neutone_sdk.utils:Extracting metadata...
INFO:neutone_sdk.utils:Running model on audio samples...
Traceback (most recent call last):
File "C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\realtimeDDSP-masterv2\export.py", line 175, in
(RealtimeAudioDDSPv2) C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\realtimeDDSP-masterv2>
hyakuchiki
commented
1 year ago Sorry, I totally forgot that neutone_sdk also tries to save mp3 files which also doesn't work on Windows torchaudio. For now, can you try editing line 176 of export.py dump_samples=True to dump_samples=False so that it doesn't save files?
marlow77
commented
1 year ago I changed and saved the file to read the following on line 176: save_neutone_model( wrapper, root_dir, freeze=False, dump_samples=False, submission=True, audio_sample_pairs=soundpairs )
(RealtimeAudioDDSPv2) C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\realtimeDDSP-masterv2>python export.py 42.ckpt Alto_Sax_Model
100%|██████████████████████████████████████████████████████████████████████████████████| 79/79 [00:17<00:00, 4.42it/s]
100%|██████████████████████████████████████████████████████████████████████████████████| 81/81 [00:16<00:00, 4.82it/s]
100%|██████████████████████████████████████████████████████████████████████████████████| 52/52 [00:11<00:00, 4.38it/s]
INFO:neutone_sdk.utils:Converting model to torchscript...
INFO:neutone_sdk.utils:Extracting metadata...
INFO:neutone_sdk.utils:Running model on audio samples...
Traceback (most recent call last):
File "C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\realtimeDDSP-masterv2\export.py", line 175, in
(RealtimeAudioDDSPv2) C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\realtimeDDSP-masterv2>
marlow77
commented
1 year ago Do you think I should try to convert the .FLAC to .wav and run it? Line 171 looks like it might be trying to read wave?
input_sample = AudioSample(wave, sr)
hyakuchiki
commented
1 year ago It seems that save_neutone_model tries to save mp3 data inside neutone model files which also wasn't working with windows. I'll tell the neutone_sdk people about it.
In the mean time, I think it can be suppressed completely by setting audio_sample_pairs=[] and submission=False, i.e. setting line 176 to:
wrapper, root_dir, freeze=False, dump_samples=False, submission=False, audio_sample_pairs=[]Okay. So it generated the .nm model into the export folder with a .json file. I am going to try to test it with the neutone plugin, and I am assuming I need both files in the same folder to load it from the VST, right? I'll report back how it goes. When running the audio_sample_pairs=[] the out showed an error, but I am hopeful that it will still work?
(RealtimeAudioDDSPv2) C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\realtimeDDSP-masterv2>python export.py 42.ckpt Alto_Sax_Model
100%|██████████████████████████████████████████████████████████████████████████████████| 79/79 [00:19<00:00, 4.07it/s]
100%|██████████████████████████████████████████████████████████████████████████████████| 81/81 [00:17<00:00, 4.66it/s]
100%|██████████████████████████████████████████████████████████████████████████████████| 52/52 [00:12<00:00, 4.15it/s]
INFO:neutone_sdk.utils:Converting model to torchscript...
INFO:neutone_sdk.utils:Extracting metadata...
INFO:neutone_sdk.utils:Running model on audio samples...
INFO:neutone_sdk.utils:Validating metadata...
INFO:neutone_sdk.utils:Saving model to exports\Alto_Sax_Model\model.nm...
INFO:neutone_sdk.utils:Loading saved model and metadata...
INFO:neutone_sdk.utils:Testing methods used by the VST...
INFO:neutone_sdk.utils:Delay reported to the DAW for 48000 Hz sampling rate and 512 buffer size: 1536
INFO:neutone_sdk.utils:Running submission checks...
INFO:neutone_sdk.utils:Assert metadata was saved correctly...
INFO:neutone_sdk.utils:Assert loaded model output matches output of model before saving...
Traceback (most recent call last):
File "C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\realtimeDDSP-masterv2\export.py", line 175, in
(RealtimeAudioDDSPv2) C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\realtimeDDSP-masterv2>
hyakuchiki
commented
1 year ago That error wasn't supposed to show up with submission=False, but anyways the model is saved before that code that caused the error, so the .nm file should be fine.
marlow77
commented
1 year ago OK. It works! And I was able to load it into the plugin and have fun!
I noticed that the plugin does not properly identify the inputs and outputs like the other examples. For instance, with my model it says "unknown in" and "unknown out" while the other ones clearly say mono or stereo, etc.
The plugin delay is REALLY long. I have all of my buffers set right for near real time with 48K and 256 sample buffer in Reaper, which works with all of my other plugins (realtime guitar fx, instruments, etc.), but with neutone the plugin delay compensation amount is 1792 (PDC) and the sound takes approximately 4 seconds to play after sound is introduced into the input until the sound begins playing. Do you have any recommendations on what I could do to reduce this delay or is there any way to optimize the plugin for more real time use?
hyakuchiki
commented
1 year ago Thanks for bearing with me through this. I'll try to improve Windows compatibility in the future. "unknown in, unknown out" is just a problem with the current version of the neutone plugin not displaying metadata of locally loaded models (models loaded from "load your own feature"). 4 seconds of delay is really abnormal, since the DDSP models don't add too much latency (~0.1s). Can you test with other DDSP models inside the neutone plugin library like DDSP.violin and also setting DAW buffer size to 2048 buffer size? Otherwise it might be an issue with the relatively new Windows version of neutone which I guess should be handed over to others in the neutone team.
marlow77
commented
1 year ago Changing to larger buffer size only made the delay even longer, probably around 5-6 seconds.
hyakuchiki
commented
1 year ago Did this delay happen only when using the trained model or did this happen with other models as well?
marlow77
commented
1 year ago All models have this exceptionally long delay, unfortunately. I could reduce it somewhat by changing buffers and getting dropouts, but it was still unbearably long. I am using Windows 10, REAPER DAW, and a FOCUSRITE SOLO with its own ASIO drivers. I have switched to ASIO4ALL, FLEXASIO, and other settings, but the settings with the least dropouts is the FOCUSRITE with its own native ASIO drivers. As a note, I also tried the MAGENTA DDSP plugin and it has probably 100ms delay from when playing a midi note and the plugin produces audio.
hyakuchiki
commented
1 year ago Seems like more of a neutone issue then. I'll forward this to ppl on the plugin side and we'll try to find a solution. It would also be great if you can report in the neutone discord. One last thing, does this delay also occur with the clipper model in the neutone library?
marlow77
commented
1 year ago There is not any delay with the clipper model. I wonder if it might be the python backend of the plugin?
Hello,
I want to use custom timbres for your Neutone plugin, but using the collab notebooks online are timing out and are expensive. For these reasons, I wanted to use your files directly from this repository. However, I am getting errors as shown below and with the complete stack trace. I have tried to use both .mp3 files as you had in your data folder as well as .wav files, all mono, 44.1khz, and 16bit. Please help me to fix this issue so I can keep developing models for your plugin.
Thank you!
(RealtimeAudioDDSPv2) C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\realtimeDDSP-master>python train.py name=Alto_Sax data.raw_dir='C:/Users/marlo/anaconda3/envs/RealtimeAudioDDSPv2/realtimeDDSP-master/data/413204-mono.mp3' data.db_path='C:/Users/marlo/anaconda3/envs/RealtimeAudioDDSPv2/realtimeDDSP-master/temp_db_dir' [2023-02-18 10:06:22,729][pytorch_lightning.utilities.seed][INFO] - Global seed set to 0 Starting Preprocessing. 0it [00:00, ?it/s] Error executing job with overrides: ['name=Alto_Sax', "data.raw_dir='C:/Users/marlo/anaconda3/envs/RealtimeAudioDDSPv2/realtimeDDSP-master/data/413204-mono.mp3'", "data.db_path='C:/Users/marlo/anaconda3/envs/RealtimeAudioDDSPv2/realtimeDDSP-master/temp_db_dir'"] Error in call to target 'diffsynth.data.SliceDataset': Exception('No data found @ C:/Users/marlo/anaconda3/envs/RealtimeAudioDDSPv2/realtimeDDSP-master/data/413204-mono.mp3') full_key: data
Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.
(RealtimeAudioDDSPv2) C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\realtimeDDSP-master>python train.py name=Alto_Sax data.raw_dir='C:/Users/marlo/anaconda3/envs/RealtimeAudioDDSPv2/realtimeDDSP-master/data/413204-mono.mp3' data.db_path='C:/Users/marlo/anaconda3/envs/RealtimeAudioDDSPv2/realtimeDDSP-master/temp_db_dir' Traceback (most recent call last): File "C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\lib\site-packages\hydra_internal\utils.py", line 213, in run_and_report return func() File "C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\lib\site-packages\hydra_internal\utils.py", line 453, in
lambda: hydra.run(
File "C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\lib\site-packages\hydra_internal\hydra.py", line 105, in run
cfg = self.compose_config(
File "C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\lib\site-packages\hydra_internal\hydra.py", line 594, in compose_config
cfg = self.config_loader.load_configuration(
File "C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\lib\site-packages\hydra_internal\config_loader_impl.py", line 141, in load_configuration
return self._load_configuration_impl(
File "C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\lib\site-packages\hydra_internal\config_loader_impl.py", line 235, in _load_configuration_impl
self._process_config_searchpath(config_name, parsed_overrides, caching_repo)
File "C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\lib\site-packages\hydra_internal\config_loader_impl.py", line 158, in _process_config_searchpath
loaded = repo.load_config(config_path=config_name)
File "C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\lib\site-packages\hydra_internal\config_repository.py", line 349, in load_config
ret = self.delegate.load_config(config_path=config_path)
File "C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\lib\site-packages\hydra_internal\config_repository.py", line 92, in load_config
ret = source.load_config(config_path=config_path)
File "C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\lib\site-packages\hydra_internal\core_plugins\file_config_source.py", line 31, in loadconfig
cfg = OmegaConf.load(f)
File "C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\lib\site-packages\omegaconf\omegaconf.py", line 192, in load
obj = yaml.load(file, Loader=get_yaml_loader())
File "C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\lib\site-packages\yaml__init__.py", line 81, in load
return loader.get_single_data()
File "C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\lib\site-packages\yaml\constructor.py", line 49, in get_single_data
node = self.get_single_node()
File "C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\lib\site-packages\yaml\composer.py", line 36, in get_single_node
document = self.compose_document()
File "C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\lib\site-packages\yaml\composer.py", line 58, in compose_document
self.get_event()
File "C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\lib\site-packages\yaml\parser.py", line 118, in get_event
self.current_event = self.state()
File "C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\lib\site-packages\yaml\parser.py", line 193, in parse_document_end
token = self.peek_token()
File "C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\lib\site-packages\yaml\scanner.py", line 129, in peek_token
self.fetch_more_tokens()
File "C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\lib\site-packages\yaml\scanner.py", line 223, in fetch_more_tokens
return self.fetch_value()
File "C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\lib\site-packages\yaml\scanner.py", line 577, in fetch_value
raise ScannerError(None, None,
yaml.scanner.ScannerError: mapping values are not allowed here
in "C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\realtimeDDSP-master\configs\config.yaml", line 3, column 9
(RealtimeAudioDDSPv2) C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\realtimeDDSP-master>