Closed dungxibo123 closed 2 years ago
Hi there. (In passing I'd note that I'm not a professor!)
So the GPUs we uses were some mid-level ones -- to quote from the paper, two GeForce RTX 2080 Ti, and two Quadro GP100. I doubt that the GPU choice is going to be responsible for poor performance though.
On the topic of model performance -- I am (obviously) quite surprised by the poor results you seem to be getting. IIRC with Speech Commands you should get to about 85% accuracy after just a few epochs, and then the rest of the training time slowly improves things a little further.
Are you definitely running the neural CDE models, and not one of the benchmark-for-comparison RNN models? The RNN models were all flaky on this dataset: sometimes they would produce excellent results, sometimes they would produce awful results, and it differed from training run to training run. That aside, it's also plausible that a change in software library somewhere has quietly broken something.
As a first place to start, I'd recommend trying the Speech Commands example from this repository. This was a follow-up paper that happened later, with a new codebase. (And substantially tidier code.) Getting some more data might help diagnose the issue.
Many thanks for your response.
I will try using the new code in your recommendation. In order to reply above questions. I tried to remove all other models. I just run NeuralCDE
model. And, due to datasets/speech_commands.py
does not contain any deprecated methods so I have not changed anything. But I think some small changes have broken something.
Again, thanks for your advices.
I'm sorry madam, In case someone falls into my fault, I will leave some comments for them.
Please make sure that torchaudio.load
goes with normalize=False
(default will be True), I have erased this term when process data.
If you want to train with fewer classes, please make sure, X
and Y
variables was generate as same size as batch_index
.
Thanks.
Dear Professor,
I'm a third-year student who is interested in your works. I'm starting with Neural Control Differential Equations for Irregular Times Series Data.
I have a question and I hope you have time to respond to me.
I wonder that what GPUs you had used for your experiments and how many days that you have to wait to achieve the result in the paper. I tried to run your work by Python with some updates for compatibility with the new version of libraries. After few days, some methods to preprocess the UEA datasets were deprecated. I start with the Speech Commands datasets. I had run code on 3 types of GPUs.
1/ 1x 1050Ti 4GB. (70s-90s / 1 epoch) 2/ 1x A100 40GB. (120s - 144s / 1 epoch) 3/ 1x 3090Ti 12GB. (40s - 50s / 1 epoch)
With the above GPUs, I received quite bad results. For more specific. It was around 0.57 ~ 0.58 accuracy on the validation set (and also the train test, too) for the first time. The second time, I run on A100. and after 70 epochs of the first run, the validation was returned is 0.213. I just want to know what your GPUs are, when you run the code, maybe I was wrong somewhere.![image](https://user-images.githubusercontent.com/41417288/135862159-a7388972-5a47-499c-af42-8a25aa28132d.png)
Best regards, Tien Dung