search

YuanGongND / ast

Code for the Interspeech 2021 paper "AST: Audio Spectrogram Transformer".
BSD 3-Clause "New" or "Revised" License
1.06k stars 203 forks source link

Colab notebook for inference is only partly usable with CPU #97

Open Cycerotki opened 1 year ago

Cycerotki commented 1 year ago

Hello, small issue to raise for your Colab notebook.

When I test your AST inference notebook on Google Colab, it runs well with GPU. However, if I switch hardware accelerator to None (i.e. running on CPU only), there are some errors as you load your model files to GPU by default. Would it be possible for you to add condition to load the model+images depending on whether the notebook is using GPU or not (thus really not requiring users to have GPU if they use notebook on their own devices)? Thanks!

YuanGongND commented 1 year ago

Hi, thanks so much for reporting this. I think it is valid, the device is hard-coded. I assume most user would like to have GPU running, is there a specific reason to use CPU instead?

In local scripts, the device is determined by the available resource, https://github.com/YuanGongND/ast/blob/9e3bd9942210680b833b08c39d09f2284ddc4d1d/src/run.py#L147.

-Yuan

Cycerotki commented 1 year ago

Currently I am using ESC-50 dataset, but was trying out AST_Inference_Demo.ipynb to learn how AST does inference. You are right that most users would prefer GPU; in my case, I am experimenting with running AST inference on CPU for web application deployment (running inference on my local computer which does not have NVIDIA GPU).

Thank you for pointing out the one liner fix, will use that for inference!

YuanGongND commented 1 year ago

Got it, thanks for the clarification. Please note both the model and the input needs to moved to CPU by .to(device) for consistency.

-Yuan