iPieter
commented
2 years ago First of all, thanks for your interest in RobBERT!
The answer to your question depends on why you want to use those notebooks:
- To actually replicate our results. If so, I would really recommend installing the old (2019) dependencies on a machine with a 1080 Ti. Otherwise, I don’t think you’ll be able to get the same results. There exist a lot of examples of small differences between GPUs and CPUs (and especially an ARM M1) to illustrate that this is not an ideal setting to replicate our results. Also, I do not have an M-series mac, so I cannot test this myself...
- Finetuning of your own model. For this, that notebook is also hopelessly outdated. Huggingface now has a very nice Trainer api, I would recommend using that. If I now finetune a model, I also use the Trainer in combination with a training logger (I use MLFlow with some custom hyperparameter sweeps, but weights and biases or comet.ml are other options). This tutorial should set you on the right path.
Good luck!
bojan2110
First, thanks for sharing this valuable project! We are very excited to test these models in our research at Erasmus Rotterdam.
My question is related to running the models on cpu-only mode, as I have a mac without nvidia (so no cuda support). I am trying to reproduce the sentiment analysis project (dbrd) using the finetune_dbrd.ipynb notebook you have shared. However, when training the model with the provided code I get a message (at the end of the training process) AssertionError: Torch not compiled with CUDA enabled.
So my question is if there is a way to run this with cpu-only pytorch? I see that in your train.py script you are recognising the device type (args.device) and I have tested that it prints out 'cpu'. Are there some parameters that I need to change in order to run the notebook on my mac laptop?