I was running into several issues with the described setup.
First cuda 11.1 did not work with the described fairscale. I resolved that by using 0.4.1. I also dropped optuna for now because the installation seems to downgrade pytorch. Just wanted to leave it here in case others run into similar issues. It seems to run now.
I was running into several issues with the described setup. First cuda 11.1 did not work with the described fairscale. I resolved that by using 0.4.1. I also dropped optuna for now because the installation seems to downgrade pytorch. Just wanted to leave it here in case others run into similar issues. It seems to run now.