Open crud89 opened 2 years ago
Thanks for the bug report. This is very strange. We explicitly added indexing='ij'
to the torch.meshgrid()
calls to remove the warnings (I assume the meshgrid warning reappeared for you when you removed the indexing argument?). We haven't seen this locally. Perhaps a file-encoding issue?
Two other things for building the PyTorch cuda extensions:
Actually, I would change the pytorch commandline to
conda install pytorch torchvision torchaudio cudatoolkit=11.3 -c pytorch
pytorch only supports cuda 11.3 . It is fine to have a more modern Cuda toolkit installed. I run with 11.5 and 11.6 locally.
Thanks for the answer!
Indeed, I am having VS 2019 installed. I also installed the toolkit using the installer beforehand. I am not seeing any warnings on my side. I have downgraded pytorch at first (to 1.10), to see if it fixes the error (it didn't), but I have re-created everything (i.e. deleted the old and created a new environment) to see if removing the parameter also fixed the issue with version 1.11 and I am not receiving any warnings here. So no matter the pytorch version, there is no warning, if I remove the indexing parameter, but there is an error, if I leave it in. Still, I am not exactly sure what is going on, as I would have expected a warning myself.
The only warning is about the encoding, but I am not sure if this is relevant or even related.
Ok, this is strange. I'll try in a fresh conda env later to see if I can reproduce. I think I have the latest PyTorch 1.11, but they may have changed something very recently. Just to double-check, did you change the cuda version used by torch from
conda install pytorch torchvision torchaudio cudatoolkit=11.5 -c pytorch
(not recommended)
conda install pytorch torchvision torchaudio cudatoolkit=11.3 -c pytorch
(our recommended setup, still while having the 11.5 toolkit installed)
I tried both and both resulted the same behavior. Currently I am on cudatoolkit=11.3
as you recommended.
I just created a new conda env, and installed PyTorch 1.11.0 in there, following the installation steps in the readme exactly. Everything runs fine on my end, and I don't see the warning
UserWarning: Error checking compiler version for cl: 'utf-8' codec can't decode byte 0x81 in position 62: invalid start byte
.
Sorry I cannot help you more. Perhaps try on another machine to see if that is an issue specific to the current desktop?
Thanks for testing it!
I will see if I can test it on another system. So far, I am okay with manually fixing this issue. If I find something, I will let you know.
Thanks again! 🙂
I solve this problem :)
The latest version of pytorch (above 1.9.0) does not support this parameter anymore. Only:
torch.meshgrid(*tensors)
Although this parameter is missing, the default is still indexing='ij'
So we just need to delete the indexing part of the error line
The latest version of pytorch (above 1.9.0) does not support this parameter anymore.
It does not support this parameter yet. This is because it has only been added in version 1.10.0.
So unfortunately this is not a solution, because removing it will currently work but wont for future releases, where the default parameter will change from ij
to xy
.
Thanks @crud89 ,
Yes, we added the indexing parameter to remove the warning in PyTorch 1.10+
I am having trouble following the instructions from the readme file. Whenever I run the training for an example, I receive an error:
I am not sure if this is related, but I also received some warnings during installation, namely when attempting to install tinycudann:
I then receive similar encoding warning every time I run the training script:
Just in case:
conda install cudatoolkit=11.5 -c conda-forge
after whichconda install pytorch torchvision torchaudio cudatoolkit=11.5 -c pytorch
also succeeded, but none of this solved the issue.Weirdly enough, removing the
indexing
parameter from all calls totorch.meshgrid()
appears to fix the issue. The parameter defaults toij
anyway, but this would only work for current releases of pytorch, as the documentation states, that it will transition toxy
as the default in the future. No idea why it works when removing the argument entirely, though.I would be grateful for any hints. Thanks! 🙂