Open AndisDraguns opened 3 weeks ago
Hi there, I was just running this code on a A10G in a Lightning Studio, and it worked fine. Do you have the error message you got or more explanation on how or why it failed? One hypothesis is that the memory in Colab may not be sufficient depending on the GPU. However, based on my test, it should only require 8.62 GB on a GPU that supports bfloat16.
@rasbt we created the issue together at iclr. I will look into it. This is in colab, not studios.
There is a cache path resolution issue in LitData, it needs to be fixed there. Thanks for the repro. https://github.com/Lightning-AI/litdata/issues/126
Running the pretraining example from GitHub fails when run in Google Colab.
@awaelchli