PixelDraw Colab throws cudaErrorInvalidDeviceFunction

rockt commented 3 years ago

Hi, I created a copy of the PixelDraw colab, run the setup (which was successful) but then ran into the following error when running the 2nd cell.

ops: runtime error:  radix_sort: failed on 1st step: cudaErrorInvalidDeviceFunction: invalid device function
Try reducing --num-cuts to save memory
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-2-df7ac889ea98> in <module>()
     45 settings = pixray.apply_settings()
     46 pixray.do_init(settings)
---> 47 pixray.do_run(settings)

6 frames
/usr/local/lib/python3.7/dist-packages/diffvg-0.0.1-py3.7-linux-x86_64.egg/pydiffvg/render_pytorch.py in backward(ctx, grad_img)
    707                       use_prefiltering,
    708                       diffvg.float_ptr(eval_positions.data_ptr()),
--> 709                       eval_positions.shape[0])
    710         time_elapsed = time.time() - start
    711         global print_timing

RuntimeError: radix_sort: failed on 1st step: cudaErrorInvalidDeviceFunction: invalid device function

Have you seen this before?

dribnet commented 3 years ago

did you also restart the runtime after running the setup cell?

rockt commented 3 years ago

Yes, I did. I got "setup complete" before running the next cell.

dribnet commented 3 years ago

Not sure. The notebook is currently working fine for me.

marcelo313 commented 3 years ago

I believe they don't have enough memory, I was using the free version of colab and got the same error. After upgrading it ran successfully.

NTaya commented 3 years ago

Same. I have 16 GB available memory (I use the Pro version of Colab), restart the runtime (this is not my first time using PixelDraw; the first cell says "setup complete") and yet most of the times get this error in the notebook nonetheless. It worked fine for me ~50 days ago in the very same notebook, but now fails. Notebook link.

It sometimes works out of nowhere after I restart the runtime a few times (>=3 times), but it often doesn't work by default.

GinYoshida commented 2 years ago

I faced the same error and I did the following process.

Step1: Changed my plan from Free to Pro Step2: Runtime => Change runtime type => Runtime shape => "High-RAM" => Restart the runtime

It is working now.

dribnet / pixray

PixelDraw Colab throws cudaErrorInvalidDeviceFunction #33