search

dattalab / keypoint-moseq

https://keypoint-moseq.readthedocs.io
Other
68 stars 28 forks source link

kernel dies while running PCA #41

Closed vickerse1 closed 1 year ago

vickerse1 commented 1 year ago

Hi,

With the Windows GPU version installed, all of the suggested jupyter lab code runs fine until the PCA cell, and then the kernel dies and there is no further error message.

When I run "python -c "import jax; print(jax.devices())"", I get:

[StreamExecutorGpuDevice(id=0, process_index=0)]

Does this mean the GPU is not being used?

Thanks,

Evan

calebweinreb commented 1 year ago

It seems like the GPU is being used. Have you followed any of the steps in troubleshooting guide? My guess is that some other process is hogging the GPU memory. To check, you can shut down the current notebook kernel and check what percent of the GPU VRAM is available. It should be close to 100%. If it's not then maybe shutdown all the other notebook kernels and check again?

vickerse1 commented 1 year ago

My GPU Load looks low, and no other problems are detected. I will retry under low CPU load conditions later (running CPU version now, want to let it finish), but I doubt that's an issue.

Found a similar issue here where problem was code bug, no idea if it might apply here though:

https://github.com/catboost/catboost/issues/1735