GPU and memory columns not present

plasma-umass / scalene

Scalene: a high-performance, high-precision CPU, GPU, and memory profiler for Python with AI-powered optimization proposals

Apache License 2.0

12.14k stars 398 forks source link

GPU and memory columns not present #794

Open aibtw opened 7 months ago

aibtw commented 7 months ago

Describe the bug I am trying to get GPU profiling, but in the cli and the html there is no column for that.

To Reproduce I used this command: scalene --cpu --gpu ./my_script.py

Expected behavior I see images on this repo of what the profiling result should be like. Mine is nothing like that. see below

Screenshots

Desktop:

OS: ubuntu 22.04
Browser: Mozilla Firefox 122.0.1

emeryberger commented 7 months ago

Hardware platform? What happens when you run nvidia-smi? (Scalene's GPU support is currently limited to NVIDIA.)

aibtw commented 7 months ago

Hardware platform? What happens when you run nvidia-smi? (Scalene's GPU support is currently limited to NVIDIA.)

sorry, forgot the details:

Intel i7-13700 with RTX 3080.
Also tried on Jetson Orin 16GB and got same issue, but let's not focus on this.
I will add that I tried different versions (latest from github, 1.5.19, I don't remember the rest, but I tried multiple ones in this range)
Python version 3.8

emeryberger commented 7 months ago

The usual issue here is an old driver, but let's be sure. Please run nvidia-smi and paste the output here.

aibtw commented 7 months ago

I see

here is the output of nvidia-smi while the scalene is running

Is it a CUDA version issue?

emeryberger commented 7 months ago

The driver version is a little bit on the old side (released in 2022); if you can update it to the latest version, please do and let's see what happens. In my experience, this usually resolves the problem.

emeryberger commented 7 months ago

@aibtw any updates?

aibtw commented 7 months ago

Sorry, been testing it with different conditions. So, after I updated the driver to 530,

I setup a new venv to test it.

If I run it on a non-threaded application, it seems to work (but GPU memory result doesn't seem right to me)

However, my application is threaded. If I run it, it is still the same.

So, does it work for threaded applications?

I really like the tool. Appreciate your effort in trying to solve this.

luizcz commented 1 month ago

Is there a command to check hardware compatibility? I would like to profile a Jetson GPU.