search

cistem-org / developmental-docs

BSD 3-Clause "New" or "Revised" License
4 stars 0 forks source link

GPU issues - In situ particle detection #22

Closed cyntiaT closed 2 years ago

cyntiaT commented 2 years ago

Hello,

And Happy New Year 2022!

I'd like to use in situ particle detection using GPUs. Everything works properly with CPUs, but when it comes to GPU, this is not the case. The calculation goes till the end, however, ​no particles are found and no histogram is written. There is no error message.

Do you have an idea where it comes from? I have tried different versions of Cuda, and does look like it is not the issue.

Thank you very much! Cyntia

bHimes commented 2 years ago

Hi Cyntia,

Assuming you used the GUI, could you please post the line from your run profile you used for the gpu test?

cyntiaT commented 2 years ago

Hello!

I indeed use the GUI. I attached the profile of an entire quick run. Nvidia-smi shows me that the GPUs are used, and I realised that my run ended on an error message :

(cisTEM:31132): Gtk-CRITICAL **: 12:01:24.648: IA__gtk_tree_selection_unselect_iter: assertion 'selection->tree_view->priv->model != NULL' failed

On Tue, 11 Jan 2022 at 01:05, B.A.Himes @.***> wrote:

Hi Cyntia,

Assuming you used the GUI, could you please post the line from your run profile you used for the gpu test?

— Reply to this email directly, view it on GitHub https://github.com/bHimes/cisTEM_docs/issues/22#issuecomment-1008904199, or unsubscribe https://github.com/notifications/unsubscribe-auth/AONRH6AZLNAAYQQKEA3FP33UVLRTVANCNFSM5LTM4WCA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you authored the thread.Message ID: @.***>

-- Dr. Cyntia Taveneau

Research Fellow

Monash University | Dept. of Biochemistry and Molecular Biology +61 3 99056425 <+61+3+99056425> https://orcid.org/0000-0002-3395-4957 23 Innovation Walk, 3800 Clayton (Australia)

bHimes commented 2 years ago

Hello! I indeed use the GUI. I attached the profile of an entire quick run. Nvidia-smi shows me that the GPUs are used, and I realised that my run ended on an error message : (cisTEM:31132): Gtk-CRITICAL : 12:01:24.648: IA__gtk_tree_selection_unselect_iter: assertion 'selection->tree_view->priv->model != NULL' failed … On Tue, 11 Jan 2022 at 01:05, B.A.Himes **@.> wrote: Hi Cyntia, Assuming you used the GUI, could you please post the line from your run profile you used for the gpu test? — Reply to this email directly, view it on GitHub <#22 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AONRH6AZLNAAYQQKEA3FP33UVLRTVANCNFSM5LTM4WCA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub. You are receiving this because you authored the thread.Message ID: @.> -- Dr. Cyntia Taveneau Research Fellow Monash University | Dept. of Biochemistry and Molecular Biology +61 3 99056425 <+61+3+99056425> https://orcid.org/0000-0002-3395-4957 23 Innovation Walk, 3800 Clayton (Australia)

I think when you reply through your email client, the attachment doesn't make it through. If you instead click the "view on github" then you should be able to attach a file just by drag and drop into the comment box here.

cyntiaT commented 2 years ago

It should work this time!

TM_cistem.txt

bHimes commented 2 years ago

Hi Cyntia,

  1. Sorry, this is the console output. The "run profile" is defined in your settings (gear icon on the bottom of the left panel) and is what directs the resources to be used.
  2. It looks as though you are trying to run a very fast search by looking over a 90* in plane step but still using c1 symmetry. I suggest you actually run the tutorial before you further trouble shoot using your current approach.
cyntiaT commented 2 years ago

Hi Benjamin,

I ran the test on a micrograph that I have checked using CPUs. I chose 90 as 4/5 ribosomes were detected using CPUs to speed up the process. I might have found the issue: the graphic cards from the cluster have been recently upgraded (A40). Since then, everything works properly. I guess that there is a requirement in terms of graphic cards? I was using K80 ones. In addition, I have more RAM now (250 GB vs 117 GB before).

bHimes commented 2 years ago

Hi Cyntia

If using on older cards, you can recompile, but it is not officially supported on any architecture older than Volta. (Kepler is 2 gens behind volta.) Glad you got some new cards.

Fyi, the tutorial data only takes ~2-3 minutes on the gpu, but more importantly, that is for a full search that produces results that I cam compare to. (Which is why I asked for that.)

Anyway, glad you got it sorted. Happy matching!