search

leginon-org / leginon-redmine-archive

1 stars 0 forks source link

Affinity propagation fails on large stacks #252

Closed leginonbot closed 6 months ago

leginonbot commented 6 months ago

Author Name: Neil Voss (@vosslab) Original Redmine Issue: 252, https://emg.nysbc.org/redmine/issues/252 Original Date: 2010-03-25 Original Assignee: Amber Herold

The affinity propagating clustering program requires that all particles in a stack be inserted into memory for the cross-correlation matrix calculation.

leginonbot commented 6 months ago

Original Redmine Comment Author Name: Neil Voss (@vosslab) Original Date: 2010-03-25T18:25:21Z

This needs to use my new processStack class in source:trunk/appion/appionlib/apImagicFile.py to systematically read particle subsets from the a stack.

leginonbot commented 6 months ago

Original Redmine Comment Author Name: Neil Voss (@vosslab) Original Date: 2010-05-20T17:48:25Z

Note: program did not do a good job of comparing old version to new version.

leginonbot commented 6 months ago

Original Redmine Comment Author Name: Amber Herold (Amber Herold) Original Date: 2010-05-20T18:48:02Z

Looks ok. Not a fan of using simf as a shorthand for similarfile but OK. Can you assign to someone to test?

leginonbot commented 6 months ago

Original Redmine Comment Author Name: Neil Voss (@vosslab) Original Date: 2010-05-20T19:42:55Z

You'll need to run on amibox after Friday, May 20. Try it on a large stack > 16GB.

leginonbot commented 6 months ago

Original Redmine Comment Author Name: Amber Herold (Amber Herold) Original Date: 2010-09-21T22:19:25Z

Dmitry, do you happen to use affinity propagation on >16GB stacks ever?

leginonbot commented 6 months ago

Original Redmine Comment Author Name: Dmitry Lyumkis (@LyumkisLab) Original Date: 2010-09-21T22:35:30Z

that's quite big. I don't have anything that size. I can look around other projects though

leginonbot commented 6 months ago

Original Redmine Comment Author Name: Amber Herold (Amber Herold) Original Date: 2010-09-21T23:23:30Z

This one is too difficult to test bc we don't have any large stacks that will work. Since Neil logged it I'm assuming it worked for the data that he was having trouble with.

leginonbot commented 6 months ago

Original Redmine Comment Author Name: Neil Voss (@vosslab) Original Date: 2010-09-22T13:10:38Z

I think Joel reported this issue to me for data at NIS. In the past it would probably fail on stacks > 2GB. The algorithm has changed so all stacks are treated equally, so if it works on a 200kB stack it should also work on a 200GB stack.

leginonbot commented 6 months ago

Original Redmine Comment Author Name: Neil Voss (@vosslab) Original Date: 2010-09-22T13:11:28Z

see last message