search

leginon-org / leginon-redmine-archive

1 stars 0 forks source link

Relion 2D tool in appion uploads wrong results #4566

Open leginonbot opened 6 months ago

leginonbot commented 6 months ago

Author Name: Venkata Dandey (Venkata Dandey) Original Redmine Issue: 4566, https://emg.nysbc.org/redmine/issues/4566 Original Date: 2016-11-04

I tried to use the relion 2D tool in appion and in curiosity was checking the results inside the folder through command line. At the end the results uploaded in appion web page was totally wrong compared to the actual results. I attached the two images .. session is 16nov2d.

leginonbot commented 6 months ago

Original Redmine Comment Author Name: Neil Voss (@vosslab) Original Date: 2016-11-04T18:27:10Z

First thing I see is that the empty classes are replaced with the stack average, so that is confusing.

Second, I align all the classes to one another, so in the RELION version the object is pointing in various directions whereas in Appion they are point in the same direction.

Third, I cannot explain the class average differences, the Appion version seems like something I would expect. The contrast is too high, but more what I expect. I would recommend opening the appion stack in another viewer (e.g., EMAN) where you can scale the contrast, because all the details are washed out.

leginonbot commented 6 months ago

Original Redmine Comment Author Name: Bridget Carragher (@bcarr-czi) Original Date: 2016-11-04T18:53:14Z

It seems for sure that these are not the same particles contrast not withstanding. This is pretty worrying that a bug this serious is in there. Please can anyone with input try and help sort this out. Thanks, Bridget

leginonbot commented 6 months ago

Original Redmine Comment Author Name: Neil Voss (@vosslab) Original Date: 2016-11-04T19:14:19Z

I was hoping if the contrast is corrected, we might see the relationship between the two. To help see what is going on.

leginonbot commented 6 months ago

Original Redmine Comment Author Name: Neil Voss (@vosslab) Original Date: 2016-11-04T19:19:15Z

I could also be that RELION added a feature (such as mirroring) since I got it working this summer.

leginonbot commented 6 months ago

Original Redmine Comment Author Name: Anchi Cheng (@anchi2c) Original Date: 2016-11-04T19:46:26Z

Is it possible that we should not put it as an Particle Alignment step ? The forcing of aligning the reference step might may be part of the problem. Sorjs said that Relion does the normalization by making the region outside the mask to have standard deviation of 1. This normalization is important in his statistical model. The references out of Relions are masked and without any noise outside the mask.

leginonbot commented 6 months ago

Original Redmine Comment Author Name: Venkata Dandey (Venkata Dandey) Original Date: 2016-11-04T19:46:40Z

I tried the EMAN2 e2display (attached), which in my experience never displays the relion results in a proper way and we cannot even adjust the contrast levels. But if I open the appion stack (part16nov02z59_average.img) in relion (attached) looks better but the results is still not the same. I think through appion it still points to Relion 1.4 stable version.

leginonbot commented 6 months ago

Original Redmine Comment Author Name: Venkata Dandey (Venkata Dandey) Original Date: 2016-11-04T19:54:17Z

Achi it is a good point to do a preprocess the stack using "relion_preprocess" command. But here the relion results I attached is pulled from the appion folder which is still from the appion wrapper command with the option "--dont_checknorm" option.

I processed this stack in relion separately and results look same. I am mainly bothered about the visualization in appion, somehow the preprocess done with the relion results to show in web viewer has some bug in it.

leginonbot commented 6 months ago

Original Redmine Comment Author Name: Venkata Dandey (Venkata Dandey) Original Date: 2016-11-04T19:58:48Z

oops!! Anchi, I did not read this statement carefully - " The forcing of aligning the reference step might may be part of the problem." I meant the same.

leginonbot commented 6 months ago

Original Redmine Comment Author Name: Neil Voss (@vosslab) Original Date: 2016-11-04T20:00:41Z

As Anchi points out, I read the alignment results from RELION and create my own aligned stack and then average the classes and upload.

Possible problems:

I tested for these extensively, but I cannot rule out error.

I do not think the pre-normalization is important, but it is possible that RELION does not use 100% of the intensity in the average, but a weighted average of the particle in the class based on the probabilities.

For aligning the references, I run a second instance of RELION, get a second set of alignment parameters and merge them with the first alignment parameters (using trig). Then I create my own aligned stack and then average the classes and upload

leginonbot commented 6 months ago

Original Redmine Comment Author Name: Anchi Cheng (@anchi2c) Original Date: 2016-11-04T21:58:37Z

I think our problem comes from making "the average of the aligned particles in a class" as a reference class at the end of the upload. We should stop doing that. Maxlike method classes are certainly weighted sum of the particles. I don't think have a complete output of the relative weight,. There are a few single number propability values in _data.star that is not enough to reconstruct the weight.

The upload process should simply upload the shift/rotated references as using what comes out from what Neil referred as "second instance of RELION" that operates on the original "classes" as Relion calls it.

Hope I am not making it more confusing. I illustrate what I think would work with this excercise: I uploaded the iter030_classes.mrcs from 16nov20d/align/rmaxlike2/iter030 into session 16nov04z and run relion maxlike 2D classification in Appion as it would be in what Neil called the "second instance". If you view the alignestack, then we have the desired effect that keeps what Relion is famous for - Well defined lobes with high resolution feature, and yet aligned relative to each other (what Appion wants to have).

http://emgweb.nysbc.org/betamyamiweb/processing/viewstack.php?file=/gpfs/appion/acheng/16nov04z/align/rmaxlike1/alignstack.hed&expId=2875&ps=6.0E-10

To display the number of particles "Belong" to a class (I think it is an over statement, it probably only means the largest propablity), then the upload script can attach the number from reading into _data.star file.

leginonbot commented 6 months ago

Original Redmine Comment Author Name: Neil Voss (@vosslab) Original Date: 2016-11-04T22:09:07Z

The discrepancies between the RELION probability classes and Appion forced align classes have never been so profound.

It is like quantum mechanics, particles can exist in multiple alignments and classes until to make a measurement and force them into a particular state, which is their most probable state. I preferred the forced align classes, because if I make a substack or do an RCT, I am using the forced parameters, not the probability distribution. But if you are doing a rough analysis of the data, you want the pretty pictures, so maybe we should provide both.

I am still concerned about the differences between the two sets, they are usually pretty close. If I had time, I would probably want to do some more testing.

leginonbot commented 6 months ago

Original Redmine Comment Author Name: Anchi Cheng (@anchi2c) Original Date: 2016-11-04T23:59:02Z

Neil, will you have time within next week to add the weighted class average (as in Relion output) so that I can assign this to you, or should I take charge or ask for another volunteer ?

leginonbot commented 6 months ago

Original Redmine Comment Author Name: Neil Voss (@vosslab) Original Date: 2016-11-05T02:08:36Z

Sadly, I have no time.

leginonbot commented 6 months ago

Original Redmine Comment Author Name: Neil Voss (@vosslab) Original Date: 2016-11-05T02:12:03Z

It may store the weights, but each particle has a weight (probability) for every shift, every rotation, and every class as it is needed in the integral, so it would be a BIG file.

leginonbot commented 6 months ago

Original Redmine Comment Author Name: Bridget Carragher (@bcarr-czi) Original Date: 2016-11-06T22:32:22Z

Thanks Anchi - that sounds like what we want. Given that Neil does not have time to work on this can you perhaps recruit Carl? Perhaps with your help? Not sure who else has the time or skill set? Best regards, Bridget

leginonbot commented 6 months ago

Original Redmine Comment Author Name: Venkata Dandey (Venkata Dandey) Original Date: 2016-11-09T16:06:21Z

Thanks Anchi, it looks correct now. I have attached the screenshot here, I tried GDH sample this time which I know it can go 3.5 angstroms and the results are pretty good. It is a quick run by binning 4. It ran for 4 hrs with 64x64 pixels, 25000 particles and asked for 32 particles with 4 nodes and 92 processors. Now, I can confidently recommend this tool in appion to users.

leginonbot commented 6 months ago

Original Redmine Comment Author Name: Anchi Cheng (@anchi2c) Original Date: 2016-11-09T16:52:40Z

Thanks.

A interesting result from what I have seen though is that for well separated classes, the unweighted average of the particles in class is more similar to thee weighted one. Your previous example was a better one to observe problems.

leginonbot commented 6 months ago

Original Redmine Comment Author Name: Venkata Dandey (Venkata Dandey) Original Date: 2016-12-01T14:46:23Z

Hi, this time I am trying a different dataset which is more heterogeneous to test as anchi suggested.

But somehow I can't run the job ..it is complaining about the total memory should be <16GB per node. But each node in our cluster has 256GB(I think).

This is the error, Error: Total MPI threads per node MPI memory/thread is greater than max memory per node. ( 23 procs/node 1 threads/proc * 4 Gb/thread = 92 Gb > 16 Gb )

leginonbot commented 6 months ago

Original Redmine Comment Author Name: Carl Negro (@carl9384) Original Date: 2016-12-01T15:24:11Z

This is an appion configuration issue, not an Appion bug. I will take care of it at NYSBC.

leginonbot commented 6 months ago

Original Redmine Comment Author Name: Ashleigh Raczkowski (Ashleigh Raczkowski) Original Date: 2017-03-28T20:52:06Z

Ran 2D classification through Appion on guppy. All but two classes are empty in the Appion upload and they're terrible. Relion 2 beta gave really nice classes from the same stack.

/gpfs/appion/lkim/qfan/17feb27b/align/rmaxlike32119

leginonbot commented 6 months ago

Original Redmine Comment Author Name: Neil Voss (@vosslab) Original Date: 2017-04-05T20:43:32Z

Hi Ashleigh,

I am confused by your bug report. Relion is notorious for having empty classes in 2d alignment and there is nothing I can do about it on the Appion side of things, which is why I always use the same program in Xmipp.

Perhaps they fixed the empty class problem in Relion 2. Can't you just upload the Relion 2 class averages? What error do you get?

leginonbot commented 6 months ago

Original Redmine Comment Author Name: Ashleigh Raczkowski (Ashleigh Raczkowski) Original Date: 2017-04-06T16:04:13Z

Hi, Neil. These are the relion 2 class averages using the same stack as the mentioned appion relion run.

leginonbot commented 6 months ago

Original Redmine Comment Author Name: Sargis Dallakyan (@dallakyan) Original Date: 2018-04-11T18:20:01Z

I've committed commit:211f1e4c that I hope can fix this. I think this is one-off issue in that relion counts classes starting with 1 whereas in python we count array starting with 0 index.