search

leginon-org / leginon-redmine-archive

1 stars 0 forks source link

Template picking false positives on the sides of the micrographs #4338

Open leginonbot opened 6 months ago

leginonbot commented 6 months ago

Author Name: Yacob Gomez-Llorente (Yacob Gomez-Llorente) Original Redmine Issue: 4338, https://emg.nysbc.org/redmine/issues/4338 Original Date: 2016-07-26 Original Assignee: Neil Voss

Template particle picking is very often picking a lot of false positives on the sides of the micrographs. This is a consistent error that I have seen in many micrographs in different sessions. Take a look at tmplrun15 on 16jul22b session: http://emgweb.nysbc.org/betamyamiweb/processing/multiimgassessor.php?expId=2334&pickId=26 There are two images that have been tested. The image on the right has half of the particles aligned on the sides of the micrograph. There are all or most of all false positives and I will have to remove them manually. I am not sure if it is a problem with the templates or a bug of the program. But I think you should take a look.

leginonbot commented 6 months ago

Original Redmine Comment Author Name: Yacob Gomez-Llorente (Yacob Gomez-Llorente) Original Date: 2016-07-26T17:28:54Z

An update: I am curious to see that it is only happening on the left and right sides of the micrographs. Not in the top and bottom ones. Maybe that gives you a hint to figure out the problem.

leginonbot commented 6 months ago

Original Redmine Comment Author Name: Anchi Cheng (@anchi2c) Original Date: 2016-07-26T18:45:57Z

Probably has to do with the fact that the micrograph is a rectangle. Programs are fooled by that quite often.

leginonbot commented 6 months ago

Original Redmine Comment Author Name: Bridget Carragher (@bcarr-czi) Original Date: 2016-07-27T18:23:50Z

Try it out on a square image and see if that helps. Not sure why we have this edge effect though? SOmething in the original Alan Roseman codes? Neil, do you have a clue?

leginonbot commented 6 months ago

Original Redmine Comment Author Name: Neil Voss (@vosslab) Original Date: 2016-08-04T21:07:12Z

Can we move this bug to Appion? Because it is under "NYSBC SEMC User Group", redmine is missing several features for me.

As far as the bug goes, it won't matter when you make a stack because those particles will automatically be removed.

I have never seen this when I pick rectangular images, but could be the issue.

leginonbot commented 6 months ago

Original Redmine Comment Author Name: Neil Voss (@vosslab) Original Date: 2016-08-05T17:23:31Z

Adding email chain:

Well we could do this but it is really an ugly hack to fix a bug in the template picking program caused by not taking rectangular images into account properly. It would be better to fix the actual bug. In the meantime one thing you could do is to:

  1. Pick using templates.
  2. make a stack that will exclude the edged.
  3. Convert the stack to picks again. Use these for the Relion etc. run. This is also a hack but one that does not require any programming. Best regards, Bridget

One possible solution would be to include a "border threshold" to exclude particles that are picked at X pixels of the border of the micrograph. That would bypass the problem. After/during particle picking but before particle extraction. I could suggest that to Neil if you like.

Thanks, Yacob

From: Gomez, Yacob

Good afternoon Bridget, Thanks for taking your time to email me and taking care of this.

Let me explain how I see it:

They way I work is: first, I treat the data with Appion after the data collection. That is the easiest, fastest and most useful thing to do after collecting data with Leginon. It allows me to process and analyze data on the fly, and adjust my data collection based on the results. Also little by little I am learning to use more features of the suite.

For several reasons I decide to switch from Appion to other EM programs after particle picking. You know most of them:

  1. Appion allows particle picking based on templates, and that is something that is interesting for my projects.
  2. But unfortunately, for later steps of the image processing, like 2D classification and all the following steps, the CPU usage is big and the users are not allowed to run those jobs on the computing resources of the NYSBC, at least not with a reasonable number of cores/job.
  3. Install a package like Appion, which also wraps many other EM programs is complicated in an external institution cluster. But I am trying.
  4. There is no feature that allows to remove automatically particles from the carbon with enough accuracy. I have to do it manually. So I use EMAN.
  5. Relion is my first choice for data processing. I am familiar with the package, with its original suite (Xmipp) and the developer. Besides, it gives excellent results.
  6. Relion likes to do its own gray-scale normalization, so particles must be extracted with Relion, as recommended by the developer.

Therefore, for my own preferences, but also for practical reasons (specially reasons 4 and 6), my possibilities are either pick the particles in Appion or not to do it, but the particle extraction must be performed by Relion. At least it is strongly recommended by the developer in order to have the best results. Correct me if I am wrong.

So I read the answer from Neil, but I didn't consider that option because the reasons that I am explaining.

Thank you!

Yacob

From: Bridget Carragher [bcarr@nysbc.org]

Hi Yacob, As you see various people are thinking about this. It is likely some issue of assuming that images are square - as indeed the always used to be! Anyway as Neil mentioned, this bug should not affect you as the stack maker should automatically ignore any parties that close to an edge of the image. Let us know if you want to discuss more. Best regards, Bridget

leginonbot commented 6 months ago

Original Redmine Comment Author Name: Neil Voss (@vosslab) Original Date: 2016-08-05T17:25:44Z

Hi Yacob,

Can you upload an image and template of yours, so I can test it on my machine?

We do have "border threshold" to exclude particles that are picked at X pixels of the border of the micrograph. But appears to be not working in your case. The "X pixels" border is based on the diameter input for the template masking.

leginonbot commented 6 months ago

Original Redmine Comment Author Name: Neil Voss (@vosslab) Original Date: 2016-08-05T17:30:48Z

Hi Yacob,

I can see this is for session 16jul22, so I can see if I get access to the image and test it out.

leginonbot commented 6 months ago

Original Redmine Comment Author Name: Yacob Gomez-Llorente (Yacob Gomez-Llorente) Original Date: 2016-08-05T18:23:30Z

Hi guys, thank you for taking your time and for your dedication to work on this. I will try your suggestion Bridget if the problem is not solve the next time I pick particles with Appion. I think you can access the files of the templates directly here, Neil:

/gpfs/appion/ygomez/16jul22b/extract/run1/origTemplate??.mrc

Also, for the micrographs, try anyone here:

/gpfs/leginon/ygomez/16jul22b/rawdata/16jul22b*enn-a3.mrc

Yacob