Priodose in mdoc file

[hamid13r]

Dear Fabian,

We are doing a lot of subtomo averaging where read dose comes into the calculation and Warp reads the dose from the mdoc file using the PriorRecordDose string. I just noticed that PACEtomo-generated mdoc files don't record the prior dose, which throws Warp off.

Is there something we are missing or this just needs to be added to pacetomo script?

Best, Hamid

[hamid13r]

Actually, I just checked and we need the prior dose to be in all the mdoc files, not just the ones generated for tilt-series.

[eisfabian]

Hey Hamid,

Thanks for letting me know! This is normal behavior. I am not aware of anything outside the built-in SerialEM tilt series controller keeping track of the dose. Did you use other scripts that were able to do it? What do you mean by other mdoc files than the tilt series mdocs?

I have never worked on a scope with calibrated dose to be honest, so I didn't realize it's an issue.

That being said, it should be possible to do. There is a ImageConditions buf command that gives the dose for the image in a buffer. I just have to keep track of it, manually sum it up and add it to the mdoc.

I will put it on my to-do list!

Thanks again and best, Fabian

[hamid13r]

Sorry, I wasn't clear. There is an mdoc file with every frame-stack (in frames/) and there is an mdoc file for every tilt-series (in PACE/). Previously we were using serialEM itself to collect our tilt-series. I am attaching an example from each (renamed txt due to github policy). It is line 28 in serialEM.txt that we are missing. Same line also exists in the tilt-serie mdoc file.

This is only a problem for subtomo averaging, so at the moment I have a python code to edit our tomostar files after they are created by Warp to get the correct doses.

Thanks for all the help!

pactomo.txt serialEM.txt

[eisfabian]

Hey Hamid,

I tried to keep track of the dose per target and added a PriorRecordDose parameter to the corresponding mdoc file. This change is in v1.6.1.

The frame mdoc files are optional and completely generated by SerialEM. I personally never save them. Does Warp require them? Can you make it use the stack's mdoc PriorDose value instead?

Hope this solves your issue for the most part! Let me know if you run into any problems with it!

Best wishes, Fabian

[hamid13r]

Thank you,

I will try it the first chance I get. Unfortunately, WARP is not very flexible on that regard and so far I have written a python code to fix the tomostar files. One of the things we keep having to deal with is where warp reads various info while processing data.

If the frame mdoc file is generated by SerialEM, is there anything we can look at to see why the prior-dose is not included when we run PACEtomo? Is this something the serialEM forum could help with?

[eisfabian]

The frame mdoc is generated by SerialEM, but SerialEM itself does not have any information about the tilt series recorded by a custom script like PACEtomo. There is actually a command to tell SerialEM about accumulated dose, but it assumes that the script is only run on one position, so it will accumulate the dose of all tilt series, which is not very helpful. I will ask David if there is another way to deal with it, but it might take a while to solve this properly.

[hamid13r]

Thank you for the help Fabian,

To be clear, PACEtomo is completely usable in its current state and we are enjoying the increased throughput. It would be nice if we can have the prior dose recorded automaticaly but at the moment we are goimg around Warp using a simple python script.

Best, Hamid

On Sun, Jun 18, 2023, 19:59 eisfabian @.***> wrote:

The frame mdoc is generated by SerialEM, but SerialEM itself does not have any information about the tilt series recorded by a custom script like PACEtomo. There is actually a command to tell SerialEM about accumulated dose, but it assumes that the script is only run on one position, so it will accumulate the dose of all tilt series, which is not very helpful. I will ask David if there is another way to deal with it, but it might take a while to solve this properly.

— Reply to this email directly, view it on GitHub https://github.com/eisfabian/PACEtomo/issues/20#issuecomment-1596418309, or unsubscribe https://github.com/notifications/unsubscribe-auth/AP7A5CNDLRLHF75GE67MNITXL66CRANCNFSM6AAAAAAX2LB7LM . You are receiving this because you authored the thread.Message ID: @.***>