Unrealistic noise level when pixel value is other than 1 Angstrom

[MohamadHarastani]

Hi James,

I noticed when setting the detector pixel size to 2 that the level of noise is too high. The image on the left is when the pixel size is 1 A, and on the right when it is 0.5 A (with half the detector size, so the two images should correspond to each other).

Cheers Mohamad

[jmp1985]

Hi Mohamad

Sorry for the late reply.

For the image with the pixel size of 0.5A can you try rebinning with a function like the following:

def rebin(data, shape):
    shape = (shape[0], data.shape[0] // shape[0],
             shape[1], data.shape[1] // shape[1])
    return data.reshape(shape).sum(-1).sum(1)

That should rebin summing the pixel values. When you have a smaller pixel size it is sometimes more difficult to see features because of the noise but if you bin the data to the same pixel size hopefully the features should be visible. Let me know if this works or not.

Best wishes James

[MohamadHarastani]

Hi James Thanks for the reply. I can of course bin the data after creating it with pixel size of 1 A. Generating data with pixel size of 0.5 A is fine, but for 2 A is not. I have data of the same molecule in vitro that I am trying to match with simulation, we'll discuss it at later moment, just keep it in mind. Cheers Mohamad

On Wed, Mar 1, 2023, 9:46 AM James Parkhurst @.***> wrote:

Hi Mohamad

Sorry for the late reply.

For the image with the pixel size of 0.5A can you try rebinning with a function like the following:

def rebin(data, shape): shape = (shape[0], data.shape[0] // shape[0], shape[1], data.shape[1] // shape[1]) return data.reshape(shape).sum(-1).sum(1)

That should rebin summing the pixel values. When you have a smaller pixel size it is sometimes more difficult to see features because of the noise but if you bin the data to the same pixel size hopefully the features should be visible. Let me know if this works or not.

Best wishes James

— Reply to this email directly, view it on GitHub https://github.com/rosalindfranklininstitute/parakeet/issues/47#issuecomment-1449588105, or unsubscribe https://github.com/notifications/unsubscribe-auth/AK2I5FO5FWG44GXQ5ZKCRO3WZ4EHZANCNFSM6AAAAAAVIQVHP4 . You are receiving this because you authored the thread.Message ID: @.***>

[jmp1985]

Hi Mohamad

Ah, sorry I misunderstood. When simulating with a 2A pixel size, it may also be better to simulate with 1A or 0.5A and then rebin to the desired size. This is because the potential is only evaluated once per pixel so you get a better representation of the potential by oversampling and then rebinning to the desired output size. I have been meaning to add an option to make this more streamlined so that it is done during exit wave calculation and rebinning happens automatically but at the moment you have to do an extra export step at the end.

Best wishes James

[MohamadHarastani]

Thanks James, its clear now! I will be using pixel size of 1 A until what you planned is done. Btw, from time to time, I match the recent changes in Parakeet with the Scipion plugin, but sometimes it is not easy to figure out what changed. May I suggest adding hints to recent changes to CHANGES.txt file like this for example? I have been using Parakeet in a project and refining things in the pluygin in the meantime, in a couple more weeks I will be back to testing new functionalities. Cheers Mohamad

[jmp1985]

Hi Mohamad

I've been considering learning how to use the towncrier (https://towncrier.readthedocs.io) utility for the purpose of creating more useful changelogs. For future updates I will try and make it clearer what has changed!

Best wishes James

[jmp1985]

Hi Mohamad

Just to let you know that I have started a file (https://github.com/rosalindfranklininstitute/parakeet/blob/master/CHANGES.rst) which can also be accessed from the README page to which I will add user friendly descriptions of changes in future.

Best wishes James

[MohamadHarastani]

Hi James I appreciate it very much! Thanks.

Cheers Mohamad

[Nordicus]

This may be related to an issue I've been having which is that with pixel sizes larger than 1A, the ice model isn't generated.

E.g.,

microscope:
  beam:
    electrons_per_angstrom: 5.6
    energy: 300
    illumination_semiangle: 0
  detector:
    dqe: false
    nx: 197
    ny: 197
# Ice generated
    pixel_size: 0.6327964285714286
# Ice generated
#    pixel_size: 1
# No ice generated
#    pixel_size: 2.5311857142857144
  lens:
    c_10: -20000
    c_30: 2.7
    c_c: 2.7
    outer_aper_ang: 3.5
sample:
  box:
  - 500
  - 500
  - 1000
  centre:
  - 250
  - 250
  - 500
  molecules:
    local:
    - filename: apo109.pdb
      instances: 3
  shape:
    cuboid:
      length_x: 500
      length_y: 500
      length_z: 250.0
    margin:
    - 0
    - 0
    - 0
    type: cuboid
scan:
  mode: manual
  positions:
  - - 31.25
    - 31.25
simulation:
  ice: true
  margin: 1
  padding: 0
  radiation_damage_model: false
  sensitivity_coefficient: 0.022
  slice_thickness: 1.0

[jmp1985]

Hi @Nordicus

I have had a look and there was a bug with a for loop going out of range in C++ which I have fixed. Hopefully this should allow the ice model for pixel sizes > 2A

Best wishes

James