cannot reproduce the simulated image by using the simulator in cisTEM of alpha version

[chhuangAli]

Dear all, I was using the simulator in cisTEM of alpha version with the tutorial written on the cisTEM-dev website: https://bhimes.github.io/cisTEM_docs/docs/sim/tutorials/tutorials.html . In that tutorial, section 3, calculate a stack of noisy particles shows steps to simulate an isolated protein. I followed the instructions and I could reproduce Figures 1 A and B by modifying and executing bgal_flat.sh on my ubuntu 20.04. However, for Figure 1 C, after removing the flag, --save-detector-wavefunction, from optional_args in shell script (bgal_flat.sh) and executing the shell script, the generated image does not look identical with Figure 1 C which is very dark. In fact the generated image looks similar to Figure 1 B. I have no clue why this difference happened. Could you help me with this situation ?

Best, Chun Lu

Below is the additional information on the issue that I encountered.

To Reproduce Steps to reproduce the behavior:

0. Download alpha version of cisTEM from https://bhimes.github.io/cisTEM_docs/docs/sim/tutorials/get_cistem.html to a folder, let's say cisTEM_alpha/
1. Move to cisTEM_alpha/ and create a PDB folder to download a PDB and a run script files with "wget https://github.com/bHimes/cisTEM_docs/raw/main/tutorial_materials/bgal_flat.pdb https://github.com/bHimes/cisTEM_docs/raw/main/tutorial_materials/bgal_flat.sh"
2. Move to PDB folder and modify the variables and flags, path_to_cistem="path_to_cisTEM_alpha/src" output_filename="betgal_1elec_per_angSq.mrc", water_scaling=1.0, and optional_args=" --skip-random-angles " in bgal_flat.sh
3. Launch by typing ./bgal_flat.sh
4. The generated image with the name, betgal_1elec_per_angSq.mrc, is generated.

Expected behavior That generated image is supposed to look like Figure 1C shown on https://bhimes.github.io/cisTEM_docs/docs/sim/tutorials/tutorials.html But my resulted image does not but looks like Figure 1B.

Screenshots My resulted image can be found through this link: https://cistem.org/sites/default/files/Simulated_betagal_in_water_with_low_dose.png

Desktop (please complete the following information):

• OS: ubuntu 20.04

Additional context My output message on terminal after running bgal_flat.sh is written below: Welcome to Simulator

         Version : 0.25
        Compiled : May  5 2021
            Mode : Scripted

Output filename : betgal_1elec_per_angSq.mrc Make a 3d scattering potential? : -320 Number of threads : 16 PDB file name : bgal_flat.pdb Add another type of particle? : no Output pixel size (Angstroms) : 1.0 Linear scaling of per atom bFactor : 1.0 Per atom (xtal) bFactor added to all atoms : 4.0

            wanted size is 320

Create a tilt-series as well? : no Create a particle stack? : 1 Making a particle stack with 1 images wanted defocus (Angstroms) : 8000 wanted additional phase shift x * PI radians, i.e. 0.5 for PI/2 shift. : 0electrons/Ang^2 in a frame at the specimen : 1.5 electrons/Pixel/sec : 3.0 number of frames per movie (micrograph or tilt) : 1 Set expert options? : yes Use an existing set of orientations : no Linear scaling water intensity : 1.0 fraction of the defocus to add as astigmatism : 0.0 Accelrating volatage (kV) : 300 Objective aperture diameter in micron : 100 Spherical aberration constant in millimeters : 2.7 Std deviation of error to use in shifts, astigmatism, rotations etc. : 0.0Pre-exposure in electron/A^2 : 0 minimum padding of images with solvent : 32 minimum thickness in Z : 10 Propagation distance in angstrom : 5 Rotation of tilt-axis from Y (degrees) : 0.0 Standard deviation on angles in plane (degrees) : 2.0 Standard deviation on tilt-angles (degrees) : 0.1 Standard deviation on magnification (fraction) : 0.0001 Beam-tilt in X (milli radian) : 0.0 Beam-tilt in Y (milli radian) : 0.0 Beam-tilt particle shift in X (Angstrom) : 0.0 Beam-tilt particle shift in Y (Angstrom) : 0.0 Max particles is 0 here

In constructor real total current 32824 32824 32824

Number of particles initialized 1

PDB center of mass at 93.002000 93.002000 93.002000 (x,y,z Angstrom)

Setting origin there.

There are 32824 non-water atoms in the specimen.

Current number of PDBs 1 Using extra phase shift of 0.000000 radians Got here emulate is 0.000000

There are 1 tilts for Tilt 0, scaling the pixel size from 1.000 to 1.000

Working on iParticle 0/ 1

For the particle stack, stretching the defocus by 0.00 percent and randmozing the astigmatism angle -90,90Scaling the defocus by 1.000000 to match the def at 300 KeV

AVG BFACTOR FROM PDB IS 19.164429

max/min xyz, 90.431000,-90.431000 68.182007,-68.181999 45.899002,-45.898998 min is 10.000000, shift is 0, max_depth is 91

    for frame 0 the size neigborhood is 2

using neighborhood of 125.00 vox^3 for waters and 125.00 vox^3 for non-waters Starting projected water calc with sizeN 5, 5 Finishing projected water calc

            wanted size is 407

Is single particle is 1 size pre rot padding 380 335 0.000000 rot size post rot 1 padding 787 742 0.000000 rot size post rot 2 padding 787 742 padX 407 padY 407 padZ 0 rot vol dimension in Ang 787.00 x 742.00 y 104.00 z

    Drift for iTilt 0, iFrame 0 is 0.0000 Ang

Atoms per nm^3 31.422, vol (in Ang^3) 787.00 742.00 104.00 specimen volume is 6.073e+04 nm expecting 1.908e+06 waters cuttoff is 1.908314e+06 5.925180e+07 0.967793 -0.967793 waters added 1.933e+06 (101.28%) Using a rmsd of 2.250000 for water perturbation wZ 104 csZ 104,rotZ 104

Working on iTilt 0 at 119.805107 degrees for frame 0 rotated_Z 104 nSlabs 21

nSlabs 21 iSlab 6 lSlab 5

Found a scattering cetner of mass at -53.511 Ang

Exposure is 1.500 for frame -53.511456 -5.000000 -2.476191 Propagator distance is -5.000e+00 Angstroms, with offset for CTF of -5.101e+01 Angstroms for the specimen.

    0 out of bounds of 32824 = percent

8.00000e+03 8.00000e+03

Astigmatism params are 0.000e+00 8.365e+01

SEED is 1639468828480218300e

./simulator_ctf_303DESKTOP-2HAG7BR.sh: line 3: ctffind: command not found tail: cannot open 'fit_303DESKTOP-2HAG7BR_1.txt.txt' for reading: No such file or directory Failed on the following record : From Line : From /groups/grigorieff/home/himesb/cisTEM_downstream_bah/src/core/numeric_text_file.cpp:261 void NumericTextFile::ReadLine(float *)

[chhuangAli]

Hi,

In order to overcome the simulation issue that I described above, I tried two ways:

1. Because I was using the binary for the simulator alone in a folder, the simulator couldn't find the ctffind. I then put ctffind file together with simulator, and launched the simulation again. But the simulator didn't find the ctffind still. Actually, it needs a command, /absolute-path/cttfind, instead of ctffind. This means I need to modify the source code, simulate.cpp , to put 'absolute-path' , which is my second way described below.
2. In simulate.cpp, I created a new flag, named as "ctffind-location", by using a wxString variable specifying the absolute path to the ctffind. The variable is passed to WaveFunctionPropagator::ReturnImageContrast() where ctffind is called.

After implementing the second modification in the codes, the simulator can call ctffind to determine defocus parameters at first stage during the simulation. Also, the simulation can finsh properly.

[bHimes]

Thanks for troubleshooting this @chhuangAli. I'll leave the issue open so that I can amend the tutorial docs/scripts to prevent this in the future.

I am curious though - why go to all the effort of modifying the source code rather than just modifying your environmental PATH variable?

Once you added the ctffind binary to the same folder as your simulate binary, it would just be a simple

export PATH=/path/to/alpha:$PATH

in the script you use to run the simulation.

[chhuangAli]

Thanks for the reply @bHimes. I included the simulator path into the envrionment variable, PATH and executed the binary of simulator, which returned the reasonable simulated images. I was thinking to add a path variable is useful because there is a shell script written all of the input variables into the simulation. So the user can also set the path in the shell script without specifying the simulator path whenever execution. Also, I was interested in the working mechanism inside the simulator in the aspect of techinique and physics, for example, the wanted size of simulation (I was confused the negative size at begining when I wanted to increase larger size of the simulated movie frame. ) so I modified directly the source code.