Differences between pyprismatic metadata and saved parameter file

[thomasaarholt]

Running a simulation with pyprismatic, and then running the same simulation but using prismatic CLI on prismatic_gui_params.txt that is saved during the first simulation, produces different results.

I first noticed this because the first simulation had over 30000 probe positions, but the second one only had 100. Comparing the output printed in the terminal when the simulation starts, I find these two differences.

# First simulation with pyprismatic
Simulation parameters:
...
integrationAngleMin = 0
integrationAngleMax = 1e-06
randomSeed = 170908
crop4Damax = 0.1

# Second simulation with prismatic -pf prismatic_gui_params.txt
integrationAngleMin = 0
integrationAngleMax = 0.001
randomSeed = 170908
crop4Damax = 0.0001

I've tried setting integrationAngleMax back to 1e-6 manually, without success.

[lerandc]

hmm neither of those settings should affect the number of probe positions, though I also can't actually find the off by 1000 discrepancy between the pyprismatic core and the metadata handling in the main source. could you share the full parameter file and the settings part of your pyprismatic script?

[thomasaarholt]

Of course! I've just tested on my local machine, and I can reproduce on Windows. prismatic_gui_params.txt model.zip

In Pyprismatic I used the following code to set the parameters. I've also included the toString() output for a full overview of what pyprismatic thinks is going on.

Also note that loading the parameter file into prismatic_gui 1.2.1 cpu_h3eaf5d3_7 conda-forge (I'm here using a conda-installed prismatic-gui from 3 days ago), shows that the beam energy, and anything involving mrad are off by a factor of 1000 (maybe fixed with @ericpre's comment in #83). The GUI also runs with 100 probe steps.

# Note that this expects the xyz file in a "cells/" directory and the sim output is in "sims/".
import pyprismatic as pr
meta = pr.Metadata(filenameAtoms="cells/n10_Prismatic_RT.xyz")
meta.filenameOutput = "sims/n10.emd"
meta.E0 = 3e5
meta.probeStepX = 0.1
meta.probeStepY = 0.1
meta.realspacePixelSizeX = 0.05
meta.realspacePixelSizeY = 0.05

meta.includeThermalEffects = False
meta.numFP = 1
meta.potBound = 2.0

meta.save3DOutput = True
meta.save4DOutput = False
meta.sliceThickness = 2.2425
meta.tileX = 1
meta.tileY = 1

meta.numThreads = 1
meta.numGPUs = 4
meta.alsoDoCPUWork = False
#meta.go()

# meta.toString()
interpolationFactorX = 4
interpolationFactorY = 4
filenameAtoms = cells/n10_Prismatic_RT.xyz
filenameOutput = sims/n10.emd
realspacePixelSizeX = 0.05
realspacePixelSizeY = 0.05
potBound = 2.0
numFP = 1
sliceThickness = 2.2425
cellDimX = 19.0314
cellDimY = 19.0314
cellDimZ = 65.85205
tileX = 1
tileY = 1
tileZ = 1
E0 = 300000.0
alphaBeamMax = 0.024
numGPUs = 4
numStreamsPerGPU = 3
numThreads = 1
batchSizeTargetCPU = 1
batchSizeTargetGPU = 2
earlyCPUStopCount = 100.0
probeStepX = 0.1
probeStepY = 0.1
probeDefocus = 0.0
C3 = 0.0
C5 = 0.0
probeSemiangle = 0.02
detectorAngleStep = 0.001
probeXtilt = 0.0
probeYtilt = 0.0
scanWindowXMin = 0.0
scanWindowXMax = 0.99999
scanWindowYMin = 0.0
scanWindowYMax = 0.99999
randomSeed = 251137
algorithm = prism
includeThermalEffects = False
alsoDoCPUWork = False
save2DOutput = False
save3DOutput = True
save4DOutput = False
integrationAngleMin = 0
integrationAngleMax = 0.001
transferMode = auto
saveDPC_CoM = False
savePotentialSlices = False
nyquistSampling = False
numSlices = 0
zStart = 0.0
scanWindowXMin_r = 0.0
scanWindowXMax_r = 1.0
scanWindowYMin_r = 0.0
scanWindowYMax_r = 1.0

[thomasaarholt]

Regarding the original 30000 vs 100 probe positions: Even when I correct all the values (I think), I still get 100 probe positions.

With a cell XY dimension of 19Å x 19Å, I would expect a 0.1 probe spacing to give 190 x 190 = 36100 positions. This is what I get with pyprismatic.

[ericpre]

I have seen this but I don't remember exactly what was the cause... it was related with parsing the initial parameter the first time you start the GUI or something along these lines. I would try to run your prismatic_gui_params.txt with the prismatic cli and I would expect the results to be different and if this is the case it would help narrowing down the bug.

[lerandc]

I think I've figured out (also working with a user right now who has similar issues)-- in your parameter file, do you have two lines --scan-window-xr:0 1 and --scan-window-yr:0 1? these options are saved automatically to the output file, but are likely overwriting the scan window settings to use real space coordinates instead of fractional coordinates-- I can almost certainly figure out a cleaner solution to this default overriding/metadata saving behavior

[thomasaarholt]

Ah! Yes, that's definitely it! Instead of fractional units, it then becomes 0 to 1Å in steps of 0.1Å! Which is 10 x 10 = 100 steps :)

[lerandc]

yup! what a pernicious bug, should be an easy fix though