ECP-WarpX / impactx

high-performance modeling of beam dynamics in particle accelerators with collective effects
https://impactx.readthedocs.io
Other
26 stars 20 forks source link

Document Conversion from Alternative Input Parameters #581

Closed zhangli28 closed 5 months ago

zhangli28 commented 5 months ago

Dear ImpactX developers,

While using impactX, I feel that giving a electron beam input parameters is not very clear for the user. I suggest adding some additional features of loading electron beam as available in FBPIC code.

Thank you.

zhangli28 commented 5 months ago

It might be good idea to give the input electron beam as one of the options

sigma_x, sigma_y, where, x & y are transverse coordinates.

sigma_tau, represent longitudinal length of the electron beam.

emit, represent emittance of the beam

gamma, is the beam energy

sigma_gam, is energy spread

chirp, amount of chirp in the electron beam phase-space

Q represent total charge of the electron beam

npart represent number of particles.

Such representation of input electron beam is much clear and easy to give input.

ax3l commented 5 months ago

We might need to explicitly mention in the docs how to calculate from these common inputs into our distributions?

X-ref: #580

zhangli28 commented 5 months ago

Dear everyone (@n01r @ax3l @cemitch99), thank you for considering the request. It will be very helpful.

zhangli28 commented 5 months ago

Hi everyone @n01r @ax3l @cemitch99

sorry for bothering again. Just curious to know when flexible input parameters capability will be available to use ?

Thank you. zhang

n01r commented 5 months ago

Hi @zhangli28, is there anything missing fpr you in making the conversion to your input parameters right now? Please let us know. :) There is no need for you to be subject to our development cycle. As I understand you can use the Python input and have now the equations that convert to our input. So you're good to go, right?

We will implement more input methods soon but that should not stop you from running simulations already. Let us know if there are still things that are unclear! 😊

zhangli28 commented 5 months ago

Hi @n01r ,

Thank you very much for answering every time. Please, see below what I was able to do so far

#-------------------------------------------------------------------------------

# Electron beam parameters that I want to use as an input are as follows:

sigma_x = 50.0e-6     # in meters (beam size in transverse x-direction).
sigma_y = 50.0e-6     # in meters (beam size in transverse y-direction).
sigma_tau = 0.1e-3    # c*tau in meters (beam length).

sigma_px = 0.21e-3    # px/pz, beam x-divergence in radian.
sigma_py = 0.21e-3    # px/pz, beam y-divergence in radian.

emitt_x_n  = 2.0      # beam normalized or projected emittance in mm. mrad.
emitt_y_n  = 2.0      # beam normalized or projected emittance in mm. mrad.

E0 = 0.1              # beam energy in GeV .
sigma_p = 1.0e-2      # beam momentum spread.
energy chirp = 0.01

charge = 50.0e-12       # beam charge in Coulomb.

nparticles = 20000

#-------------------------------------------------------------------------------

# From the above electron beam parameters I derived twiss parameters.
# alpha_twiss, beta_twiss and gamma_twiss are:

emit_x  = 1.0547793826948454e-08    # beam geometrical emittance in x-dir
emit_y  = 1.0719290266414234e-08    # beam geometrical emittance in y-dir
beta_x  = 0.23668661414799747
beta_y  = 0.2381422534294453
alpha_x = 0.010820028000209588
alpha_y = 0.006184366654799564
gamma_x = 4.225490641310891
gamma_y = 4.199331416367921

# Now how to proceed further becuase alpha_t, beta_t and gamma_t are missing ??
cemitch99 commented 5 months ago

Hi @zhangli28. You can obtain this information from the three values: sigma_tau, sigma_p, and the energy chirp. Let us assume that sigma_p is the uncorrelated relative momentum spread (dimensionless), and the energy chirp (call it h) is in units of 1/meter. Then, at values of relativistic beta ~ 1 (as in your case):

emit_t = sigma_tausigma_p beta_t = sigma_tau / sigma_p alpha_t = -h beta_t

This comes from writing down the 2x2 covariance matrix in the longitudinal variables. Just be careful with the sign of alpha_t, because this depends on the sign convention used to define the energy chirp.

n01r commented 5 months ago

Hi @zhangli28, I think I see the problem.

You have the $\sigmat$ and $\sigma{p_t}$ but need a correlation term to convert to our inputs. I think this correlation term is in the parameter you call energy_chirp. But it may not yet be in the correct units. Could you check what units this parameter is in?

n01r commented 5 months ago

Ah, sorry @cemitch99, we answered at the same time. But your answer gets us further. 🙂

zhangli28 commented 5 months ago

Hi @n01r and @cemitch99 ,

Thank you so much to both of you. You all are great guys. Let me try one again to run the chicane example. I will report you whatever the result is.

zhangli28 commented 5 months ago

Hi @n01r @cemitch99 ,

I have some doubt about the chicane set up used in the example file "input_chicane.in", whose values are taken from the reference https://www.desy.de/csr/

I searched about my doubt on internet but not able to get it clear. So, I returned back to you.

Please can you see the attached pdf file (ImpactX_Chicane_setup). Here I try to explain my question. If these questions can be answered, I am almost ready to do what I am trying from many days.

Thank you. Zhang

ImpactX_Chicane_setup.pdf

n01r commented 5 months ago

Hi @zhangli28, when I look at the MAD-X input file for the DESY-Zeuthen chicane benchmark I can see the following

! magnet-to-magnet path lengths

  LD   := 5.0                 ! outer bend-to-bend "Z" distance (m)
  LDo  := LD/COS(AN)          ! outer bend-to-bend path length (m)
  LDi  := 1.0                 ! inner bend-to-bend "Z" distance (m)

and from the schematics on the web page and in your PDF it seems to me that the 5 meters are the projected lengths in z-direction. But ImpactX is an s-based code which gives the length of elements as the actual path length. When I calculate $5.00 / \cos(2.77^\circ)$ I am getting 5.0058489435 like what is written here in the input file. https://github.com/ECP-WarpX/impactx/blob/08e55a24a5a4e08b422d1db6d9a2c1294b8604b6/examples/chicane/input_chicane.in#L33

@cemitch99 am I correct in assuming that this was the rationale behind these parameters? The parameter sbend.psi = 0.048345620280243 is just the 2.77° angle in radians.

If these questions can be answered, I am almost ready to do what I am trying from many days.

What else is blocking you from doing what you intended? Or what did you intend to do?

cemitch99 commented 5 months ago

Hi @n01r, that's exactly right. Regarding the magnets themselves: the value of ds in the ImpactX input file is the arc length of the magnet, while the schematic on the web page gives the projected length along the z-direction. Some geometry will show you that the two correspond.

zhangli28 commented 5 months ago

Hi @n01r @cemitch99 ,

thank you so much for your help. Now, everything is there to start my run, however, one thing is not looking correct in the example file and that is the value sbend1.ds = 0.50037.

The length of the magnet is 0.5 m. If we calculate like LDo := LD/COS(AN) then this should be sbend1.ds = 0.50058489435. Please, correct me if I am wrong ?

n01r commented 5 months ago

LD is for the drift. The magnet is first creating the angle of 2.77 deg, so the arc length is less than 0.5 m / cos(psi). Only during the drift the beam propagates at an angle of 2.77 deg and hence that is where LDo := LD/COS(AN) applies. You are mixing up the bends and the drifts, it seems.

sbend1.type = sbend
sbend1.ds = 0.50037       # projected length 0.5 m, angle 2.77 deg
sbend1.rc = -10.35

drift1.type = drift
drift1.ds = 5.0058489435  # projected length 5 m
zhangli28 commented 5 months ago

Hi @n01r, yes you are right. Then how sbend1.ds = 0.50037 instead of 0.5 (I am still not able to figure it out).

zhangli28 commented 5 months ago

Hi @n01r ,

following the the MAD-X input file for the DESY-Zeuthen chicane, I did as follows

LB = 0.5               # BC2-Bend "Z" length [m]
Cb = 1.0e10/c.        # c is speed of light
EBC2 = 5.0
Brho = Cb*EBC2
BB = -16.120960797916
RB = Brho/BB
AN = np.arcsin(LB/RB)
LBS = RB*AN            # chicane bend path length (m)

but this gives me LBS=0.5001948493121843 still not sbend1.ds = 0.50037

cemitch99 commented 5 months ago

Hi @zhangli28,

Yes, it looks like the result you quote above for LBS is correct. Attached is a figure illustrating the geometry. Thanks for pointing out this apparent 10^(-4) relative error in ds, which will be addressed. RectangularBendGeometry.pdf

zhangli28 commented 5 months ago

Hi @n01r, @cemitch99

very very thanks to both of you. Without your help it was not possible for me to understand things clearly. Now, I am fully ready to make the test run I was looking for. I will tell you later how it goes.

Cheers, Zhang

zhangli28 commented 5 months ago

Hi @n01r, @cemitch99 ,

today, before running my new test script, just to be sure that everything is working fine, I once again made test runs of the examples given for the chicane: run_chicane.in and run_chicane.py.

The python script (run_chicane.py) is working fine generating output in diags folder, but when I was trying to run run_chicane.in it gave me an error

./impactx: /sqfs/software/Anaconda3/lib/libstdc++.so.6: version `GLIBCXX_3.4.26' not found (required by ./impactx)

./impactx: /sqfs/software/Anaconda3/lib/libstdc++.so.6: version `GLIBCXX_3.4.26' not found (required by /sqfs/home/zhangli4/impactx/build/lib/libamrex_3d.so)

./impactx: /sqfs/software/Anaconda3/lib/libstdc++.so.6: version `GLIBCXX_3.4.30' not found (required by /sqfs/home/zhangli4/impactx/build/lib/libamrex_3d.so)

./impactx: /sqfs/software/Anaconda3/lib/libstdc++.so.6: version `GLIBCXX_3.4.26' not found (required by /sqfs/home/zhangli4/conda_env/impactx-cpu/lib/./libadios2_core.so.2)

The same error message appear when I tried my new input script (*.in format). It looks like I can run the python script but not the *.in files. The new input script file is based on structure of input_kvdist_twiss.in. Therefore, I cannot run it until the problem is resolved.

Do you have any suggestions how to fix this.

ax3l commented 5 months ago

Hi @zhangli28,

the reason this is failing at runtime is that you likely activated a conda environment (and/or a specific set of HPC software modules) when you compiled ImpactX - but then did not load it again at runtime.

zhangli28 commented 5 months ago

Hi @ax3l

I removed the build directory and re-build again as follows:

module purge
module load gcc/11.2.0
module load anaconda/3
module load shared
module load hdf5/1.12.1

source activate impactx-cpu

#cmake -S . -B build -DImpactX_PYTHON=ON

cmake -DMPI_C_COMPILER=/sqfs/home/zhangli4/conda_env/impactx-cpu/bin/mpicc -DMPI_CXX_COMPILER=/sqfs/home/zhangli4/conda_env/impactx-cpu/bin/mpicxx -S . -B build -DImpactX_MPI=ON -DImpactX_PYTHON=ON -DPython_EXECUTABLE=/sqfs/home/zhangli4/conda_env/impactx-cpu/bin/python

and then I install as

module load gcc/11.2.0
module load anaconda/3
module load shared
module load hdf5/1.12.1

source activate impactx-cpu

#cmake --buld build --target pip_install -j 4
cmake --buld build -j 4

then, I copied the impactx* executable in my working directory and uses the command ./impactx filename.in in my job script.

n01r commented 5 months ago

@zhangli28, what @ax3l was referring to was that not only do you need to have your modules loaded (ideally via a impactx.profile file for simplicity) during the compilation/installation of the code but also when you run it.

Can you confirm that all modules were loaded and paths were set?

@ax3l, during the installation @zhangli28 had to specify MPI_C_COMPILER and MPI_CXX_COMPILER directly ... probably because some paths on their home cluster are set automatically on login and thus it wouldn't recognize the compilers directly.

zhangli28 commented 5 months ago

@ax3l , @n01r when I type strings /usr/lib/libstdc++.so.6 | grep GLIBCXX on the terminal, it shows:

GLIBCXX_3.4
GLIBCXX_3.4.1
GLIBCXX_3.4.2
GLIBCXX_3.4.3
GLIBCXX_3.4.4
GLIBCXX_3.4.5
GLIBCXX_3.4.6
GLIBCXX_3.4.7
GLIBCXX_3.4.8
GLIBCXX_3.4.9
GLIBCXX_3.4.10
GLIBCXX_3.4.11
GLIBCXX_3.4.12
GLIBCXX_3.4.13
GLIBCXX_3.4.14
GLIBCXX_3.4.15
GLIBCXX_3.4.16
GLIBCXX_3.4.17
GLIBCXX_3.4.18
GLIBCXX_3.4.19
GLIBCXX_3.4.20
GLIBCXX_3.4.21
GLIBCXX_3.4.22
GLIBCXX_3.4.23
GLIBCXX_3.4.24
GLIBCXX_3.4.25
GLIBCXX_DEBUG_MESSAGE_LENGTH
GA+GLIBCXX_ASSERTIONS
GA+GLIBCXX_ASSERTIONS

This means the error ./impactx: /sqfs/software/Anaconda3/lib/libstdc++.so.6: version `GLIBCXX_3.4.26' not found (required by ./impactx) is obvious. Anyways, I am trying to install everything from scratch. Let see if things works or not.

zhangli28 commented 5 months ago

Hi @n01r @cemitch99 @ax3l ,

I made the build directory using the command

cmake -DMPI_C_COMPILER=/sqfs/home/zhangli4/conda_env/impactx-cpu/bin/mpicc -DMPI_CXX_COMPILER=/sqfs/home/zhangli4/conda_env/impactx-cpu/bin/mpicxx -S . -B build -DImpactX_MPI=ON -DImpactX_PYTHON=ON -DPython_EXECUTABLE=/gpfs/home/zhangli4/conda_env/impactx-cpu/bin/python

Now, suppose I used the command below for installation.

make --build build -j 4

After some time, if I wan to change the installation type and wan to use

cmake --build build --target install

can I do it without re-building ?

In general, can I use any of the command below as many times, and in any order, without any need to touch the build directory ?

make --build build -j 4 cmake --build build --target install cmake --build build --target pip_install -j 4

sorry for so many questions. I hope it will be ending soon.

zhangli28 commented 5 months ago

Hi @n01r , Even after re-installing the error remains.

./impactx: /sqfs/software/Anaconda3/lib/libstdc++.so.6: version `GLIBCXX_3.4.26' not found (required by ./impactx)

./impactx: /sqfs/software/Anaconda3/lib/libstdc++.so.6: version `GLIBCXX_3.4.26' not found (required by /sqfs/home/zhangli4/impactx/build/lib/libamrex_3d.so)

./impactx: /sqfs/software/Anaconda3/lib/libstdc++.so.6: version `GLIBCXX_3.4.30' not found (required by /sqfs/home/zhangli4/impactx/build/lib/libamrex_3d.so)

./impactx: /sqfs/software/Anaconda3/lib/libstdc++.so.6: version `GLIBCXX_3.4.

I am trying to google search about this erro and if you get any idea of removing this error please let me know.

Thank you.

zhangli28 commented 5 months ago

Hi @n01r @ax3l @cemitch99 ,

Maybe I am approaching towards the solution. This is because when I used the command

strings /sqfs/home/zhangli4/conda_env/impactx-cpu/lib/libstdc++.so.6 | grep GLIBCXX

I got the following:

GLIBCXX_3.4
GLIBCXX_3.4.1
GLIBCXX_3.4.2
GLIBCXX_3.4.3
GLIBCXX_3.4.4
GLIBCXX_3.4.5
GLIBCXX_3.4.6
GLIBCXX_3.4.7
GLIBCXX_3.4.8
GLIBCXX_3.4.9
GLIBCXX_3.4.10
GLIBCXX_3.4.11
GLIBCXX_3.4.12
GLIBCXX_3.4.13
GLIBCXX_3.4.14
GLIBCXX_3.4.15
GLIBCXX_3.4.16
GLIBCXX_3.4.17
GLIBCXX_3.4.18
GLIBCXX_3.4.19
GLIBCXX_3.4.20
GLIBCXX_3.4.21
GLIBCXX_3.4.22
GLIBCXX_3.4.23
GLIBCXX_3.4.24
GLIBCXX_3.4.25
GLIBCXX_3.4.26
GLIBCXX_3.4.27
GLIBCXX_3.4.28
GLIBCXX_3.4.29
GLIBCXX_3.4.30
GLIBCXX_DEBUG_MESSAGE_LENGTH
_ZNKSt14basic_ifstreamIcSt11char_traitsIcEE7is_openEv@GLIBCXX_3.4
_ZNSt13basic_istreamIwSt11char_traitsIwEE6ignoreEv@@GLIBCXX_3.4.5
_ZNKSbIwSt11char_traitsIwESaIwEE11_M_disjunctEPKw@GLIBCXX_3.4
_ZNKSt14basic_ifstreamIwSt11char_traitsIwEE7is_openEv@@GLIBCXX_3.4.5
GLIBCXX_3.4.21
GLIBCXX_3.4.9
_ZSt10adopt_lock@@GLIBCXX_3.4.11
GLIBCXX_3.4.10
GLIBCXX_3.4.16
GLIBCXX_3.4.1
_ZNSt19istreambuf_iteratorIcSt11char_traitsIcEEppEv@GLIBCXX_3.4
GLIBCXX_3.4.28
_ZNSs7_M_copyEPcPKcm@GLIBCXX_3.4
GLIBCXX_3.4.25
_ZNSt19istreambuf_iteratorIcSt11char_traitsIcEEppEv@@GLIBCXX_3.4.5
_ZNSs7_M_moveEPcPKcm@@GLIBCXX_3.4.5
_ZNKSt13basic_fstreamIwSt11char_traitsIwEE7is_openEv@GLIBCXX_3.4
_ZNKSt13basic_fstreamIcSt11char_traitsIcEE7is_openEv@GLIBCXX_3.4
_ZNSbIwSt11char_traitsIwESaIwEE4_Rep26_M_set_length_and_sharableEm@@GLIBCXX_3.4.5
_ZNSs4_Rep26_M_set_length_and_sharableEm@GLIBCXX_3.4
_ZSt10defer_lock@@GLIBCXX_3.4.11
_ZN10__gnu_norm15_List_node_base4swapERS0_S1_@@GLIBCXX_3.4
_ZNSs9_M_assignEPcmc@@GLIBCXX_3.4.5
_ZNKSbIwSt11char_traitsIwESaIwEE15_M_check_lengthEmmPKc@@GLIBCXX_3.4.5
_ZNKSt14basic_ifstreamIcSt11char_traitsIcEE7is_openEv@@GLIBCXX_3.4.5
_ZNSbIwSt11char_traitsIwESaIwEE7_M_moveEPwPKwm@GLIBCXX_3.4
GLIBCXX_3.4.24
_ZNVSt9__atomic011atomic_flag12test_and_setESt12memory_order@@GLIBCXX_3.4.11
GLIBCXX_3.4.20
_ZNSt11char_traitsIwE2eqERKwS2_@@GLIBCXX_3.4.5
GLIBCXX_3.4.12
_ZNSi6ignoreEv@@GLIBCXX_3.4.5
GLIBCXX_3.4.2
_ZNSt11char_traitsIcE2eqERKcS2_@@GLIBCXX_3.4.5
GLIBCXX_3.4.6

This means all the missing parts are there only need to link them properly. Do you have any suggestion how to do it ?

Thank you.

zhangli28 commented 5 months ago

Hi @n01r , @cemitch99 @ax3l ,

Finally, starts running. Maybe the module ananconda3 is conflicting with the python modules in conda_env impactx-cpu. It is sometimes difficutly to understand by doing what the error appear and how it fixed !!

Thank so much to all of you. Zhang