search

atcollab / at

Accelerator Toolbox
Apache License 2.0
48 stars 31 forks source link

Load MAD-X files #770

Closed lfarv closed 2 months ago

lfarv commented 4 months ago

This PR brings the possibility of parsing MAD-X files to define AT structures. It is expected to support the full MAD-X language as defined in the MAD-X Reference Guide. In particular it supports:

However, as I am not a MAD-X expert, there may still remain a few problems, which can be easily fixed. I successfully parsed the Xsuite test data from CERN, but I wait for more tests!

Usage:

Short:

ring = at.load_madx(file1, file2,…, use="sequence_name")

MAD-X data is usually spread in different files, so you can give them all, in any order, to the load function. Alternatively, the CALL command allows nested file parsing.

Long:

The MAD-X parser allows more detailed interaction:

parser = at.MadxParser()
parser.parse_files("file1") # parse a 1st file
parser.parse_files("file2") # parse a 2nd file
parser["var1"] # display the variable “var1”
parser["qf1"]  # display the “qf1” element
parser["var2"] = 42.0 # add a new variable
ring = parser.lattice(use="element_name")  # generate an AT Lattice

Known limitations:

Please test and send you feedback!

lfarv commented 4 months ago

This branch is based on the json_files branch, which explains most of the modified files compared to master. #766 should be merged 1st before this one.

simoneliuzzo commented 3 months ago

Dear @lfarv

I tried an FCC lattice and the conversion gives this error:

TypeError: Line 75 'kdecdl := -9.0e2-8.0e3*cs_comp^2.0', unsupported operand type(s) for ^: 'float' and 'float'

best regards Simone

simoneliuzzo commented 3 months ago

here the file I tried to convert: '/machfs/liuzzo/FCC/92b4/LATTICE/fcc_92b4.seq'

simoneliuzzo commented 3 months ago

with this command:

r_madx = at.load_madx('/machfs/liuzzo/FCC/92b4/LATTICE/fcc_92b4.seq', use='RING_FULL')

simoneliuzzo commented 3 months ago

Dear @lfarv,

I tryed also an EBS, simpler, test file:

r_madx = at.load_madx('/machfs/liuzzo/EBS/S28C/LATTICE/madx/LatticeWithCorrectorsAndMonitors/S28CINJ_ARC_BPM.madx', use='ARCA', verbose=True)

This time the error is: End /machfs/liuzzo/EBS/S28C/LATTICE/madx/LatticeWithCorrectorsAndMonitors/S28CINJ_ARC_BPM.madx

Missing definitions: {'', 's', 'rf_on', 'low_emit_ring', 'twiss', 'oct_on', 'arca', 'low_emit_ring_inj', 'arca_inj', 'of1b', 'arc_inj', 'arcb_inj', 'stop', 'oj1b', 'arc2'}

KeyError: ARCA

'ARCA' is not defined

But those variables are defined in the file and the MADX file runs correctly.

lfarv commented 3 months ago

@simoneliuzzo: This new version solves the 2 cases you encountered:

  1. For the FCC lattice, it was just a typo, easily solved,
  2. For 2nd example, the solution is more tricky: there are 2 undefined variables in the data file: rf_on and oct_on, It seems the MAD-X silently assigns 0 to undefined values. However, this is not documented anywhere in the documentation, and I am reluctant to implement an undocumented feature since it may change at any time… But I also noticed that many MAD-X data files contain a line "none = 0;". The meaning of this line is also not documented, but I understand it as: "Set the value of undefined variables to the value of none". But I may be wrong…

So here is the solution I propose: I add a strict bool keyword to load_madx and to the MadxParser constructor:

So your example succeeds if you add strict=False to the arguments of load_madx.

This can be modified if you find a better solution (like using strict=False as the default).

Side remark: in your files, there is a line

CA05 : RFCavity, L=0.0000000001,VOLT=RF_ON*2.1666666667 ;, freq=352371967.6749427319 ;

I guess that the semicolon in the middle of the line is a mistake. In load_madx, the second part of the line is ignored (and reported as such): no command before the comma. I don't know what MAD-X does in this case.

simoneliuzzo commented 3 months ago

Dear Laurent,

Here the phase space plot comparison for FCC.

There was some difference compared to the AT, but it was due to the different NumIntSteps.

The comparison is good.

Screenshot 2024-06-10 at 11 53 28

simoneliuzzo commented 3 months ago

Dear Laurent,

in 6D, there is discrepancy. I simply set the radiation and rf on with

r.enable_6d(dipole_pass='BndMPoleSymplectic4RadPass',
                     quadrupole_pass='StrMPoleSymplectic4RadPass',
                     sexupole_pass='StrMPoleSymplectic4RadPass')

        r.harmonic_number = harm
        r.set_cavity(Voltage=volt)
        r.set_rf_frequency()

        r.set_cavity_phase()

The two trackings AT (Blue), MADX->AT(orange) are different.

Screenshot 2024-06-10 at 12 03 09

In 4D they are identical, so it must be cavity settings: AT: RFCavity('RFC', 1e-12, 5000000000.0, 446421827.2262573, 135000, 182500000000.0, TimeLag=0.049486207651028624) MADX->AT: RFCavity('rfc', 1e-12, 5000000000.0, 446421752.7269621, 135000, 1000000000.0, TimeLag=4.2432823948240586e-11, source='rfcavity')

So the energy was not set ok, it could be an other optional input of load_madx?

With corrected energy: Screenshot 2024-06-10 at 12 13 16

Better, but there is still some issue: RFCavity('RFC', 1e-12, 5000000000.0, 446421827.2262573, 135000, 182500000000.0, TimeLag=0.049486207651028624) RFCavity('rfc', 1e-12, 5000000000.0, 446421752.7269621, 135000, 182500000000.0, TimeLag=4.2432823948240586e-11, source='rfcavity')

now time-lag is not appropriately set, despite the set_cavity_phase command. Here I do not know anymore how to fix it. Apparently, the same cavity settings commands, do not set the cavity to identical values.

lfarv commented 3 months ago

@simoneliuzzo : A few explanations on the behaviour of the translation;

Anyway I found other problems, so it's not ready for merging now, but you are welcome to add new tests, thanks !

simoneliuzzo commented 3 months ago

@simoneliuzzo : A few explanations on the behaviour of the translation;

* `NumIntSteps`: I did not find any equivalent in the properties of MAD-X quadrupoles. So `NumIntSteps` is set the AT's default: 10. I could add a keyword in `load_madx` to set it globally.

* energy: the energy (and particle) are taken from the MAD-X `BEAM` object. Since there is not `BEAM` in your data, it defaults to MAD-X default beam: positrons at 1 GeV. But you can force both with the `energy` and `particle` keywords in `load_madx`. It's already available.

* radiation state: it's also governed by the RADIATE property of the MAD-X `BEAM` object, no radiation by default.

* `TimeLag`: MAD-X uses phase lag. The conversion to time lag is not done yet, and at the moment `TimeLag` is set to 0. The 4e-11 value that you get from `set_cavity_phase` looks strange: it is as if there was almost no radiation. Can you check the energy loss?

Anyway I found other problems, so it's not ready for merging now, but you are welcome to add new tests, thanks !

Dear @lfarv,

thank you for this explanation. May be this information could be included in the help of load_madx? May be it is already part of it.

For the lattice I am using (182.5GeV, FCC@ttbar):

p.emittances array([ 2.21211903e-09, -3.00828910e-36, 2.11662136e-06]) p.U0 8808215337.3667

simoneliuzzo commented 3 months ago

Dear @lfarv

let me know when I should test again.

best regards Simone

lfarv commented 3 months ago

@simoneliuzzo : You are welcome to do more tests. I looked at your FCC example: I don't see anything wrong in the translation. And after calling set_cavity_phase(), I find a value of .0494857943661414 very similar to your reference value. I suspect a problem in your test suite. It look that the start coordinates are different.

However since that, I improved the translation of RF cavities: according to the MAD reference guide, if the frequency is not specified, it is set to the nominal frequency, using the harmon attribute. This makes the call to set_rf_frequency() unnecessary.

I also added a mad8 converter (load_mad8, Mad8Parser). I tried on your file /machfs/liuzzo/EBS/S28C/LATTICE/mad8/LatticeWithCorrectorsAndMonitors/S28CINJ_ARC_BPM.mad with some success, though I noticed 2 things:

  1. most commands end up with a semicolon. There is no detrimental effect, but it generates on each line an empty command (from the semicolon to the end of line), which very slightly slows down the process
  2. in RF cavities, the harmonic number is given as harm instead of harmon, which results in the frequency being set to zero
simoneliuzzo commented 3 months ago

Dear @lfarv,

I have updated the repository. The 6D lattice is still not ok for me, the time lag setting is not correct. I am using a fresh environment with this branch in it only. I will look more in detail, even if this was not supposed to be the objective of these tests. I always end up with issues with cavity/radiation/phase settings of some sort. I hope other users do not get these issues as often as me.

The commands I use for cavity setting are: 

r.enable_6d(dipole_pass='BndMPoleSymplectic4RadPass',
                     quadrupole_pass='StrMPoleSymplectic4RadPass',
                     sexupole_pass='StrMPoleSymplectic4RadPass')
r.harmonic_number = harm
r.set_cavity(Voltage=volt)
r.set_rf_frequency()
r.set_cavity_phase()

I run them to get a 6D lattice all the time to make sure the cavity is "correctly set", but apparently they do not set my cavity correctly all the time. Would you have a better "guaranteed" solution, independent of the input lattice details?

I tested the mad8 conversion of the same lattice: 

r_mad8 = at.load_mad8('/machfs/liuzzo/FCC/92b4/LATTICE/FCC_92b4_save.md8', use='RING_FULL', strict=False)
r_mad8_f = at.load_mad8('/machfs/liuzzo/FCC/FCC_92b4/FCC_92b4.md8', use='RING_FULL', strict=False)

but it returns an error:

Processing /machfs/liuzzo/FCC/92b4/LATTICE/FCC_92b4_save.md8
End /machfs/liuzzo/FCC/92b4/LATTICE/FCC_92b4_save.md8
Traceback (most recent call last):
  File "/machfs/liuzzo/accelerator_toolbox_tests/python/load_madx/main.py", line 79, in <module>
    r_mad8 = at.load_mad8('/machfs/liuzzo/FCC/92b4/LATTICE/FCC_92b4_save.md8', use='RING_FULL', strict=False)
  File "/machfs/liuzzo/accelerator_toolbox_tests/python/load_madx/load_madx_venv/lib/python3.9/site-packages/at/load/mad8.py", line 143, in load_mad8
    parser.parse_files(*absfiles)
  File "/machfs/liuzzo/accelerator_toolbox_tests/python/load_madx/load_madx_venv/lib/python3.9/site-packages/at/load/file_input.py", line 517, in parse_files
    self.parse_lines(f, final=final and (nf == last))
  File "/machfs/liuzzo/accelerator_toolbox_tests/python/load_madx/load_madx_venv/lib/python3.9/site-packages/at/load/file_input.py", line 486, in parse_lines
    self._finalise(final=final)
  File "/machfs/liuzzo/accelerator_toolbox_tests/python/load_madx/load_madx_venv/lib/python3.9/site-packages/at/load/madx.py", line 640, in _finalise
    super()._finalise(final=final)
  File "/machfs/liuzzo/accelerator_toolbox_tests/python/load_madx/load_madx_venv/lib/python3.9/site-packages/at/load/file_input.py", line 572, in _finalise
    self._decode(*args)
  File "/machfs/liuzzo/accelerator_toolbox_tests/python/load_madx/load_madx_venv/lib/python3.9/site-packages/at/load/file_input.py", line 560, in _decode
    super()._decode(label, cmdname, *argnames)
  File "/machfs/liuzzo/accelerator_toolbox_tests/python/load_madx/load_madx_venv/lib/python3.9/site-packages/at/load/file_input.py", line 363, in _decode
    label, result = self._assign(label, left, right[0])
  File "/machfs/liuzzo/accelerator_toolbox_tests/python/load_madx/load_madx_venv/lib/python3.9/site-packages/at/load/madx.py", line 635, in _assign
    return label, _Line(self.evaluate(val), name=label)
  File "/machfs/liuzzo/accelerator_toolbox_tests/python/load_madx/load_madx_venv/lib/python3.9/site-packages/at/load/mad8.py", line 107, in evaluate
    return super().evaluate(expr)
  File "/machfs/liuzzo/accelerator_toolbox_tests/python/load_madx/load_madx_venv/lib/python3.9/site-packages/at/load/file_input.py", line 269, in evaluate
    return eval(expr, self.env, self)
  File "<string>", line 1, in <module>
TypeError: unsupported operand type(s) for *: 'int' and 'sextupole'

thank you

best regards Simone

lfarv commented 3 months ago

@simoneliuzzo: I take note of your problem with phase settings. It looks unrelated to the parsing of MAD files, so I propose that we look at that outside of this PR. If you send me the details of your sequence, I'll look at that.

For MAD8 parsing, the new version succeeds with your 1st example ('/machfs/liuzzo/FCC/92b4/LATTICE/FCC_92b4_save.md8'). Can you check again?

Note: this file works without specifying strict=False. You should always try 1st like that, because if there is really a missing element definition, strict=False will replace it with a zero, which is probably stupid and gives crazy error messages. Only switch to strict=False if you see that there are indeed missing variables which can be replaced by zero.

But I still have problems with the 2nd file ('/machfs/liuzzo/FCC/FCC_92b4/FCC_92b4.md8'):

simoneliuzzo commented 3 months ago

Dear @lfarv,

with the help of @swhite2401 the issue of 6D is solved.

here the figure of the same lattice in AT, loaded from MADX and loaded from MAD8 (save). They are identical!

Screenshot 2024-06-27 at 15 16 43 Screenshot 2024-06-27 at 15 16 48

lfarv commented 3 months ago

@simoneliuzzo : very nice !

elafmusa commented 2 months ago

Dear @lfarv,

I have recently converted a sequence file (attached) for fccee lattice using at.load_madx as follows:

import at ring = at.load_madx("fccee_HFD_79_line.seq", use="fccee_p_ring", energy = 45.6e9) (sequence file below) https://syncandshare.desy.de/index.php/s/W6W8aAFt32JrzNJ

Then in pyAT, I turned the radiation and cavity on for the nominal lattice using the following commands:

ring.energy = 45.6e9 ring.enable_6d(dipole_pass='BndMPoleSymplectic4RadPass', quadrupole_pass='StrMPoleSymplectic4RadPass', sexupole_pass='StrMPoleSymplectic4RadPass') ring.harmonic_number = 135000 ring.set_cavity(Voltage=50e6) ring.set_rf_frequency() ring.tapering()

I obtained the voltage and harmonic number values from the sequence file. After setting these parameters, the cavity parameters are as follows for each cavity:

RFCavity: FamName: RFC Length: 1e-12 PassMethod: RFCavityPass Energy: 45600000000.0 Frequency: 446421827.23974824 HarmNumber: 135000 TimeLag: 0.0 Voltage: 12500000.0 madtype: rfcavity

I noticed that the time lag and cavity voltage deviate from the nominal values (0.4 time lag and 50 Mev voltage).

Could you please assist me with how to set the RF parameters properly?

Thank you in advance.

TeresiaOlsson commented 2 months ago

Hi @elafmusa

Do you have more than one cavity in your lattice? Then I'm guessing that's why the cavity voltage is different. I think the number in set_cavity should be the total RF voltage and not per cavity. Then the code will divide it by the number of cavities you have.

For the phase, I use ring.set_cavity_phase(). It should calculate which timelag you need from the energy loss per turn and set it.

elafmusa commented 2 months ago

Hi @TeresiaOlsson,

Thank you for your response, yes there is 4 cavities in this lattice. Following your comment i have adjusted the voltage and used ring.set_rf_frequency() to adjust the time lag, as following:

ring.energy = 45.6e9 ring.enable_6d(dipole_pass='BndMPoleSymplectic4RadPass', quadrupole_pass='StrMPoleSymplectic4RadPass', sexupole_pass='StrMPoleSymplectic4RadPass') ring.harmonic_number = 135000 ring.set_cavity(Voltage=50e6 * len(rf_ind))

for i in rf_ind:

ring[i].TimeLag = 0.4

ring.set_cavity_phase() ring.set_rf_frequency() ring.tapering()

The Cavity parameters are now ( for each cavity):

RFCavity: FamName: RFC Length: 1e-12 PassMethod: RFCavityPass Energy: 45600000000.0 Frequency: 446421827.23974824 HarmNumber: 135000 TimeLag: 0.01851831590482653 Voltage: 50000000.0 madtype: rfcavity

Can i use the commented lines above to adjust the nominal time lag for each cavity instead of using ring.set_cavity_phase() ?

for i in rf_ind:

ring[i].TimeLag = 0.4

TeresiaOlsson commented 2 months ago

Yes, it should work but if you have another value that what set_cavity_phase gives I would recommend to double-check the convention for the TimeLag so you are setting it to the phase you really want. In my experience the TimeLag has a very strange convention compared to the phase in other codes.