restructure model definitions

[coldfix]

If implementing #33 (especially then, but also independent from it) it would make sense to think about restructuring the model definitions. I find the current format somewhat too nested and I am not sure about the purpose/distinction of some of the fields:

• how exactly are these related: 'model', 'sequence', 'optic'?
• is there a reason, that one .cpymad.json file holds more than a single model? This makes loading/iterating over models somewhat awkward. I think also the inheritance could be implemented in a more general fashion when sticking to one model per file.
• why are locations either 'resource' or 'repository' if all of them can be loaded locally? I think it would be more straight forward to either (1) remove the location field or (2) allow to define any number of custom locations
• do the fields { .., "parse": "STRENGTHS" } and { ..., "overlay": true } have some meaning to pymad or do you just use them for some application

For my current application a structure like this would be sufficient:

• single model per file, filename defines model name
• the file directly contains the attributes of the model on top level ('extends', 'real', ...)
• single sequence per model, i.e. the attributes of the default sequence are also top level

[Eothred]

Right, many questions, let me try to answer a few at least.

• For model/sequence/optics, the logic is perhaps better understood when using JMad. The model is typically a machine (e.g. the LHC). A machine can have several sequences, typically for different beams etc (lhc beam 1 and beam 2). Most machines have several optics designed for them (different beta*, injection optics, special optics for specialized purposes). Each model can have several sequences, and each sequence can have several optics.
• There is in principle no strict need to have more than one model per file. I think the reason I did it was to e.g. share the optics for lhc beam 1 and beam 2. And with that I broke the logic of not differing from JMad model definitions, so it was probably a bad idea. I was in the past the one developer (and one user?) of cpymad, so I figured it was worth a try to see how it worked out.
• We have our optics repositories on afs shares (andrew file system, not recommended). Sometimes there might be updates to these files, and the user might want to always use them to make sure, or want to check that they give the same result. The repdata folder exactly replicates the folder structure found on afs or elsewhere (see dbdirs attribute), so that you can simply change the first part of the path to call the remote files instead. The files in 'resdata' are not found in the repository folder on afs, so it has to be loaded locally.
• I think parse/overlay are coming from JMad, and not used by CPymad at the moment. E.g. the parse strengths attribute was a way to tell JMad that it should look for knobs in this file (as a way of automatically define knobs in the model). That is, if my memory is correct.

To answer your suggestions:

• I agree to single model per file makes it cleaner.
• I would like to define a name for the model which might contain spaces or other characters, while I would like the filename to not contain any spaces (e.g. 'LHC (LSA)'). Could be easily solved by having an attribute 'name'='LHC (LSA)'.
• For the moment I would like to keep the design of allowing more than one sequence per model. We have model files for e.g. the LHC which when called defines more than one sequence in memory, and I would like cpymad to correctly take advantage of that. The alternative would double the needed memory to load both sequences (and memory is already a significant issue with Mad-X).

[coldfix]

Thanks for all the explanations!

Okay, I see why the pathes are set up in the way they are then. Does the current implementation allow the pymad user to load from AFS if he wants? I don't see code for that anywhere. Maybe I have removed it in some of the model_locator patches. What do you think about subclassing ModelLocator to load any data from the AFS if available. This enables the user to decide whether he wants to use AFS or not by using the correct locator type.

[Eothred]

Hi, yes, it would be good if the model_locator could optionally use any of the paths defined in dbdir as an alternative to the installed files.