Interpretation of lattice versions

[tomerten]

I am currently looking into versioned data storage, inspired by Felix, for more complicated data (epics databases, meta data, ...). One possible issue that I see is the following: Do you consider lattices where a single quad setting is changed as different versions of a lattice? If yes, that will lead to a large set of lattice files (in the order of millions for what I am planning). So how do you define the "version" of a lattice?

[felix-andreas]

Just to make sure we don't misunderstand each other: The "version" key introduced in #51 is used to specify the LatticeJSON version, not the version of the Lattice.

To store multiple versions of a Lattice with slightly changed quadrupole settings, you could use a "settings" key under the "tool" key (#55). The default quad values would then be overridden by the specified settings. But this would be considered a "plugin" and not part of LatticeJSON "core".

For a FODO lattice with the 3 additional Quadrupole Settings "highQ1", "highQ2" and "strong" this could look like:

{
  "version": "2.0",
  "title": "FODO Lattice",
  "info": "This is the simplest possible strong focusing lattice.",
  "root": "RING",
  "elements": {
    "D1": ["Drift", {"length": 0.55}],
    "Q1": ["Quadrupole", {"length": 0.2, "k1": 1.2}],
    "Q2": ["Quadrupole", {"length": 0.4, "k1": -1.2}],
    "B1": ["Dipole", {"length": 1.5, "angle": 0.392701, "e1": 0.1963505, "e2": 0.1963505}]
  },
  "lattices": {
    "CELL": ["Q1", "D1", "B1", "D1", "Q2", "D1", "B1", "D1", "Q1"],
    "RING": ["CELL", "CELL", "CELL", "CELL", "CELL", "CELL", "CELL", "CELL"]
  },
  "tool": {
    "settings": {
      "highQ1": {
         "Q1": {"k1": 1.4}
      },
      "highQ2": {
         "Q2": {"k1": -1.4}
      },
      "strong": {
         "Q1": {"k1": 1.4},
         "Q2": {"k1": -1.4}
      }
    }
  }
}

You could then implement an option to load this lattice:

lattice = latticejson.load("/path/to/lattice", plugin={"settings": {"use": "highQ1"}})

[TMsangohan]

@andreasfelix

Isn't that repeating a bit what I had with the power supplies? Wouldn't it then make more sense to use an external settings json with its own schema (maybe even also have set range checking directly in the settings json).

The thing is that in the digital twin or in commissioning, I will always need an intermediate step from lattice json to a settings json, in my new prototype digital twin I generate an epics database substitution file that will load data from a config json (still designing it) and a settings file for the magnets, epics converts it into engineering units in this version and loads it into the epics pvs.

In summary, a lattice json should be somehow "immutable" (with version etc...) and a settings file specifies the specific settings (with own metadata, version, creator, desc, ...). In principle the same settings files can be applied to different lattices, for example when a magnet is displaced a little bit for whatever reason, it strength setting does not necessarily need to change. Important to keep in mind is that we need an easy way to link lattice, settings and sim output (eg twiss) in such a way that it is easy to find, use and reproduce/compare simulations. Sorry for pushing but I need things up and running quite soon now.

Tom