Configuration files setup

[beteva]

Find a way to share the mockup configuration files between mxcubecore and mxcubeweb repositories,

Mockup configuration files are needed both for testing the hardware objects in the mxcubecore and the implementation in mxcubeweb. The files are now in both repositories and very often differ from each other. Even worse - they are sometimes inconsistent (e.g. udiff_omega.xml in the mxcubeweb erepository does not correspond to the MotorMockup class in mxcubecore)

[marcus-oscarsson]

I think we should cleanup the configuration file directory in mxcubecore.

https://github.com/mxcube/mxcubecore/tree/develop/mxcubecore/configuration

We should also in my opinion move it out of the mxcubecore module as its not python code or a resource used directly by the code. The configuration files could become their own repository but its perhaps more practical to have them at the root of the mxcubecore repository.

Like this we can download these files from the UI's and use them for the tests, its a bit more cumbersome but it removes the duplication that @beteva mentions above.

Its perhaps a good occasion to look at this now as we are also dealing with #931 .

[rhfogh]

How do you see this working across the Web and Qt implementations? Would you (still) be able to share mock configuration for hardware objects across the two?

[marcus-oscarsson]

Oh yes, actually I see this also in the perspective of #931. Its probably sensible to first cleanup https://github.com/mxcube/mxcubecore/tree/develop/mxcubecore/configuration before also converting those files. It was also the reason why I thought it would be good to tackle #902.

To share the exact same configuration file between web and qt versions would be very nice and I think the XML to YAML is a good step in that direction but it will of course require more work than only that.

Looking at the configuration files with @beteva we realized that keeping them in the mxcubecore module is in our opinion not the most sensible thing to do.

• Its not python code or a resource directly used by the code so it should probably not be in the source code.

• Its also a bit odd to download part of the internals of mxcubecore to get the configuration files

So we would be in favor of moving the to the root of the repository like this they are at least not part of the python module, I guess it would even be more elegant if they are kept in their own repository.

[marcus-oscarsson]

As a side note our ambition is to start to try out the YAML configuration file system as soon a s possible so that we can fix any remaining bugs. We would of-course very much appreciate if someone else did the same :)

[elmjag]

I think the config files used for mxcubecore tests should be kept separate from the ones in mxcubeweb. They are used for different purposes.

The ones in mxcubeweb make it possible for everyone to run a full instance of the MXCuBE web. But we also need a set of config files to run the unit tests of mxcubecore code.

I suggest following:

1) In mxcubeweb rename test/HardwareObjectsMockup.xml to MockupBeamline. This will make it a bit more clear what is the purpose of these config files.

2) In mxcubecore, move all config files used by the tests inside the test/data folder. In principle these files does not necessary need to define a complete beamline, depending on need of specific tests.

I don't think it make sense to try unify contests of files in 1 and 2. For 1 the focus should be configure a fully featured beamline with mockup HWOs. For 2 the focus is test coverage of mxcubecore code.

[fabcor-maxiv]

MockupBeamline

I have not really followed this topic, but at least for me it feels like this could be named DemoBeamline in order to not get confused with "mockups" that are meant to be used for automated testing (pytest, unit tests, and so on).

[fabcor-maxiv]

If I understood correctly, Rasmus mentioned it might be possible to split configuration over multiple directories.

So if I take the example of the configuration for a "demo beamline" does that mean it would possible to have the mxcubecore-specific parts of this configuration in mxcubecore and the web- (respectively qt-) specific parts of this config in mxcubeweb (mxcubeqt)? Would that make sense? Would that be helpful in some way? Would that make things more spread out all over the place and more confusing?

[rhfogh]

@fabcor-maxiv Yes you can do that, by passing more than one directory in the -r option to mxcubeweb-server. The directories will be checked in the order they are given, and you get the first result found. Just to show an example, my standard run command begins with:

mxcubeweb-server -r \~/pycharm/mxcubeweb/mxcubeweb/test/HardwareObjectsMockup.xml/gphl:~/pycharm/mxcubeweb/mxcubeweb/test/HardwareObjectsMockup.xml/

so that the ./gphl subdirectory contents override the general configuration files.

[marcus-oscarsson]

What we perhaps could do is to use the demo directory as default for the tests as well and add ("prepend") a second configuration directory when one needs override the configuration for a specific hardware object.

I'm not sure it makes things less confusing, but like this we at least keep the duplication to a minimum.