GernotMaier commented 1 month ago

Add CORSIKA parameters to the database and change the code to read and digest the values.

Preamble: CORSIKA has many input parameters.

no need to put every single parameter into the database
concentrate on those which likely changes either between productions or are changed by a user
disentangle the values so that only those relevant

For this pull request, the following changes are applied:

addition of parameter schema simtools/schemas/model_parameters/corsika_* (done in #1057)
removal of obsolete parameter file data/parameters/corsika_parameters.yml
addition of reading of corsika parameters from the database in db_handler and model_parameters
(most modifications) addaption of corsika_config to handle the new parameter definitions

This PR requires a new version of the model_parameters; this means you should change the database name in your .env definitions (after merge of https://gitlab.cta-observatory.org/cta-science/simulations/simulation-model/model_parameters/-/merge_requests/43 and release of a new model parameter version).

GernotMaier commented 1 month ago

Unit and integration tests are failing at this point, as the model parameter DB has not been update (but which needs code from this PR to be updated!).

Locally, all tests pass.

Do run tests locally, do:

git checkout corsika-parameters-in-db
cd database_scripts
./setup_local_db.sh
./upload_from_model_repository_to_local_db.sh

Change your .env file to:

# Environmental variables
SIMTOOLS_DB_API_PORT=27017 #Port on the MongoDB server
SIMTOOLS_DB_SERVER='simtools-mongodb'
SIMTOOLS_DB_API_USER='api' # username for MongoDB
SIMTOOLS_DB_API_PW='password' # Password for MongoDB
SIMTOOLS_DB_API_AUTHENTICATION_DATABASE='admin'
SIMTOOLS_DB_SIMULATION_MODEL='Staging-CTA-Simulation-Model-v0-3-0'
SIMTOOLS_SIMTEL_PATH='/workdir/sim_telarray'

and run a dev container connected to this Network:

podman run --rm -it -v "$(pwd)/:/workdir/external" --network simtools-mongo-network ghcr.io/gammasim/simtools-dev:latest bash -c "source /workdir/env/bin/activate && cd /workdir/external/simtools && pip install -e . && bash

Test should pass with this configuration.