MolSSI / QCFractal

A distributed compute and database platform for quantum chemistry.
https://molssi.github.io/QCFractal/
BSD 3-Clause "New" or "Revised" License
144 stars 47 forks source link

`FractalComputeConfig` inherit from `BaseSettings` instead of `BaseModel` #729

Open hadim opened 1 year ago

hadim commented 1 year ago

The main reason would be to allow parsing env variables instead of providing a config YAML file (more convenient in a container-based env for example).

hadim commented 1 year ago

For now I am using the following patch: https://github.com/hadim/QCFractal/commit/494a78a55b06c411348f4524e3b9773eadc22a75

It works with the below config for example:

export QCF_COMPUTE_BASE_FOLDER=/tmp/qcf_compute
export QCF_COMPUTE_CLUSTER=manager_demo_1
export QCF_COMPUTE_LOGLEVEL=INFO
export QCF_COMPUTE_UPDATE_FREQUENCY=30

export QCF_COMPUTE_SERVER_FRACTAL_URI=http://127.0.0.1:7777
export QCF_COMPUTE_SERVER_USERNAME=compute_user
export QCF_COMPUTE_SERVER_PASSWORD=compute_password
export QCF_COMPUTE_SERVER_VERIFY=false

export QCF_COMPUTE_ENVIRONMENTS_USE_MANAGER_ENVIRONMENT=true
export QCF_COMPUTE_ENVIRONMENTS_CONDA='[]'
export QCF_COMPUTE_ENVIRONMENTS_APPTAINER='[]'

export QCF_COMPUTE_EXECUTORS='{"local": {"type": "local", "queue_tags": ["demo"], "cores_per_worker": 16, "memory_per_worker": 16, "max_workers": 1}}'

The only thing that is not ideal is QCF_COMPUTE_EXECUTORS but I haven't spend too much time trying to make it work with multiple env variables instead of a single JSON string.

hadim commented 1 year ago

I had to add another patch to make it works (and the tests to pass): https://github.com/hadim/QCFractal/commit/ee8311847efd4b198e3d7d811ec30192d760ca98

Also somehow I also need to set QCF_COMPUTE_SERVER or it fails cuz of pydantic error.

export QCF_COMPUTE_BASE_FOLDER=/tmp/qcf_compute
export QCF_COMPUTE_CLUSTER=manager_demo_1
export QCF_COMPUTE_LOGLEVEL=INFO
export QCF_COMPUTE_UPDATE_FREQUENCY=30

export QCF_COMPUTE_SERVER='{}'  # somehow this is needed....
export QCF_COMPUTE_SERVER_FRACTAL_URI=http://127.0.0.1:7777
export QCF_COMPUTE_SERVER_USERNAME=compute_user
export QCF_COMPUTE_SERVER_PASSWORD=compute_password
export QCF_COMPUTE_SERVER_VERIFY=false

export QCF_COMPUTE_ENVIRONMENTS_USE_MANAGER_ENVIRONMENT=true
export QCF_COMPUTE_ENVIRONMENTS_CONDA='[]'
export QCF_COMPUTE_ENVIRONMENTS_APPTAINER='[]'

export QCF_COMPUTE_EXECUTORS='{"local": {"type": "local", "queue_tags": ["demo"], "cores_per_worker": 16, "memory_per_worker": 16, "max_workers": 1}}'

It works well for me.