search

entropy-lab / entropy

BSD 3-Clause "New" or "Revised" License
30 stars 13 forks source link

Relational DB support for ParamStore #326

Closed urig closed 2 years ago

urig commented 2 years ago

This PR introduces a significant improvement to the Entropy ParamStore. ParamStore can now be configured to use a relational database to store its contents (i.e. Params, Tags and Commits). Using a relational database provides faster response times compared to the existing option of using a JSON file for storage. This holds particularly true around writing commits to disk.

To use the ParamStore with a relational database, use the new url argument when instantiating the ParamStore class. The value for url should be a URL that points to the location of the database.

For example, to connect to a SQLite database file (requires no further installation):

store = ParamStore(url="sqlite:///my_project/.entropy/param_store.db")

To connect to a PostgreSQL database server:

store = ParamStore(url="postgresql://username:password@server/database")

Optionally, you can place the URL in a configuration file called settings.toml inside an Entropy Project's .entropy folder

The PR includes the following breaking changes:

  1. The ParamStore abstract base class has been removed.

  2. The InProcessParamStore class has been renamed ParamStore and moved a new package called entropylab.pipeline.params.
    As a result, any Entropy Projects that contain an existing ParamStore JSON file will need to be upgraded. To upgrade such a project, make a backup by copying the project folder to a safe location. In a Terminal or Command Prompt window run the Entropy CLI command: entropy upgrade <path to entropy project folder>.

  3. The ParmStore method list_commits() now matches exactly by given label. Previously it would match when the label argument was contained in a commit label.

github-actions[bot] commented 2 years ago

Unit Test Results

305 tests   299 :heavy_check_mark:  56s :stopwatch:     1 suites      6 :zzz:     1 files        0 :x:

Results for commit 495ad204.

:recycle: This comment has been updated with latest results.