Storing data in pickle fields is a security issue. We should use JSON with a datetime-aware encoder instead. To do this, we need to work with the constraints of SQLite (where we can't just add a column easily). We also need a script step added in updates, and a test fixture of a project directory (created with bw2io.backup_project_directory where we can apply the migration.
Data first needs to be loaded from the pickled columns and tested to make sure it can be serialized to JSON (i.e. no functions or executable code, just basic data types). If this condition is violated an error should be raised.
Storing data in pickle fields is a security issue. We should use JSON with a datetime-aware encoder instead. To do this, we need to work with the constraints of SQLite (where we can't just add a column easily). We also need a script step added in
updates, and a test fixture of a project directory (created withbw2io.backup_project_directorywhere we can apply the migration.Data first needs to be loaded from the pickled columns and tested to make sure it can be serialized to JSON (i.e. no functions or executable code, just basic data types). If this condition is violated an error should be raised.
I think all the pickle fields are in https://github.com/brightway-lca/brightway2-data/blob/main/bw2data/backends/schema.py, but this needs to be checked.