Json details

[mmaelicke]

This PR will turn the details into a JSONB datatype to gain more flexibility here.

Open todos:

• [x] check unittests (some might fail...)
• [x] add a database revision
• [x] add a unittest about value mutability
• [x] check if the API relies on Detail.value being a string somewhere
• [x] check if Entry relies on Detail.value being a string somewhere
• [x] check with @MarcusStrobl if django is using the details table directly somewhere

[mmaelicke]

@AlexDo1 wait with the review until this PR is finished. I will tell you...

[codecov[bot]]

Codecov Report

Merging #125 (a6bc96a) into master (b9fa485) will increase coverage by 0.86%. The diff coverage is 88.63%.

@@            Coverage Diff             @@
##           master     #125      +/-   ##
==========================================
+ Coverage   56.10%   56.96%   +0.86%     
==========================================
  Files          57       58       +1     
  Lines        2588     2626      +38     
==========================================
+ Hits         1452     1496      +44     
+ Misses       1136     1130       -6     

Flag	Coverage Δ
e2e	56.96% <88.63%> (+0.86%)	:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
metacatalog/db/revisions/__init__.py	100.00% <ø> (ø)
metacatalog/db/revisions/rev5.py	60.00% <60.00%> (ø)
metacatalog/api/find.py	52.63% <83.33%> (+2.27%)	:arrow_up:
metacatalog/models/details.py	97.77% <100.00%> (+1.77%)	:arrow_up:
metacatalog/models/entry.py	72.38% <100.00%> (+2.42%)	:arrow_up:

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update b9fa485...a6bc96a. Read the comment docs.

[mmaelicke]

@AlexDo1 can you use this branch to run your Eddy example with the details being composed of a dictionary. I guess most stuff should already work.

Then create the Entry with details and run the Entry.details_dict and entry.details_table(fmt='markdown') on the example. If no errors occur, paste the output here, because I guess it will need some tweaking...

Would be great help because I am doing something else on my local db right now.

[mmaelicke]

@AlexDo1 in addition: I just added the database revision. I did not try it, there might be bugs and typos. Can you run it once you have the dict-like details loaded. I want to see how the details look like after downgrading the revision and then upgrading it again. I guess there is no way to make them match 100%.

[AlexDo1]

Sorry it took some time.

Here is the dict I created:

This is the output of Entry.details_dict:

And this is the output of Entry.details_table(fmt='markdown')

[mmaelicke]

Thanks. Nahh. Not good. The JSON is not parsed. I'll have a look. Thank you very much

[AlexDo1]

@AlexDo1 in addition: I just added the database revision. I did not try it, there might be bugs and typos. Can you run it once you have the dict-like details loaded. I want to see how the details look like after downgrading the revision and then upgrading it again. I guess there is no way to make them match 100%.

Regarding the revision: I ran python -m metacatalog migrate downgrade which resulted in the following error:

An unexpected error occured: Could not parse rfc1738 URL from string 'default' Full error traceback in 'error.log'

error.log:

File "c:\users\alexander\documents\github\metacatalog\metacatalog\command_line.py", line 115, in main args.func(args) File "c:\users\alexander\documents\github\metacatalog\metacatalog\cmd\migration.py", line 18, in migrate downgrade(args) File "c:\users\alexander\documents\github\metacatalog\metacatalog\cmd\migration.py", line 37, in downgrade session = connect(args) File "c:\users\alexander\documents\github\metacatalog\metacatalog\cmd_util.py", line 21, in connect session = connect_database(conn, echo=echo) File "c:\users\alexander\documents\github\metacatalog\metacatalog\api\db.py", line 56, in connect_database session = get_session(*args, kwargs) File "c:\users\alexander\documents\github\metacatalog\metacatalog\db\session.py", line 90, in get_session engine = get_engine(*args, *kwargs) File "c:\users\alexander\documents\github\metacatalog\metacatalog\db\session.py", line 71, in get_engine engine = create_engine(args, kwargs) File "", line 2, in create_engine File "C:\Users\Alexander\anaconda3\envs\mc_develop\lib\site-packages\sqlalchemy-1.4.0b2-py3.8-win-amd64.egg\sqlalchemy\util\deprecations.py", line 298, in warned return fn(*args, **kwargs) File "C:\Users\Alexander\anaconda3\envs\mc_develop\lib\site-packages\sqlalchemy-1.4.0b2-py3.8-win-amd64.egg\sqlalchemy\engine\create.py", line 511, in create_engine u = _url.make_url(url) File "C:\Users\Alexander\anaconda3\envs\mc_develop\lib\site-packages\sqlalchemy-1.4.0b2-py3.8-win-amd64.egg\sqlalchemy\engine\url.py", line 711, in make_url return _parse_rfc1738_args(name_or_url) File "C:\Users\Alexander\anaconda3\envs\mc_develop\lib\site-packages\sqlalchemy-1.4.0b2-py3.8-win-amd64.egg\sqlalchemy\engine\url.py", line 777, in _parse_rfc1738_args raise exc.ArgumentError(

The same error occurs when running python -m metacatalog migrate upgrade