lucianopaz / compress_pickle

Standard python pickle, thinly wrapped with standard compression libraries
MIT License
42 stars 11 forks source link

Add JSON support #40

Closed eode closed 2 years ago

eode commented 2 years ago

Pretty much in the title. This adds JSON as an option for serialization/deserialization.

Possibly unrelated, but noteworthy: If you're interested, I have a few minor text output changes I can either merge with this PR or do in a separate PR, like:

codecov[bot] commented 2 years ago

Codecov Report

Merging #40 (8c3493b) into master (3a1ecd0) will not change coverage. The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff            @@
##            master       #40   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           22        23    +1     
  Lines          435       455   +20     
=========================================
+ Hits           435       455   +20     
Impacted Files Coverage Δ
compress_pickle/picklers/__init__.py 100.00% <100.00%> (ø)
compress_pickle/picklers/json.py 100.00% <100.00%> (ø)
eode commented 2 years ago

I added a test that includes all of the basic JSON data types.

lucianopaz commented 2 years ago

This looks really great @eode!! Thanks so much for opening a PR! My only request is that you also add the json pickler here.

Your code seems to pass the CI but a newer version of mypy broke things. I'll fix that so that you can rebase and get your PR merged.

lucianopaz commented 2 years ago

About the typo fixes, they are more than welcome.

lucianopaz commented 2 years ago

@eode, I fixed the mypy issue. Feel free to rebase this PR and I'll merge it in

eode commented 2 years ago

Sorry for the awkward merge rather than a rebase. I pulled from my repo github page, which was irreversible. However, we can rebase upon closing this PR.

This looks really great @eode!! Thanks so much for opening a PR! My only request is that you also add the json pickler here.

Well, @lucianopaz -- thanks for having such a great project! I like your code.

Regarding the changes in Update fixtures to support JSON:

JSON doesn't support binary (bytes) data, but does support str data. That being the case, I've changed the return type of tests.fixtures.random_message() to str instead of bytes. This allows the same tests to be performed as were previously done, just with a different (JSON-compatible) data type.

Unnecessary detail:

Known Issues:

eode commented 2 years ago

To address the Known Issues area above requires a different notion of extensions than is currently coded for the compress_pickle package. That is, it's not a trivial change. Since this PR is otherwise acceptable (and is good progress), my vote is to go ahead an merge this PR if it looks good to you, and I'll do a separate PR for inferences that interpret extensions for both compression and for pickle/serialization formats.

The result of that PR would be:

However, as long as a user specifies that they want JSON in the call to dump/load, it works fine.