XanaduAI / MrMustard

A differentiable bridge between phase space and Fock space
https://mrmustard.readthedocs.io/
Apache License 2.0
78 stars 27 forks source link

Serialization #451

Closed timmysilv closed 3 months ago

timmysilv commented 3 months ago

Context: For execution in certain environments, it helps to be able to de/serialize objects

Description of the Change:

Implementation details

Benefits: Useful for certain execution environments

Possible Drawbacks:

Related GitHub Issues: N/A

codecov[bot] commented 3 months ago

Codecov Report

Attention: Patch coverage is 94.24460% with 8 lines in your changes missing coverage. Please review.

Project coverage is 89.21%. Comparing base (2df4c6d) to head (ca0398b). Report is 1 commits behind head on develop.

Files Patch % Lines
mrmustard/lab_dev/circuit_components.py 84.37% 5 Missing :warning:
mrmustard/physics/representations.py 82.35% 3 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## develop #451 +/- ## =========================================== + Coverage 89.12% 89.21% +0.09% =========================================== Files 101 102 +1 Lines 7059 7193 +134 =========================================== + Hits 6291 6417 +126 - Misses 768 776 +8 ``` | [Files](https://app.codecov.io/gh/XanaduAI/MrMustard/pull/451?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=XanaduAI) | Coverage Ξ” | | |---|---|---| | [...ustard/lab\_dev/circuit\_components\_utils/b\_to\_ps.py](https://app.codecov.io/gh/XanaduAI/MrMustard/pull/451?src=pr&el=tree&filepath=mrmustard%2Flab_dev%2Fcircuit_components_utils%2Fb_to_ps.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=XanaduAI#diff-bXJtdXN0YXJkL2xhYl9kZXYvY2lyY3VpdF9jb21wb25lbnRzX3V0aWxzL2JfdG9fcHMucHk=) | `100.00% <100.00%> (ΓΈ)` | | | [...mustard/lab\_dev/circuit\_components\_utils/b\_to\_q.py](https://app.codecov.io/gh/XanaduAI/MrMustard/pull/451?src=pr&el=tree&filepath=mrmustard%2Flab_dev%2Fcircuit_components_utils%2Fb_to_q.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=XanaduAI#diff-bXJtdXN0YXJkL2xhYl9kZXYvY2lyY3VpdF9jb21wb25lbnRzX3V0aWxzL2JfdG9fcS5weQ==) | `100.00% <100.00%> (ΓΈ)` | | | [mrmustard/lab\_dev/circuits.py](https://app.codecov.io/gh/XanaduAI/MrMustard/pull/451?src=pr&el=tree&filepath=mrmustard%2Flab_dev%2Fcircuits.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=XanaduAI#diff-bXJtdXN0YXJkL2xhYl9kZXYvY2lyY3VpdHMucHk=) | `93.02% <100.00%> (+0.76%)` | :arrow_up: | | [mrmustard/lab\_dev/transformations/base.py](https://app.codecov.io/gh/XanaduAI/MrMustard/pull/451?src=pr&el=tree&filepath=mrmustard%2Flab_dev%2Ftransformations%2Fbase.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=XanaduAI#diff-bXJtdXN0YXJkL2xhYl9kZXYvdHJhbnNmb3JtYXRpb25zL2Jhc2UucHk=) | `97.19% <ΓΈ> (ΓΈ)` | | | [mrmustard/math/parameter\_set.py](https://app.codecov.io/gh/XanaduAI/MrMustard/pull/451?src=pr&el=tree&filepath=mrmustard%2Fmath%2Fparameter_set.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=XanaduAI#diff-bXJtdXN0YXJkL21hdGgvcGFyYW1ldGVyX3NldC5weQ==) | `97.36% <100.00%> (-0.07%)` | :arrow_down: | | [mrmustard/utils/serialize.py](https://app.codecov.io/gh/XanaduAI/MrMustard/pull/451?src=pr&el=tree&filepath=mrmustard%2Futils%2Fserialize.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=XanaduAI#diff-bXJtdXN0YXJkL3V0aWxzL3NlcmlhbGl6ZS5weQ==) | `100.00% <100.00%> (ΓΈ)` | | | [mrmustard/utils/settings.py](https://app.codecov.io/gh/XanaduAI/MrMustard/pull/451?src=pr&el=tree&filepath=mrmustard%2Futils%2Fsettings.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=XanaduAI#diff-bXJtdXN0YXJkL3V0aWxzL3NldHRpbmdzLnB5) | `100.00% <100.00%> (ΓΈ)` | | | [mrmustard/physics/representations.py](https://app.codecov.io/gh/XanaduAI/MrMustard/pull/451?src=pr&el=tree&filepath=mrmustard%2Fphysics%2Frepresentations.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=XanaduAI#diff-bXJtdXN0YXJkL3BoeXNpY3MvcmVwcmVzZW50YXRpb25zLnB5) | `98.19% <82.35%> (-1.32%)` | :arrow_down: | | [mrmustard/lab\_dev/circuit\_components.py](https://app.codecov.io/gh/XanaduAI/MrMustard/pull/451?src=pr&el=tree&filepath=mrmustard%2Flab_dev%2Fcircuit_components.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=XanaduAI#diff-bXJtdXN0YXJkL2xhYl9kZXYvY2lyY3VpdF9jb21wb25lbnRzLnB5) | `98.36% <84.37%> (-1.64%)` | :arrow_down: | ------ [Continue to review full report in Codecov by Sentry](https://app.codecov.io/gh/XanaduAI/MrMustard/pull/451?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=XanaduAI). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=XanaduAI) > `Ξ” = absolute (impact)`, `ΓΈ = not affected`, `? = missing data` > Powered by [Codecov](https://app.codecov.io/gh/XanaduAI/MrMustard/pull/451?dropdown=coverage&src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=XanaduAI). Last update [2df4c6d...ca0398b](https://app.codecov.io/gh/XanaduAI/MrMustard/pull/451?dropdown=coverage&src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=XanaduAI). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=XanaduAI).
apchytr commented 3 months ago

Might have missed it but does the serialization keep track of a version? Could be useful even if we arent concerned about backward compatibility

timmysilv commented 3 months ago

it doesn't. can definitely add it, though I'm not sure what it'll add. I'd be curious to know more on how people intend on using this before adding more features. I kept it pretty barebones because I honestly don't know what we want from this tool, and kinda need more info before adding more advanced features to the module. that said, I can add version when saving and pop it when loading if we want that.

apchytr commented 3 months ago

it doesn't. can definitely add it, though I'm not sure what it'll add. I'd be curious to know more on how people intend on using this before adding more features. I kept it pretty barebones because I honestly don't know what we want from this tool, and kinda need more info before adding more advanced features to the module. that said, I can add version when saving and pop it when loading if we want that.

I agree we should keep it barebones for now but I do think theres some benefit including it. At the very least it would make it clear what version the saved files are associated with if breaking changes happen. That is to say only if we don't rely on the static 0.7.3 version. I think either it could be the latest commit hash from develop (temporary until we have actual releases) or we could start updating the pyproject.toml with an alpha (e.g. 0.7.3a1). What do you think?

timmysilv commented 3 months ago

🚨 EDIT: this is not true anymore! please ignore 🚨

unless you wish for me to turn back time πŸ•¦

Serializing one gate

>>> Dgate([0], x=1.1).serialize()
PosixPath('/home/ubuntu/src/github.com/XanaduAI/MrMustard/.serialize_cache/Dgate_fe40cc9a65e940ec83850ade04a5cac4.json')
Filesystem after serializing a single gate ``` (mr_mustard) ~/src/github.com/XanaduAI/MrMustard (serialization βœ”) tree .serialize_cache .serialize_cache β”œβ”€β”€ Bargmann_70ba7fe5deef479e8677775e096ef9a0.json β”œβ”€β”€ Bargmann_70ba7fe5deef479e8677775e096ef9a0.npz └── Dgate_fe40cc9a65e940ec83850ade04a5cac4.json ```
The Dgate JSON ```json { "name": "Dgate", "representation": "/home/ubuntu/src/github.com/XanaduAI/MrMustard/.serialize_cache/Bargmann_70ba7fe5deef479e8677775e096ef9a0.json", "wires": "[[], [], [0], [0]]", "class": "mrmustard.lab_dev.transformations.dgate.Dgate" } ```
The Bargmann JSON ```json { "class": "mrmustard.physics.representations.Bargmann", "arrays": "/home/ubuntu/src/github.com/XanaduAI/MrMustard/.serialize_cache/Bargmann_70ba7fe5deef479e8677775e096ef9a0.npz", "backend": "numpy" } ```

Serializing a circuit

>>> Circuit([Dgate([0], x=1.1)]).serialize()
PosixPath('/home/ubuntu/src/github.com/XanaduAI/MrMustard/.serialize_cache/Circuit_82b2f9e1484d42ed9feb0d9cca6299a6.json')
Filesystem after also serializing a circuit (new zip and json) ``` (mr_mustard) ~/src/github.com/XanaduAI/MrMustard (serialization βœ”) tree .serialize_cache .serialize_cache β”œβ”€β”€ Circuit_82b2f9e1484d42ed9feb0d9cca6299a6.json └── collection_3591dab7fd1241bdaf2a45a7fce1134f.zip ```
The Circuit JSON ```json { "order": [ "Dgate_9ec072981e4a4a429c2c9e9bc4eb24f3.json" ], "zipfile": "/home/ubuntu/src/github.com/XanaduAI/MrMustard/.serialize_cache/collection_3591dab7fd1241bdaf2a45a7fce1134f.zip", "subdir": "/home/ubuntu/src/github.com/XanaduAI/MrMustard/.serialize_cache/subcache_ca97053f8d994beb99a74fff29f26d0a", "class": "mrmustard.lab_dev.circuits.Circuit" } ```
timmysilv commented 3 months ago

since the last review, I've made two notable changes:

@apchytr I've added basic versioning just using __version__. I'm open to adding more informative versioning in the future but I find the alpha suffixes cumbersome to maintain, so I'd rather not do it unless we see clear value. Perhaps this can be motivation to make things more stable? So you may only break things between versions (even the most minor version change)

@zeyueN I've posted examples of all the things above πŸ˜„

timmysilv commented 3 months ago

ok since that last comment things have really changed. Check out the updated PR description for details, but basically only Circuit.serialize() writes to file now.

timmysilv commented 3 months ago

@zeyueN I just added the path property to the circuit serialization. Note that it's only evaluated if the user called make_path, so it'll be empty otherwise. I don't save the _graph property, because it depends on non-serialized/non-functional data (a random number tied to the wires object that isn't serialized either)