search

py5coding / py5generator

Meta-programming project that creates the py5 library code.
https://py5coding.org/
GNU General Public License v3.0
52 stars 13 forks source link

Implement `load_strings()`, `save_strings()`, `load_bytes()`, etc. and all the other relevant IO methods #164

Closed hx2A closed 2 years ago

hx2A commented 2 years ago

Methods for load_strings(), save_strings(), load_bytes(), etc. would be useful to beginners.

hx2A commented 2 years ago

I've written the code for load_strings(), save_strings(), load_bytes(), and save_bytes(). Will write the reference docs next.

hx2A commented 2 years ago

Almost done with this. Today I added load and save methods for pickle files, which is a nice Python-specific addition.

hx2A commented 2 years ago

This was closed with #174

villares commented 2 years ago

This is wonderful. About the pickle feature. I remember it was frustrating not to be able to pickle PImage, can we pickle Py5Image objects now?

hx2A commented 2 years ago

No py5 objects can be pickled, just like file handles, sockets, and several other things cannot be pickled.

https://docs.python.org/3/library/pickle.html#what-can-be-pickled-and-unpickled

Consider that py5 objects are linked to Processing Java objects that live in the JVM. Pickle has no way of recreating those Java objects in the JVM, and even if it did, what would happen if the JVM wasn't running at that time? Also, All py5 objects like Py5Image are "linked" to the parent Sketch that created them and it can possibly cause weird exceptions if a Py5Image were to be used with a different Sketch than the one that created it. However, it wouldn't be that hard to "re-parent" them to point to a different Sketch, but this isn't how Processing is designed to be used.

I remember it was frustrating not to be able to pickle PImage, can we pickle Py5Image objects now?

Nevertheless, I do want to understand this frustration and your needs here. Why not just save Py5Image objects to a png file? That works just fine. Perhaps you have an object or data structure that contained Py5Image objects or other py5 objects and you wanted to pickle the whole thing to one pickle file. In that case, the pickle library would throw an exception when it got to a py5 object. I believe it is possible to write code that controls how pickle will serialize certain objects, so maybe we could write some code like this to convert the Py5Image object to a PIL image first, and then pickle that instead. When reading the pickle file, it could convert the PIL image back to a Py5Image object (assuming the JVM is running and this is being done with py5, otherwise you would get a PIL image, which would be the next best thing). This would be achievable, but I don't see an analogous approach for other py5 objects.

If this Py5Image to PIL Image idea is useful, we can add it to the list of potential enhancements.

villares commented 2 years ago

Yeah, I know it is not a trivial enhancement, but the context was:

I use pickle as a very handy/easy way of saving state in complex games/editors/generative applications, between sessions.

Imagine my sketch let's the user load arbitrary images for manipulation, or that it produces sprites/Py5Image buffers in the course of operations.

It would be handy to be able to save/close the the sketch and be able to resume/load the previous state. But also, to have snapshots of process, and to be able to share them with other people easily.

My solution, at the time, was to keep track of a link to an external saved image, and pickle just that. On resume, reload the external resources.

So don't worry too much about this feature. If searialization could be added, great, otherwise, a tutorial explaining how to get around this limitation is on my radar.

On Thu, 13 Oct 2022, 10:03 hx2A, @.***> wrote:

No py5 objects can be pickled, just like file handles, sockets, and several other things cannot be pickled.

https://docs.python.org/3/library/pickle.html#what-can-be-pickled-and-unpickled

Consider that py5 objects are linked to Processing Java objects that live in the JVM. Pickle has no way of recreating those Java objects in the JVM, and even if it did, what would happen if the JVM wasn't running at that time? Also, All py5 objects like Py5Image are "linked" to the parent Sketch that created them and it can possibly cause weird exceptions if a Py5Image were to be used with a different Sketch than the one that created it. However, it wouldn't be that hard to "re-parent" them to point to a different Sketch, but this isn't how Processing is designed to be used.

I remember it was frustrating not to be able to pickle PImage, can we pickle Py5Image objects now?

Nevertheless, I do want to understand this frustration and your needs here. Why not just save Py5Image objects to a png file? That works just fine. Perhaps you have an object or data structure that contained Py5Image objects or other py5 objects and you wanted to pickle the whole thing to one pickle file. In that case, the pickle library would throw an exception when it got to a py5 object. I believe it is possible to write code that controls how pickle will serialize certain objects, so maybe we could write some code like this to convert the Py5Image object to a PIL image first, and then pickle that instead. When reading the pickle file, it could convert the PIL image back to a Py5Image object (assuming the JVM is running and this is being done with py5, otherwise you would get a PIL image, which would be the next best thing). This would be achievable, but I don't see an analogous approach for other py5 objects.

If this Py5Image to PIL Image idea is useful, we can add it to the list of potential enhancements.

— Reply to this email directly, view it on GitHub https://github.com/py5coding/py5generator/issues/164#issuecomment-1277574357, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA4GADDVXOJRGUQC7MEZOVDWDACAHANCNFSM6AAAAAAQUNZOBU . You are receiving this because you commented.Message ID: @.***>

hx2A commented 2 years ago

Hmmm, you could create a Sketch that had a separate "state" object and then pickled that. The state object could contain PIL Images if need be. You can't Pickle an entire py5 Sketch though. Whatever is relevant to the state you wish to recreate could be added the pickle-able state object.

hx2A commented 2 years ago

I just remembered that Java does support serialization of Java objects:

https://www.baeldung.com/java-serialization

Since Python allows you to customize how pickling is done, it would be possible to add code to serialize the Java objects to bytes and then add those bytes to the pickle file. Then more custom code could be used when the pickle file is read back, to convert the bytes back to objects.

So theoretically, any py5 object could be pickled, but making it do so would be non-trivial amount of work.