Make pyglui pickable

[papr]

TL;DR: Please make your objects pickable so I can run separate processes without forking.

Context: I am using your Pupil software. My task is to write a plugin which provides an API to Pupil software for PsychoPy scripts. To do this I created a plugin which starts the PsychoPy script in a separate process. Since you run into a bunch of problems if you do this without care on OS X you need to use the billiard module and disable forking for OS X. More information on this here.

If you disable forking a completely new process is created and everything is pickled and forwarded to it. The problem is that the whole ui is already instantiated at the time I start my PsychoPy script process. Therefore the UI has to be pickled too. But since it does not support pickling yet it crashes. I uploaded the Traceback to Gist.

If you know about another idea how to solve this problem, please let me know. :) Thanks for your great work! <3

[mkassner]

Hi,

I know the perils of this sort of thing... Before going into details here. I m surprised that you are seeing this error. We spawn new processes sometimes in a similar fashion and don't see this error: https://github.com/pupil-labs/pupil/blob/master/pupil_src/player/export_launcher.py

If this does not work, would using something like:

subprocess.call("python your_script.py",terminal=True) 

be acceptable?

This will be very clean. (no shared state whatsoever) All you need then is a way of talking to the script. Maybe via pupil_server? (I m rewriting the msg format right now to be less stupid: https://github.com/pupil-labs/pupil/tree/pupil_server_upgrade)

I still think the first option should work and is preferred...

Let me know if you see a difference in our and your implementation.

best,

Moritz

[papr]

I guess the problem is that I am trying to pass g_pool as argument to the process. Therefore it tries to pickle g_pool and also to pickle the ui elements in g_pool.gui. I guess if I pass a specific selection of what I need the error would not appear anymore. I will test this today.

I would argue that if this is the problem that this ticket can be closed or given a very low priority. What do you think.

[mkassner]

Ah, that explains why I never saw this.

could you just do this?

import copy
remote_g_pool = copy.copy(g_pool)
del remote_g_pool.gui

[papr]

I still get a RuntimeError with following text: Synchronized objects should only be shared between processes through inheritance Traceback.

Maybe it is just a bad idea to pass g_pool. Maybe it is one of its attributes. I will try to remove more of the attributes.

[mkassner]

Ah yes, you cant send queues (g_pool.pupil_queue) so you will need to get rid of that too. I would generally recommend being more explicit with the things you pass to your new process.

[papr]

Yeah. The C-Sync-Wrapper are causing problems, too. Actually the only thing I really need is the g_pool.capture.get_timestamp(). But I guess I could calculate he offset to time.time() once and pass that offset instead of the function itself.

[mkassner]

Can you pass the timebase object?

[papr]

No, which makes sense though. Since it is a <Synchronized wrapper for c_double(0.0)>. There is no way to have a shared memory object between different processes, afaik. I can pass its value at the time of starting the process. Does not help a lot, though.

[mkassner]

Hmm this is annoying.

One option you have is using a pupil_sync node in your new process. It will work locally just fine: https://github.com/pupil-labs/pupil-helpers/blob/master/pupil_sync/pupil_sync_complete.py

This example uses a class so all you need to know is in the first couple fns. With this you can get synchronised time across process and machine boundaries. It will also relay any plugin notifications.

I just did work on this so it will only work with the recent master of pupil.

I hope this can help!

[papr]

Ok, this sounds like the best long-term solution. Thank you very much for your help. Even though this issue might not have been the best place to have this conversation. I am going to post a summary on the pupil discussion group.

Done, I guess. :)