sirk390 / wxasync

asyncio support for wxpython
MIT License
77 stars 9 forks source link

WxAsyncApp must be created before use of AsyncBind is allowed #2

Closed abulka closed 5 years ago

abulka commented 5 years ago

My app is built as per the wxPython advice

Normally you would derive from this class and implement an OnInit method that creates a frame and then calls self.SetTopWindow(frame), however wx.App is also usable on it’s own without derivation.

I'm trying to convert my application to use wxasync - its a very exciting prospect to potentially be able to introduce concurrency into my wxPython app. However wxasync seems to insist that the WxAsyncApp is created first, and any AsyncBind calls must happen later - say, during the frame creation - as per the main wxasync example.

As mentioned, my app simply isn't structured that way. My app creates a frame within the app's OnInit() and my menu binding (where I want to use AsyncBind) is also done in the app's OnInit(). Unfortunately my attempts to use AsyncBind inside OnInit() result in Exception: Create a 'WxAsyncApp' first.

Do I need to refactor my app and frame into separate objects then wire them together later, in order to successfully use wxasync? Or is there a way I can stay with my existing architecture?

sirk390 commented 5 years ago

Hi Andy, Could you send me a minimal example of what you have in your normal wx.App. I can have a look at it then. Maybe it might be worth it to update wxasync if this use-case is supported by wx.App. Thanks,

On Thu, Feb 14, 2019 at 1:24 PM Andy Bulka notifications@github.com wrote:

My app is built as per the wxPython advice https://wxpython.org/Phoenix/docs/html/wx.App.html

Normally you would derive from this class and implement an OnInit method that creates a frame and then calls self.SetTopWindow(frame), however wx.App is also usable on it’s own without derivation.

I'm trying to convert my application to use wxasync - its a very exciting prospect to potentially be able to introduce concurrency into my wxPython app. However wxasync seems to insist that the WxAsyncApp is created first, and any AsyncBind calls must happen later - say, during the frame creation

  • as per the main wxasync example.

As mentioned, my app simply isn't structured that way. My app creates a frame within the app's OnInit() and my menu binding (where I want to use AsyncBind) is also done in the app's OnInit(). Unfortunately my attempts to use AsyncBind inside OnInit() result in Exception: Create a 'WxAsyncApp' first.

Do I need to refactor my app and frame into separate objects then wire them together later, in order to successfully use wxasync? Or is there a way I can stay with my existing architecture?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/sirk390/wxasync/issues/2, or mute the thread https://github.com/notifications/unsubscribe-auth/AA8CG58MmvZ31YdDoGKVgS_wyCBvNH8qks5vNVWPgaJpZM4a7fAR .

abulka commented 5 years ago

Here is the simplest normal wxPython app which demonstrates my architecture:

import wx

class MainApp(wx.App):
    def OnInit(self):
        self.frame = wx.Frame(None, -1, "test",)
        self.frame.CreateStatusBar()
        self.frame.Show(True)
        return True

def main():
    application = MainApp(0)
    application.MainLoop()

if __name__ == "__main__":
    main()

Here is a slightly enhanced version of the above, with a menu added, which triggers the display of three status messages, with a sleep in between. It runs ok. If you uncomment the commented bits, it should turn into a wxasync app, but it doesn't - it gives me the exception I was referring to.

import wx
import time

from wxasync import AsyncBind, WxAsyncApp, StartCoroutine
import asyncio
from asyncio.events import get_event_loop

# class MainApp(WxAsyncApp):
class MainApp(wx.App):

    def OnInit(self):
        self.frame = wx.Frame(None, -1, "test",)
        self.frame.CreateStatusBar()
        self.frame.Show(True)
        self.InitMenus()
        return True

    def InitMenus(self):
        menuBar = wx.MenuBar()
        menu1 = wx.Menu()
        id = wx.NewIdRef()
        menu1.Append(id, "test - async call\tCtrl-1")

        self.Bind(wx.EVT_MENU, self.callback, id=id)
        # AsyncBind(wx.EVT_MENU, self.async_callback, id)

        menuBar.Append(menu1, "&Experiments")
        self.frame.SetMenuBar(menuBar)

    def callback(self, event):
        self.frame.SetStatusText("Menu item clicked")
        wx.SafeYield()
        time.sleep(2)
        self.frame.SetStatusText("Working")
        wx.SafeYield()
        time.sleep(2)
        self.frame.SetStatusText("Completed")

    async def async_callback(self, event):
        self.frame.SetStatusText("Button clicked")
        await asyncio.sleep(1)
        self.frame.SetStatusText("Working")
        await asyncio.sleep(1)
        self.frame.SetStatusText("Completed")

def main():
    application = MainApp(0)
    application.MainLoop()

def main_async():
    # see https://github.com/sirk390/wxasync
    application = MainApp(0)
    loop = get_event_loop()
    loop.run_until_complete(application.MainLoop())

if __name__ == "__main__":
    main()
    # main_async()
sirk390 commented 5 years ago

There were 3 different issues: 1/ I replaced this: if type(app) is not WxAsyncApp: By this if not isinstance(app, WxAsyncApp):

2/ Then the order of initialization had to change also in the constructor of WxAsyncApp.

3/ The argument "id" was missing in "AsyncBind" which I added.

Now it works but you have to call: AsyncBind(wx.EVT_MENU, self.async_callback, menu1, id=id) (e.g. with the "menu1" object and not on the "MainApp" otherwise it raises exceptions at shutdown. I still need to see why)

Have a look here if this works for you: https://github.com/sirk390/wxasync/commit/7a7574126bbea3c1e151995bf6f03ed07e0b2fbf

abulka commented 5 years ago

Thanks! Your changes which I installed with

pip uninstall wxasync
pip install git+https://github.com/sirk390/wxasync.git@7a7574126bbea3c1e151995bf6f03ed07e0b2fbf

made the example app I gave you run asynchronously - excellent. This will save me a massive amount of work, I won't need to re-architect my rather large app, which happens to be the Python UML tool Pynsource.

My next challenge is that the async loading of a particular http resource in my app is forcing me to declare async/await all the way up my call hierarchy, which is getting out of control and spreading through my codebase like wildfire - some have criticised async/await as a anti-pattern for this very reason. I'm sure there is a way out of this mess - I'll have to investigate and research this more deeply, as I'm just a beginner with this async/await stuff. Anyway - that's not your problem - your prompt changes have allowed my wxPython app to at least get onto the async/await playing field...

abulka commented 5 years ago

I have noticed that async activity stops whilst buttons are held down, and whilst menus are being displayed. Presumably this is something deep in the wxpython loop and cannot be easily fixed?

abulka commented 5 years ago

I had an idea about avoiding the "turtles" problem in async/await, where any use of async/await necessitates using async/await syntax on all callers, all the way up the call hierarchy and unfortunately one thing leads to another and it spreads rapidly throughout the codebase. The idea is that, as long as you don't need to wait for the result, it often might be enough for some bit of code to call StartCoroutine(), thereby work happens and the GUI is still responsive. In fact the main wxasync example demonstrates this. You can in fact trigger StartCoroutine calls from regular button clicks bound with regular bind. Nice. However, the original problem Exception: Create a 'WxAsyncApp' first still happens if you use the binding within OnInit() architecture:

import wx
import time
from wxasync import AsyncBind, WxAsyncApp, StartCoroutine
import asyncio
from asyncio.events import get_event_loop
import wx.lib.newevent

SomeNewEvent, EVT_SOME_NEW_EVENT = wx.lib.newevent.NewEvent()

class MainApp(WxAsyncApp):

    def OnInit(self):
        self.frame = wx.Frame(None, -1, "test",)
        self.frame.CreateStatusBar()
        self.frame.Show(True)
        self.InitMenus()
        return True

    def InitMenus(self):
        menuBar = wx.MenuBar()
        menu1 = wx.Menu()

        id = wx.NewIdRef()
        menu1.Append(id, "test - call StartCoroutine() \tCtrl-1")
        self.Bind(wx.EVT_MENU, self.func_calls_start_coroutine, id=id)

        # Finalise
        menuBar.Append(menu1, "&Experiments")
        self.frame.SetMenuBar(menuBar)

    def func_calls_start_coroutine(self, event):
        StartCoroutine(self.async_callback, self)

    async def async_callback(self, event):
        self.frame.SetStatusText("Button clicked")
        await asyncio.sleep(1)
        self.frame.SetStatusText("Working")
        await asyncio.sleep(1)
        self.frame.SetStatusText("Completed")

def main_async():
    # see https://github.com/sirk390/wxasync
    application = MainApp(0)
    loop = get_event_loop()
    loop.run_until_complete(application.MainLoop())

if __name__ == "__main__":
    main_async()

The above code generates an error. Whereas it works, if the WxAsyncApp() is created first:

import wx
from wxasync import AsyncBind, WxAsyncApp, StartCoroutine
import asyncio
from asyncio.events import get_event_loop
import time

import wx.lib.newevent
SomeNewEvent, EVT_SOME_NEW_EVENT = wx.lib.newevent.NewEvent()
SomeNewEventAsync, EVT_SOME_NEW_EVENT_ASYNC = wx.lib.newevent.NewEvent()

class TestFrame(wx.Frame):
    def __init__(self, parent=None):
        super(TestFrame, self).__init__(parent)
        vbox = wx.BoxSizer(wx.VERTICAL)
        button1 =  wx.Button(self, label="AsyncBind (original wxasync example)")
        button2 =  wx.Button(self, label="Start/stop clock via StartCoroutine()")
        button3 =  wx.Button(self, label="Emit Custom Event (sync)")
        button4 =  wx.Button(self, label="Emit Custom Event (async)")
        self.edit =  wx.StaticText(self, style=wx.ALIGN_CENTRE_HORIZONTAL|wx.ST_NO_AUTORESIZE)
        self.edit_timer =  wx.StaticText(self, style=wx.ALIGN_CENTRE_HORIZONTAL|wx.ST_NO_AUTORESIZE)
        vbox.Add(button1, 2, wx.EXPAND|wx.ALL)
        vbox.Add(button2, 2, wx.EXPAND|wx.ALL)
        vbox.Add(button3, 2, wx.EXPAND|wx.ALL)
        vbox.Add(button4, 2, wx.EXPAND|wx.ALL)
        vbox.AddStretchSpacer(1)
        vbox.Add(self.edit, 1, wx.EXPAND|wx.ALL)
        vbox.Add(self.edit_timer, 1, wx.EXPAND|wx.ALL)
        self.SetSizer(vbox)
        self.Layout()
        self.clock_on = False

        """Original direct binding - works ok"""
        AsyncBind(wx.EVT_BUTTON, self.async_callback, button1)

        """
        Regular method calls StartCoroutine() - works ok
        No need for async/await syntax except on the final async method! Turtle avoidance success 
        because no need for async/await syntax turtles all the way up the calling chain.
        PROVISO: WxAsyncApp() object must be created first, frame creation within OnInit fails.
        """
        self.Bind(wx.EVT_BUTTON, self.regular_func_starts_coroutine, button2)

        """
        Regular method broadcast a custom event - works ok 
        But this is the synchronous version.  Its the async version we want to get working.
        """
        self.Bind(wx.EVT_BUTTON, self.regular_func_raises_custom_event, button3)
        self.Bind(EVT_SOME_NEW_EVENT, self.callback)  # bind custom event to synchronous handler - works OK

        """
        Regular method broadcast a custom event - doesn't work
        bind custom event to asynchronous handler - doesn't work
        """
        self.Bind(wx.EVT_BUTTON, self.regular_func_raises_custom_async_event, button4)
        id = wx.NewIdRef()
        AsyncBind(EVT_SOME_NEW_EVENT_ASYNC, self.async_callback, self, id=id)  # <- how to we fix this?

    def regular_func_raises_custom_event(self, event):
        print("trigger demo via custom event")
        # Create and post the event
        evt = SomeNewEvent(attr1="hello", attr2=654)
        wx.PostEvent(self, evt)

    def regular_func_raises_custom_async_event(self, event):
        print("trigger async demo via custom event")
        # Create and post the event
        evt = SomeNewEventAsync(attr1="hello", attr2=654)
        wx.PostEvent(self, evt)

    def regular_func_starts_coroutine(self, event):
        self.clock_on = not self.clock_on
        if self.clock_on:
            print(f"triggering an async call via StartCoroutine()")
            StartCoroutine(self.update_clock, self)
        else:
            print("clock flag off, coroutine will stop looping, drop through and complete")

    def callback(self, event):
        self.edit.SetLabel("Button clicked (synchronous)")
        wx.SafeYield()
        time.sleep(1)
        self.edit.SetLabel("Working (synchronous)")
        wx.SafeYield()
        time.sleep(1)
        self.edit.SetLabel("Completed (synchronous)")
        time.sleep(1)
        self.edit.SetLabel("")

    async def async_callback(self, event):
        self.edit.SetLabel("Button clicked")
        await asyncio.sleep(1)
        self.edit.SetLabel("Working")
        await asyncio.sleep(1)
        self.edit.SetLabel("Completed")
        await asyncio.sleep(1)
        self.edit.SetLabel("")

    async def update_clock(self):
        while self.clock_on:
            self.edit_timer.SetLabel(time.strftime('%H:%M:%S'))
            await asyncio.sleep(0.5)
        self.edit_timer.SetLabel("")

app = WxAsyncApp()
frame = TestFrame()
frame.Show()
app.SetTopWindow(frame)
loop = get_event_loop()
loop.run_until_complete(app.MainLoop())

In the above (rather long) example the coroutine call from a regular (non async method) works - because the app is created before the frame.

The reason the above example is so long is that in it I'm testing another theory: that I can generate a custom event and emit it from any place in my code base, to trigger an async method which is bound to that custom event. That would be really handy, especially if we can't get the StartCoroutine() technique to work from within an app with OnInit() architecture. Unfortunately I can't get the custom event idea to work (run the illustrative example code above). Yes I can trigger a synchronous method from the custom event, but not an asynchronous one. I'm pretty sure the problem syntax is:

self.Bind(wx.EVT_BUTTON, self.regular_func_raises_custom_async_event, button4)
id = wx.NewIdRef()
AsyncBind(EVT_SOME_NEW_EVENT_ASYNC, self.async_callback, self, id=id)  # <- how to we fix this?

which may relate to your comment:

AsyncBind(wx.EVT_MENU, self.async_callback, menu1, id=id) (e.g. with the "menu1" object and not on the "MainApp" otherwise it raises exceptions at shutdown. I still need to see why)

re the mysterious third parameter to AsyncBind.

Sorry for the long post! I guess I'm exercising what I can do with wxasync and wxpython and its limits, which is arguably healthy :-)

sirk390 commented 5 years ago

Yes Andy, I noticed this issue as well. I just forgot to make an extra change in "StartCoroutine": so that it is now also: "if not isinstance(app, WxAsyncApp):" instead of "if type(app) is not WxAsyncApp:"

This commit should fix that: https://github.com/sirk390/wxasync/commit/a2aa71eca525169f996be46a4515674028c8e6f5

For the second point, you shouldn't use any "id" parameter. I use this a lot in the file "test/test_perfs.py" where I do:

TestEvent, EVT_TEST_EVENT = NewEvent()
AsyncBind(EVT_TEST_EVENT, self.wx_loop_func, self)

wx.PostEvent(self, TestEvent(t1=time.time()))
sirk390 commented 5 years ago

I didn't know about the menu blocking the event loop, but I knew about the "Window Resize" blocking it. These are indeed deeper issues, so more difficult to fix. I should write that down next time to describe the solution because I investigated it a lot. I think it is possible but would maybe require a patch to WxWindows.

abulka commented 5 years ago

@sirk390 The new version fixes the StartCoroutine() call problem. And your kind advice about omitting the id made the custom event technique work too - thanks!

So I created a final version of my test app using menu items instead of buttons, and using the OnInit approach. It works perfectly:

# Demonstrates invoking async methods from inside regular methods using
# a call to StartCoroutine() and also by the technique of emitting a custom event
# which is bound using AsyncBind to an asynchronous method. All works!
# Uses architecture where frame is created inside app OnInit.

import wx
import time
from wxasync import AsyncBind, WxAsyncApp, StartCoroutine
import asyncio
from asyncio.events import get_event_loop
import wx.lib.newevent

SomeNewEvent, EVT_SOME_NEW_EVENT = wx.lib.newevent.NewEvent()
SomeNewEventAsync, EVT_SOME_NEW_EVENT_ASYNC = wx.lib.newevent.NewEvent()

class MainApp(WxAsyncApp):
    def OnInit(self):
        self.frame = wx.Frame(None, -1, "test",)
        self.frame.CreateStatusBar()
        self.frame.Show(True)
        self.clock_on = False
        self.InitMenus()
        return True

    def InitMenus(self):
        menuBar = wx.MenuBar()
        menu1 = wx.Menu()

        id = wx.NewIdRef()
        menu1.Append(id, "AsyncBind (original wxasync example)\tCtrl-1")
        AsyncBind(wx.EVT_MENU, self.async_callback, menu1, id=id)

        id = wx.NewIdRef()
        menu1.Append(id, "Start/stop clock via StartCoroutine()\tCtrl-2")
        self.Bind(wx.EVT_MENU, self.regular_func_starts_coroutine, id=id)

        id = wx.NewIdRef()
        menu1.Append(id, "Emit Custom Event (sync)\tCtrl-3")
        self.Bind(wx.EVT_MENU, self.regular_func_raises_custom_event, id=id)
        self.Bind(EVT_SOME_NEW_EVENT, self.callback)  # bind custom event to synchronous handler - works OK

        id = wx.NewIdRef()
        menu1.Append(id, "Emit Custom Event (async)\tCtrl-4")
        self.Bind(wx.EVT_MENU, self.regular_func_raises_custom_async_event, id=id)
        AsyncBind(EVT_SOME_NEW_EVENT_ASYNC, self.async_callback, self)  # don't specify id

        # Finalise menu
        menuBar.Append(menu1, "&Experiments")
        self.frame.SetMenuBar(menuBar)

    def regular_func_raises_custom_event(self, event):
        print("trigger demo via custom event")
        # Create and post the event
        evt = SomeNewEvent(attr1="hello", attr2=654)
        wx.PostEvent(self, evt)

    def regular_func_raises_custom_async_event(self, event):
        print("trigger async demo via custom event")
        # Create and post the event
        evt = SomeNewEventAsync(attr1="hello", attr2=654)
        wx.PostEvent(self, evt)

    def regular_func_starts_coroutine(self, event):
        self.clock_on = not self.clock_on
        if self.clock_on:
            print(f"triggering an async call via StartCoroutine()")
            StartCoroutine(self.update_clock, self)
        else:
            print("clock flag off, coroutine will stop looping, drop through and complete")

    # --------

    def callback(self, event):
        self.frame.SetStatusText("Menu item clicked (synchronous)")
        wx.SafeYield()
        time.sleep(1)
        self.frame.SetStatusText("Working (synchronous)")
        wx.SafeYield()
        time.sleep(1)
        self.frame.SetStatusText("Completed (synchronous)")
        wx.SafeYield()
        time.sleep(1)
        self.frame.SetStatusText("")

    async def async_callback(self, event):
        self.frame.SetStatusText("Menu item clicked")
        await asyncio.sleep(1)
        self.frame.SetStatusText("Working")
        await asyncio.sleep(1)
        self.frame.SetStatusText("Completed")
        await asyncio.sleep(1)
        self.frame.SetStatusText("")

    async def update_clock(self):
        while self.clock_on:
            self.frame.SetStatusText(time.strftime('%H:%M:%S'))
            await asyncio.sleep(0.5)
        self.frame.SetStatusText("")

def main():
    application = MainApp(0)
    application.MainLoop()

def main_async():
    # see https://github.com/sirk390/wxasync
    application = MainApp(0)
    loop = get_event_loop()
    loop.run_until_complete(application.MainLoop())

if __name__ == "__main__":
    # main()
    main_async()

The only minor remaining problem is when I quit the app, I get KeyError: <__main__.MainApp object which is generated by wxasync del self.BoundObjects[obj]?

sirk390 commented 5 years ago

The 2 exceptions are happening because wx.App receives the wx.EVT_WINDOW_DESTROY event 3 times (One for the wx.Frame and twice for wx.Window). I'm not sure why. I would like to do more tests and try to understand why before the simple solution "check if it was already received and do nothing in that case. "

In the meantime, you can send events to the "Frame" which work without printing exceptions:

1/ Replace this: AsyncBind(EVT_SOME_NEW_EVENT_ASYNC, self.async_callback, self) # don't specify id by AsyncBind(EVT_SOME_NEW_EVENT_ASYNC, self.async_callback, self.frame) # don't specify id

2/ And replace this: wx.PostEvent(self, evt) by this: wx.PostEvent(self.frame, evt)

sirk390 commented 5 years ago

I updated the pip package for the "WxAsyncApp must be created before use of AsyncBind". Tell me if everything is ok and I will close this issue

abulka commented 5 years ago

Just tried the new pip version 0.3 and it works ok.

Your advice re sending the custom event to the frame works too - so all good. Thank you so much for your assistance and for making wxasync available to the wxPython community!