Incompatibility with PyQt and/or Flask?

[Exioncore]

I'm currently working on an application made in PyQt5 meant to show system stats through the use of a Flask server. I wanted to add the ability to show steam chat notifications on it however I'm encountering issues with making the steam api work. My issue is I'm unable to perform the login. For better context I will provide snippets of what I believe are the main pieces of my program that could lead to issues with the api.

Main:

# Create PyQt5 app
app = QApplication(sys.argv)
# Flask server
server = Server('Ryder Engine')
# This creates the UI of my application and binds some UI components to end points of the Flask server.
window = RyderDisplay()
window.initialize(server)
# Run Server
threading.Thread(target=server.run, daemon=True).start()

Home page of the PyQt5 application (window object in Main, the steam api is initialized through an object inside here):

def __init__(self, window, server : Server):
    self._window = window
    self._client = Client()
    self._server = server

    self._client.subscribeToRyderEngine()
    server.add_endpoint('/status', 'status', self._newStatus)

def create_ui(self, path):
    # Initialize
    path =  path + '/config.json'
    self._fps, self._ui = HomeConfigurationParser.parse(self._window, self._client, self._server, path)
    # Refresher
    self._timer = QTimer()
    self._timer.timeout.connect(self.update)
    self._timer.start(1000 / self._fps)

def _newStatus(self, request):
    self._status = request

def update(self):
    # Update UI
    for elem in self._ui:
        elem.update(self._status)
    # Reset
    if self._status is not None:
        self._status = None

The Server object:

class EndpointAction(object):
    def __init__(self, action):
        self.action = action
        self.response = Response(status=200, headers={})

    def __call__(self, *args):
        self.action(request.get_json())
        return self.response

class Server(object):
    def __init__(self, name):
        self.app = Flask(name)

    def run(self, port=9520):
        print("Server started")
        WSGIServer(('0.0.0.0', port), self.app).serve_forever()

    def add_endpoint(self, endpoint=None, endpoint_name=None, handler=None):
        print("Endpoint \"" + endpoint_name + "\" added")
        self.app.add_url_rule(endpoint, endpoint_name, EndpointAction(handler), methods=['POST'])

The object where the steam api itself is handled

class SteamNotifier(object):
    def __init__(self, client:Client, server:Server, steam_notification, path):
        self._cache = path + '/cache/'
        if not os.path.exists(self._cache):
            os.makedirs(self._cache)

        self._steamClient = SteamClient()
        self._client = client
        self._steam_notification = steam_notification
        # Bind Server
        server.add_endpoint('/steamLogin', 'steamLogin', self._steamLoginData)
        server.add_endpoint('/steam2fa', 'steam2fa',self._steam2faData)
        # Hook Steam Client Events
        self._steamClient.on(SteamClient.EVENT_AUTH_CODE_REQUIRED, self.auth_code_prompt)
        self._steamClient.on("FriendMessagesClient.IncomingMessage#1", self.handle_message)
        self._steamClient.on(SteamClient.EVENT_LOGGED_ON, self.login_success)
        self._steamClient.on(SteamClient.EVENT_CHANNEL_SECURED, self.login_secured)
        self._steamClient.on(SteamClient.EVENT_ERROR, self.login_error)
        self._steamClient.on(SteamClient.EVENT_CONNECTED, self.connected)
        self._steamClient.on(SteamClient.EVENT_DISCONNECTED, self.disconnected)
        # Start Login Sequence
        self._steamClient.set_credential_location(self._cache)
        self._steam_notification('Steam', 'Login', 'Requesting Login Data')
        client.querySteamLogin()

    def _steamLoginData(self, request):
        self._login_data = [request[0], request[1]]
        print('1-Steam logging in')
        self._steamClient.login(username=self._login_data[0], password=self._login_data[1])
        print('2-Steam logging in')

    def _steam2faData(self, request):
        print("Steam 2FA: "+request)
        self._steamClient.login(two_factor_code=request, username=self._login_data[0], password=self._login_data[1])

    # Handle SteamClient events
    def connected(self):
        print("Connected")

    def disconnected(self):
        print("Disconnected")

    def login_secured(self):
        print("Login secured")
        if self._steamClient.relogin_available:
            self._steamClient.relogin()

    def login_error(self, data):
        print("Login error")

    def auth_code_prompt(self, is2fa, code_mismatch):
        print("Steam2FA Required")
        self._steam_notification('Steam', 'Login', 'Requesting 2 Factor Authentication')
        self._client.querySteam2FA()

    def handle_message(self, msg):
        if msg.body.chat_entry_type == EChatEntryType.ChatMsg and not msg.body.local_echo:
            user = self._steamClient.get_user(msg.body.steamid_friend)
            text = msg.body.message
            self._steam_notification('Steam', user.name, text)

    def login_success(self):
        self._steam_notification('Steam', self._steamClient.username, 'Logged in!')

Debug Trace:

Endpoint "steamLogin" added
Endpoint "steam2fa" added
PYDEV DEBUGGER WARNING:
sys.settrace() should not be used when the debugger is being used.
This may cause the debugger to stop working correctly.
If this is needed, please check: 
http://pydev.blogspot.com/2007/06/why-cant-pydev-debugger-work-with.html
to see how to restore the debug tracing back correctly.
Call Location:
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\site-packages\gevent\threadpool.py", line 151, in _before_run_task
    def _before_run_task(self, func, args, kwargs, thread_result,

Endpoint "notification" added
Server started
It seems that the gevent monkey-patching is being used.
Please set an environment variable with:
GEVENT_SUPPORT=True
to enable gevent support in the debugger.
192.168.1.114 - - [2021-02-16 01:57:21] "POST /status HTTP/1.1" 200 115 0.004003
192.168.1.114 - - [2021-02-16 01:57:21] "POST /foregroundProcessName HTTP/1.1" 200 115 0.009007
192.168.1.114 - - [2021-02-16 01:57:21] "POST /foregroundProcessName HTTP/1.1" 200 115 0.001001
192.168.1.114 - - [2021-02-16 01:57:21] "POST /status HTTP/1.1" 200 115 0.001000
... 192.168.1.114 are just messages of my own app
192.168.1.114 - - [2021-02-16 01:57:30] "POST /status HTTP/1.1" 200 115 0.001000
1-Steam logging in
192.168.1.114 - - [2021-02-16 01:57:30] "POST /foregroundProcessName HTTP/1.1" 200 115 0.001000
Connected
192.168.1.114 - - [2021-02-16 01:57:31] "POST /status HTTP/1.1" 200 115 0.001001
192.168.1.114 - - [2021-02-16 01:57:32] "POST /status HTTP/1.1" 200 115 0.001000
192.168.1.114 - - [2021-02-16 01:57:33] "POST /status HTTP/1.1" 200 115 0.001000
... No more messages related to the Steam API but my application continues to function as normal

As it can be seen in the debug trace it would appear that the login of the steam api never completes for some reason whilst it is supposed to throw the event SteamClient.EVENT_AUTH_CODE_REQUIRED. (I do not get a steam guard code notification on the mobile authenticator either, the results are the same if I put in wrong login data) Apologies for the very lengthy post, I've tried to mention only the things which I feel like are likely interfering with the correct functioning of the steam api. I had the api working before on its own but in this specific application that I've been working on it consistently refuses to work. Any clues as of to what might be going wrong here? Again, apologies if this is the wrong place to ask about this.

[rossengeorgiev]

It mostly like because you are not yielding to the gevent loop, and SteamClient cannot run. I don't see where instantiate your SteamNotifier object. You should be able to use a separate thread and then keep PyQT in the main thread. However, you need to make sure the WSGI server running flask is cooperative, see the following recipe:

https://github.com/ValvePython/steam/blob/master/recipes/2.SimpleWebAPI/run_webapi.py

[main thread: pyqt] + [second thread: flask + steamclient]

gevent will create a separate loop for each thread

[Exioncore]

Thanks for the reply, your insight indeed proved to be correct. I solved the issue by dedicating a thread to the steam client but also by "converting the PyQt app into gevent".

Main file (source: https://github.com/tmc/pyqt-by-example/commit/b5d6c61daaa4d2321efe89679b1687e85892460a)

def pyqtLoop(app):
    while True:
        app.processEvents()
        gevent.sleep(0.005)

if __name__ == "__main__":
    # Create PyQt5 app
    app = QApplication(sys.argv)
    # Flask server
    server = Server('Ryder Engine')
    # Create the instance of our Window
    window = RyderDisplay()
    window.initialize(server)
    # Run
    gevent.joinall([gevent.spawn(server.run), gevent.spawn(pyqtLoop, app)])

The object where the steam api itself is handled.

class SteamNotifier(threading.Thread):
    def __init__(self, client:Client, server:Server, steam_notification, path):
        super(SteamNotifier, self).__init__(name='Steam Notifier Thread')

        self._cache = path + '/cache/'
        if not os.path.exists(self._cache):
            os.makedirs(self._cache)

        self._client = client
        self._steam_notification = steam_notification
        # Bind Server
        server.add_endpoint('/steamLogin', 'steamLogin', self._steamLoginData)
        server.add_endpoint('/steam2fa', 'steam2fa',self._steam2faData)   

    def run(self):
        self._steamClient = SteamClient()
        # Hook Steam Client Events
        self._steamClient.on(SteamClient.EVENT_AUTH_CODE_REQUIRED, self.auth_code_prompt)
        self._steamClient.on("FriendMessagesClient.IncomingMessage#1", self.handle_message)
        self._steamClient.on(SteamClient.EVENT_LOGGED_ON, self.login_success)
        self._steamClient.on(SteamClient.EVENT_CHANNEL_SECURED, self.login_secured)
        self._steamClient.on(SteamClient.EVENT_ERROR, self.login_error)
        self._steamClient.on(SteamClient.EVENT_CONNECTED, self.connected)
        self._steamClient.on(SteamClient.EVENT_DISCONNECTED, self.disconnected)
        self._steamClient.on(SteamClient.EVENT_NEW_LOGIN_KEY, self.new_login_key)
        # Start Login Sequence
        if os.path.exists(self._cache + 'steam.txt'):
            f = open(self._cache + 'steam.txt', 'r')
            data = f.readlines()
            f.close()
            self._steamClient.login(username=data[0].replace('\n',''), login_key=data[1])
        else:
            self._steamClient.set_credential_location(self._cache)
            self._client.querySteamLogin()
            self._steam_notification('Steam', 'Login', 'Requesting Login Data')
        while True:
            gevent.sleep(0.1)

    def _steamLoginData(self, request):
        self._login_data = [request[0], request[1]]
        self._steamClient.login(username=self._login_data[0], password=self._login_data[1])
        print('Steam ' + request[0] + ' logging in')

    def _steam2faData(self, request):
        print("Steam 2FA: "+request)
        self._steamClient.login(two_factor_code=request, username=self._login_data[0], password=self._login_data[1])

    # Handle SteamClient events
    def connected(self):
        print("Connected")
        if self._steamClient.relogin_available:
            self._steamClient.relogin()

    def disconnected(self):
        print("Disconnected")
        if self._steamClient.relogin_available:
            self._steam_notification('Steam', self._steamClient.username, 'Connection lost! Re-trying...')
            self._steamClient.reconnect(maxdelay=30)

    def login_secured(self):
        print("Login secured")
        if self._steamClient.relogin_available:
            self._steamClient.relogin()

    def login_error(self, data):
        print("Login error")
        print(data)

    def auth_code_prompt(self, is2fa, code_mismatch):
        print("Steam2FA Required")
        self._steam_notification('Steam', 'Login', 'Requesting 2 Factor Authentication')
        self._client.querySteam2FA()

    def handle_message(self, msg):
        if msg.body.chat_entry_type == EChatEntryType.ChatMsg and not msg.body.local_echo:
            user = self._steamClient.get_user(msg.body.steamid_friend)
            text = msg.body.message
            self._steam_notification('Steam', user.name, text)

    def login_success(self):
        print("Login successfull")
        self._steam_notification('Steam', self._steamClient.username, 'Logged in!')

    def new_login_key(self):
        print("New login key")
        f = open(self._cache + 'steam.txt', 'w')
        f.write(self._steamClient.username + '\n' + self._steamClient.login_key)
        f.close()

This last object is instantiated and run as such:

self._steam = SteamNotifier(client, server, self.newNotification, path)
gevent.spawn_later(2, self._steam.run)

Furthermore, given that the app is now effectively multi-threaded I've added mutex locks around places where race-conditions might occur. I posted this in case it might help someone solve the issue if they were to encounter it. Do note that the SteamNotifier class stores login-data such that upon next logins it doesn't require any manual input.

[rossengeorgiev]

Cpython is not multi-thread because of GIL. Python threads can only really allow you to no block when waiting on io. gevent uses greenlets, and so called green-threads (not real threads). It essentially allows for code to cooperate, but ultimately it runs serially. Nothing ever runs in parallel.

Your solution could now be stalled by either the UI loop, or by a greenlet. If implement my suggestion, the UI can only stall the main thread, and all the greenlets can only stall the other thread. They won't be running in parallel still, but they wont be able to stall each other. Also, for this to work, you must not monkey patch the threading module.