openziti / ziti-sdk-py

Ziti SDK for Python
Apache License 2.0
72 stars 2 forks source link

Issue with openziti & gunicorn #59

Open scox-antillion opened 1 year ago

scox-antillion commented 1 year ago

I'm getting an error when trying to bind openziti to the gunicorn run function. I don't get this error with waitress or the built in flask run functions

the example of the binding

class Server(BaseApplication):
...
    bind_opts = {
        'ztx': ziti_path('identity.json'),
        'service': 'example-service'
    }

    @openziti.zitify(bindings={
        '0.0.0.0:5000': bind_opts,
    })
    def run_zitty(self):
        # self.bind_opts['ztx'] = sys.argv[1]
        # self.bind_opts['service'] = sys.argv[2]
        self.run()

and the app is running on the same ip and port

this is the error we are getting

[2023-10-31 16:59:56 +0000] [91556] [INFO] Worker exiting (pid: 91556)
[2023-10-31 16:59:56 +0000] [91509] [ERROR] Worker (pid:91556) exited with code 255
[2023-10-31 16:59:56 +0000] [91509] [ERROR] Worker (pid:91556) exited with code 255.
[2023-10-31 16:59:56 +0000] [91559] [INFO] Booting worker with pid: 91559
[2023-10-31 16:59:56 +0000] [91559] [ERROR] Exception in worker process
Traceback (most recent call last):
  File "/opt/loenserver/env/lib64/python3.11/site-packages/gunicorn/arbiter.py", line 609, in spawn_worker
    worker.init_process()
  File "/opt/loenserver/env/lib64/python3.11/site-packages/gunicorn/workers/base.py", line 142, in init_process
    self.run()
  File "/opt/loenserver/env/lib64/python3.11/site-packages/gunicorn/workers/sync.py", line 126, in run
    self.run_for_one(timeout)
  File "/opt/loenserver/env/lib64/python3.11/site-packages/gunicorn/workers/sync.py", line 70, in run_for_one
    self.accept(listener)
  File "/opt/loenserver/env/lib64/python3.11/site-packages/gunicorn/workers/sync.py", line 29, in accept
    client, addr = listener.accept()
                   ^^^^^^^^^^^^^^^^^
  File "/opt/loenserver/env/lib64/python3.11/site-packages/openziti/zitisock.py", line 119, in accept
    fd, peer = zitilib.accept(self.fileno())
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/loenserver/env/lib64/python3.11/site-packages/openziti/zitilib.py", line 276, in accept
    check_error(clt)
  File "/opt/loenserver/env/lib64/python3.11/site-packages/openziti/zitilib.py", line 219, in check_error
    raise Exception(err, msg)
Exception: (22, 'unexpected error')
dovholuknf commented 1 year ago

That' was the same error as before that this pr was trying to fix. When I look at that PR, line 219 should be different now. Can you confirm the version of the sdk you're using and how you're getting it? Seems like maybe it's not been published yet.

scox-antillion commented 1 year ago

I am on 0.8.1, there seems to be 12 pre-release versions, on testpypi the latest prerelease is 0.8.1.post0.dev12 issue 51 i thought was a similar bug but a comment that was left said that it wasn't.

Ill try and repo this with the latest pre-release version and see if the issue persists

dovholuknf commented 1 year ago

Ah, it's entirely possible I just "assumed" it was the fix since I saw that PR just yesterday and noticed the issue closed. :) Thanks. I'll poke @ekoby and see what he thinks.

scox-antillion commented 1 year ago

So I retested it using both the default "sync" worker type and the evenlet worker type and they both still fail with this pre-release version. Its getting a different but very similar error

[2023-11-03 12:26:09 +0000] [110910] [INFO] Worker exiting (pid: 110910)
[2023-11-03 12:26:09 +0000] [110719] [ERROR] Worker (pid:110910) exited with code 255
[2023-11-03 12:26:09 +0000] [110719] [ERROR] Worker (pid:110910) exited with code 255.
[2023-11-03 12:26:09 +0000] [110913] [INFO] Booting worker with pid: 110913
[2023-11-03 12:26:09 +0000] [110913] [ERROR] Exception in worker process
Traceback (most recent call last):
  File "/opt/loenserver/lib64/python3.11/site-packages/gunicorn/arbiter.py", line 609, in spawn_worker
    worker.init_process()
  File "/opt/loenserver/lib64/python3.11/site-packages/gunicorn/workers/base.py", line 142, in init_process
    self.run()
  File "/opt/loenserver/lib64/python3.11/site-packages/gunicorn/workers/sync.py", line 126, in run
    self.run_for_one(timeout)
  File "/opt/loenserver/lib64/python3.11/site-packages/gunicorn/workers/sync.py", line 70, in run_for_one
    self.accept(listener)
  File "/opt/loenserver/lib64/python3.11/site-packages/gunicorn/workers/sync.py", line 29, in accept
    client, addr = listener.accept()
                   ^^^^^^^^^^^^^^^^^
  File "/opt/loenserver/lib64/python3.11/site-packages/openziti/zitisock.py", line 119, in accept
    fd, peer = zitilib.accept(self.fileno())
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/loenserver/lib64/python3.11/site-packages/openziti/zitilib.py", line 280, in accept
    check_error(clt)
  File "/opt/loenserver/lib64/python3.11/site-packages/openziti/zitilib.py", line 223, in check_error
    raise OSError(err, msg)
OSError: [Errno 22] Invalid argument

This seem lines up with the latest change that adds the raise OSError(err, msg) line

ekoby commented 1 year ago

as I mentioned in the other issue, gunicorn spawns child/worker processes. that pattern is no supported (yet) by underlying Ziti SDK (issue https://github.com/openziti/ziti-sdk-c/issues/568)