search

CaliDog / certstream-server

Certificate Transparency Log aggregation, parsing, and streaming service written in Elixir
https://certstream.calidog.io
MIT License
271 stars 75 forks source link

certstream-server-python!!! #84

Closed ld73 closed 2 years ago

ld73 commented 2 years ago

Good afternoon! I understand that this message does not relate to this section, but I have not found any other solution. Question to the developer, do you have a project https://github.com/CaliDog/certstream-server-python and it has not been supported for more than 4 years, how long is this version workable? Of course, I had problems during the start of the program:

[INFO:certstream.watcher] 2022-02-11 10:36:39,558 - + Trust Asia Log2023 loop: [ERROR:asyncio] 2022-02-11 10:36:39,560 - unhandled exception during asyncio.run() shutdown task: <Task finished coro=<_run_app() done, defined at /usr/local/lib64/python3/site-packages/aiohttp/web.py:287> exception=RuntimeError('web.Application instance initialized with different loop')> Traceback (most recent call last): File "/usr/local/lib64/python3/site-packages/aiohttp/web.py", line 514, in run_app loop.run_until_complete(main_task) File "uvloop/loop.pyx", line 1501, in uvloop.loop.Loop.run_until_complete File "/usr/local/lib64/python3/site-packages/aiohttp/web.py", line 321, in _run_app await runner.setup() File "/usr/local/lib64/python3/site-packages/aiohttp/web_runner.py", line 279, in setup self._server = await self._make_server() File "/usr/local/lib64/python3/site-packages/aiohttp/web_runner.py", line 373, in _make_server self._app._set_loop(loop) File "/usr/local/lib64/python3/site-packages/aiohttp/web_app.py", line 224, in _set_loop "web.Application instance initialized with different loop" RuntimeError: web.Application instance initialized with different loop Traceback (most recent call last): File "/opt/certstream-server-python/run_server.py", line 3, in certstream.run() File "/opt/certstream-server-python/certstream/init.py", line 29, in run webserver.run_server() File "/opt/certstream-server-python/certstream/webserver.py", line 69, in run_server ,ssl_context=ssl_ctx File "/usr/local/lib64/python3/site-packages/aiohttp/web.py", line 514, in run_app loop.run_until_complete(main_task) File "uvloop/loop.pyx", line 1501, in uvloop.loop.Loop.run_until_complete File "/usr/local/lib64/python3/site-packages/aiohttp/web.py", line 321, in _run_app await runner.setup() File "/usr/local/lib64/python3/site-packages/aiohttp/web_runner.py", line 279, in setup self._server = await self._make_server() File "/usr/local/lib64/python3/site-packages/aiohttp/web_runner.py", line 373, in _make_server self._app._set_loop(loop) File "/usr/local/lib64/python3/site-packages/aiohttp/web_app.py", line 224, in _set_loop "web.Application instance initialized with different loop" RuntimeError: web.Application instance initialized with different loop [ERROR:asyncio] 2022-02-11 10:36:39,587 - Task was destroyed but it is pending! task: <Task pending coro=<WebServer.mux_ctl_stream() running at /opt/certstream-server-python/certstream/webserver.py:79>> [ERROR:asyncio] 2022-02-11 10:36:39,588 - Task was destroyed but it is pending! task: <Task pending coro=<WebServer.ws_heartbeats() running at /opt/certstream-server-python/certstream/webserver.py:199>> sys:1: RuntimeWarning: coroutine 'WebServer.mux_ctl_stream' was never awaited sys:1: RuntimeWarning: coroutine 'WebServer.ws_heartbeats' was never awaited sys:1: RuntimeWarning: coroutine 'TransparencyWatcher.watch_for_updates_task' was never awaited

Is there a chance of a successful launch of this version?

Fitblip commented 2 years ago

Hi, that project proved to be too inefficient for the volume of messages, which is why I chose to rewrite it in elixir which is pretty purpose built for this sort of job/workload.

If you or someone else wants to try to make it more performant or do some bugfixes I'll happily review some PRs but I'm not planning on doing much of anything with that repo.

d-Rickyy-b commented 2 years ago

Hi @Fitblip, not sure if this is the right way to communicate this, but I recently wrote a clone of certstream-server in Go. Maybe it's also helpful for others that don't want to run the server on elixir. Hope you appreciate it :). If not, just remove this message.

Fitblip commented 2 years ago

Ah nice this is super cool @d-Rickyy-b! I wonder if we should create a "Certstream" umbrella github organization? :)

d-Rickyy-b commented 2 years ago

Sounds nice. I'd be happy to contribute, although I am only quite fluent in go and python. Elixir feels like magic to me :D. If you want to have a chat somewhere else than GitHub, hit me up on Twitter or Telegram.

ld73 commented 2 years ago

Sounds nice. I'd be happy to contribute, although I am only quite fluent in go and python. Elixir feels like magic to me :D. If you want to have a chat somewhere else than GitHub, hit me up on Twitter or Telegram.

Rickky good afternoon! 17.10.2022 support for the M1 format on M3 has ended, will your project be updated taking into account the changes in the json format?

ld73 commented 2 years ago

Hi, that project proved to be too inefficient for the volume of messages, which is why I chose to rewrite it in elixir which is pretty purpose built for this sort of job/workload.

If you or someone else wants to try to make it more performant or do some bugfixes I'll happily review some PRs but I'm not planning on doing much of anything with that repo.

Thank you so much for the answer!