search

Podcastindex-org / podping-hivewriter

The hive writer component of podping.
MIT License
15 stars 5 forks source link

Dependency issues on Windows with ZMQ #10

Open brianoflondon opened 3 years ago

brianoflondon commented 3 years ago

Trying to run from the source code on Windows gives the following dependency problems only in the ZMQ section.

This is not an issue for Mac or Ubuntu so far in my testing.

The application, however, will still send a single URL using the podping write https://urlhere.com 

2021-08-31T09:27:28+0300
2021-08-31T09:27:29+0300 | ERROR | Account @podping.win not authorised to send Podpings
2021-08-31T09:27:29+0300 | INFO | Podping startup sequence initiated, please stand by, full bozo checks in operation...
2021-08-31T09:27:29+0300 | INFO | Testing Account Resource Credits - before 100.00%
2021-08-31T09:27:30+0300 | INFO | Transaction sent: db7070dfdb9215859e3d062b94e50be072bd07c9 - JSON size: 203
2021-08-31T09:27:30+0300 | INFO | Testing Account Resource Credits.... 5s
2021-08-31T09:27:45+0300 | INFO | Testing Account Resource Credits - after 99.64%
2021-08-31T09:27:45+0300 | INFO | Capacity for further podpings : 278.7
2021-08-31T09:27:45+0300 | INFO | Transaction sent: efd4cae0409297e921c22d99a8a85f56482d93ce - JSON size: 231
2021-08-31T09:27:45+0300 | INFO | Startup of Podping status: SUCCESS! Hit the BOOST Button.
2021-08-31T09:27:45+0300 | INFO | Hive account: @podping.win
2021-08-31T09:27:45+0300 | INFO | Running ZeroMQ server on 127.0.0.1:9999
2021-08-31T09:27:45+0300 | ERROR | Proactor event loop does not implement add_reader family of methods required for zmq. zmq will work with proactor if tornado >= 6.1 can be found. Use `asyncio.set_event_loop_policy(WindowsSelectorEventLoopPolicy())` or install 'tornado>=6.1' to avoid this error. occurred
Traceback (most recent call last):
File "C:\Users\graem\AppData\Local\pypoetry\Cache\virtualenvs\podping-hivewriter-bFUsWBAJ-py3.9\lib\site-packages\asgiref\sync.py", line 482, in thread_handler
raise exc_info[1]
File "C:\Users\graem\AppData\Local\pypoetry\Cache\virtualenvs\podping-hivewriter-bFUsWBAJ-py3.9\lib\site-packages\asgiref\sync.py", line 482, in thread_handler
raise exc_info[1]
File "C:\Users\graem\OneDrive\win-code\podping-hivewriter\src\podping_hivewriter\cli\podping.py", line 192, in server
loop = asyncio.get_running_loop()
RuntimeError: no running event loop
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\graem\AppData\Local\pypoetry\Cache\virtualenvs\podping-hivewriter-bFUsWBAJ-py3.9\lib\site-packages\zmq\asyncio.py", line 49, in _get_selector_windows
from tornado.platform.asyncio import AddThreadSelectorEventLoop
ModuleNotFoundError: No module named 'tornado'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\graem\OneDrive\win-code\podping-hivewriter\src\podping_hivewriter\podping_hivewriter.py", line 318, in _zmq_response_loop
iri: str = await socket.recv_string()
File "C:\Users\graem\AppData\Local\pypoetry\Cache\virtualenvs\podping-hivewriter-bFUsWBAJ-py3.9\lib\site-packages\zmq\sugar\socket.py", line 736, in recv_string
msg = self.recv(flags=flags)
File "C:\Users\graem\AppData\Local\pypoetry\Cache\virtualenvs\podping-hivewriter-bFUsWBAJ-py3.9\lib\site-packages\zmq_future.py", line 238, in recv
return self._add_recv_event('recv', dict(flags=flags, copy=copy, track=track))
File "C:\Users\graem\AppData\Local\pypoetry\Cache\virtualenvs\podping-hivewriter-bFUsWBAJ-py3.9\lib\site-packages\zmq_future.py", line 405, in _add_recv_event
self._add_io_state(POLLIN)
File "C:\Users\graem\AppData\Local\pypoetry\Cache\virtualenvs\podping-hivewriter-bFUsWBAJ-py3.9\lib\site-packages\zmq_future.py", line 575, in _add_io_state    self._update_handler(self._state)
File "C:\Users\graem\AppData\Local\pypoetry\Cache\virtualenvs\podping-hivewriter-bFUsWBAJ-py3.9\lib\site-packages\zmq_future.py", line 591, in _update_handler
self._get_loop()
File "C:\Users\graem\AppData\Local\pypoetry\Cache\virtualenvs\podping-hivewriter-bFUsWBAJ-py3.9\lib\site-packages\zmq_future.py", line 38, in _get_loop
self._init_io_state(self._current_loop)
File "C:\Users\graem\AppData\Local\pypoetry\Cache\virtualenvs\podping-hivewriter-bFUsWBAJ-py3.9\lib\site-packages\zmq\asyncio.py", line 137, in _init_io_state
self._get_selector(io_loop).add_reader(
File "C:\Users\graem\AppData\Local\pypoetry\Cache\virtualenvs\podping-hivewriter-bFUsWBAJ-py3.9\lib\site-packages\zmq\asyncio.py", line 133, in _get_selector    return _get_selector(io_loop)
File "C:\Users\graem\AppData\Local\pypoetry\Cache\virtualenvs\podping-hivewriter-bFUsWBAJ-py3.9\lib\site-packages\zmq\asyncio.py", line 51, in _get_selector_windows
raise RuntimeError(
RuntimeError: Proactor event loop does not implement add_reader family of methods required for zmq. zmq will work with proactor if tornado >= 6.1 can be found. Use `asyncio.set_event_loop_policy(WindowsSelectorEventLoopPolicy())` or install 'tornado>=6.1' to avoid this error.
2021-08-31T09:27:45+0300 | INFO | Status - Hive Node: <Hive node=https://api.deathwing.me, nobroadcast=False> - Uptime: 0:00:16.987637 - IRIs Received: 0 - IRIs Deduped: 0 - IRIs Sent: 0
agates commented 3 years ago

What happens if you run it after running `poetry install tornado` from the windows environment?

brianoflondon commented 3 years ago

I had to do poetry add tornado then I get this warning. 

2021-08-31T14:46:43+0300 | INFO | podping 1.0.0a0 starting up in server mode
2021-08-31T14:46:44+0300 | INFO | Podping startup sequence initiated, please stand by, full bozo checks in operation...
2021-08-31T14:46:44+0300 | INFO | Testing Account Resource Credits - before 99.05%
2021-08-31T14:46:44+0300 | INFO | Transaction sent: 50d34e24fca1d7113c91dbeb091919e1abc2a3b7 - JSON size: 203
2021-08-31T14:46:44+0300 | INFO | Testing Account Resource Credits.... 5s
2021-08-31T14:47:00+0300 | INFO | Testing Account Resource Credits - after 98.69%
2021-08-31T14:47:00+0300 | INFO | Capacity for further podpings : 276.2
2021-08-31T14:47:00+0300 | INFO | Transaction sent: 6cf26892be28d5e10e795fa5f56aa785a6bd69df - JSON size: 231
2021-08-31T14:47:00+0300 | INFO | Startup of Podping status: SUCCESS! Hit the BOOST Button.
2021-08-31T14:47:00+0300 | INFO | Hive account: @podping.win
2021-08-31T14:47:00+0300 | INFO | Running ZeroMQ server on 127.0.0.1:9999
C:\Users\graem\AppData\Local\pypoetry\Cache\virtualenvs\podping-hivewriter-bFUsWBAJ-py3.9\lib\site-packages\zmq\_future.py:591: RuntimeWarning: Proactor event loop does not implement add_reader family of methods required for zmq. Registering an additional selector thread for add_reader support via tornado. Use `asyncio.set_event_loop_policy(WindowsSelectorEventLoopPolicy())` to avoid this warning.
  self._get_loop()
2021-08-31T14:47:01+0300 | INFO | Status - Hive Node: <Hive node=https://api.deathwing.me, nobroadcast=False> - Uptime: 0:00:17.766454 - IRIs Received: 0 - IRIs Deduped: 0 - IRIs Sent: 0
brianoflondon commented 3 years ago

Tests now working on windows!

============================= test session starts =============================
platform win32 -- Python 3.9.7, pytest-6.2.4, py-1.10.0, pluggy-0.13.1
rootdir: c:\Users\graem\OneDrive\win-code\podping-hivewriter, configfile: pyproject.toml, testpaths: tests
plugins: asyncio-0.15.1, cov-2.12.1, timeout-1.4.2
collected 7 items

tests\integration\test_get_allowed_accounts.py .                         [ 14%]
tests\integration\test_update-podping-settings.py ..                     [ 42%]
tests\integration\test_write_cli_multiple.py .                           [ 57%]
tests\integration\test_write_cli_single.py .                             [ 71%]
tests\integration\test_write_zmq_multiple.py .                           [ 85%]
tests\integration\test_write_zmq_single.py .                             [100%]

============================== warnings summary ===============================
tests/integration/test_write_zmq_multiple.py::test_write_zmq_multiple_url
tests/integration/test_write_zmq_single.py::test_write_zmq_single_url
  C:\Users\graem\AppData\Local\pypoetry\Cache\virtualenvs\podping-hivewriter-bFUsWBAJ-py3.9\lib\site-packages\zmq\_future.py:591: RuntimeWarning: Proactor event loop does not implement add_reader family of methods required for zmq. Registering an additional selector thread for add_reader support via tornado. Use `asyncio.set_event_loop_policy(WindowsSelectorEventLoopPolicy())` to avoid this warning.
    self._get_loop()

tests/integration/test_write_zmq_multiple.py::test_write_zmq_multiple_url
  C:\Users\graem\OneDrive\win-code\podping-hivewriter\tests\integration\test_write_zmq_multiple.py:81: DeprecationWarning: Socket(io_loop) argument is deprecated in pyzmq 22.2. The currently active loop will always be used.
    socket = context.socket(zmq.REQ, io_loop=event_loop)

tests/integration/test_write_zmq_single.py::test_write_zmq_single_url
  C:\Users\graem\OneDrive\win-code\podping-hivewriter\tests\integration\test_write_zmq_single.py:73: DeprecationWarning: Socket(io_loop) argument is deprecated in pyzmq 22.2. The currently active loop will always be used.
    socket = context.socket(zmq.REQ, io_loop=event_loop)

-- Docs: https://docs.pytest.org/en/stable/warnings.html
- generated xml file: C:\Users\graem\AppData\Local\Temp\tmp-15084OdL3dL1hDWJv.xml -
================== 7 passed, 4 warnings in 352.98s (0:05:52) ==================
Task was destroyed but it is pending!
task: <Task pending name='Task-25' coro=<HiveWrapper._rotate_nodes_loop() running at C:\Users\graem\OneDrive\win-code\podping-hivewriter\src\podping_hivewriter\hive_wrapper.py:69> wait_for=<Future cancelled>>
Task was destroyed but it is pending!
task: <Task pending name='Task-27' coro=<PodpingHivewriter._iri_batch_loop() running at C:\Users\graem\OneDrive\win-code\podping-hivewriter\src\podping_hivewriter\podping_hivewriter.py:264> wait_for=<Future cancelled>>
Task was destroyed but it is pending!
task: <Task pending name='Task-73' coro=<PodpingHivewriter._iri_batch_loop.<locals>.get_from_queue() running at C:\Users\graem\OneDrive\win-code\podping-hivewriter\src\podping_hivewriter\podping_hivewriter.py:243> wait_for=<Future cancelled>>
Task was destroyed but it is pending!
task: <Task pending name='Task-81' coro=<PodpingHivewriter._iri_batch_loop() running at C:\Users\graem\OneDrive\win-code\podping-hivewriter\src\podping_hivewriter\podping_hivewriter.py:264> wait_for=<Future cancelled>>
Task was destroyed but it is pending!
task: <Task pending name='Task-110' coro=<PodpingHivewriter._iri_batch_loop.<locals>.get_from_queue() running at C:\Users\graem\OneDrive\win-code\podping-hivewriter\src\podping_hivewriter\podping_hivewriter.py:243> wait_for=<Future cancelled>>
Task was destroyed but it is pending!
task: <Task pending name='Task-79' coro=<HiveWrapper._rotate_nodes_loop() running at C:\Users\graem\OneDrive\win-code\podping-hivewriter\src\podping_hivewriter\hive_wrapper.py:69> wait_for=<Future cancelled>>
agates commented 3 years ago

This is strange. You shouldn't need tornado. Wonder if it's a pyzmq bug. Will look into it.

brianoflondon commented 3 years ago

It only failed on Windows. Who does dev or hosting on Windoze anyway?

agates commented 3 years ago

The CLI on windows is a legitimate use case for the average joe podcaster who just wants to send an update when they publish a new episode.

brianoflondon commented 3 years ago

I know... The answer "get a real computer" isn't too helpful.

I also tried to create an exe but I can see that is a bit more of an art than I'm ready to tackle.

It does run on windows once trafalgar is there. I have no idea what that is I didn't even look.

agates commented 3 years ago

Brian, could you test the CLI install on Windows again with v1.0.0-beta.0? The server should at least fail to run until you attempt to install with the extra flag (pip install podping-hivewriter[server]).

I figured adding some mitigating factors at least helps, and feel we should push people toward docker/WSL if they really want to run the server. But I'm not against hunting this down if there is interest.

brianoflondon commented 3 years ago

I'll give it a try later today!

brianoflondon commented 3 years ago 
PS C:\Users\graem\AppData\Roaming\Python\Python39\Scripts> .\podping server
Hive account: Aborted!

It works! no errors apart from needing to add Scripts to my windows path.

All I did was run the pip install podping-hivewriter[server]

brianoflondon commented 3 years ago

Spoke too soon.... Beem error after putting in the pasword.

Hive account: brianoflondon
Hive posting key:
Repeat for confirmation:
2021-09-07T13:50:40+0300 | INFO | podping 1.0.0b0 starting up in server mode
2021-09-07T13:50:41+0300 | ERROR | Task exception was never retrieved
future: <Task finished name='Task-2' coro=<HiveWrapper._startup() done, defined at C:\Users\graem\AppData\Roaming\Python\Python39\site-packages\podping_hivewriter\hive_wrapper.py:37> exception=ValueError('Error loading Base58 object')>
Traceback (most recent call last):
  File "C:\Users\graem\AppData\Roaming\Python\Python39\site-packages\podping_hivewriter\cli\podping.py", line 194, in server
    loop = asyncio.get_running_loop()
RuntimeError: no running event loop

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\graem\AppData\Roaming\Python\Python39\site-packages\podping_hivewriter\hive_wrapper.py", line 42, in _startup
    self._hive: beem.Hive = get_hive(
  File "C:\Users\graem\AppData\Roaming\Python\Python39\site-packages\podping_hivewriter\hive.py", line 17, in get_hive
    hive = beem.Hive(node=nodes, keys=posting_keys, nobroadcast=nobroadcast)
  File "C:\Users\graem\AppData\Roaming\Python\Python39\site-packages\beem\blockchaininstance.py", line 203, in __init__
    self.wallet = Wallet(blockchain_instance=self, **kwargs)
  File "C:\Users\graem\AppData\Roaming\Python\Python39\site-packages\beem\wallet.py", line 99, in __init__
    self.setKeys(kwargs["keys"])
  File "C:\Users\graem\AppData\Roaming\Python\Python39\site-packages\beem\wallet.py", line 135, in setKeys
    pub = self.publickey_from_wif(wif)
  File "C:\Users\graem\AppData\Roaming\Python\Python39\site-packages\beem\wallet.py", line 209, in publickey_from_wif
    return str(self.privatekey(str(wif)).pubkey)
  File "C:\Users\graem\AppData\Roaming\Python\Python39\site-packages\beem\wallet.py", line 206, in privatekey
    return PrivateKey(key, prefix=self.prefix)
  File "C:\Users\graem\AppData\Roaming\Python\Python39\site-packages\beemgraphenebase\account.py", line 736, in __init__
    self._wif = Base58(wif)
  File "C:\Users\graem\AppData\Roaming\Python\Python39\site-packages\beemgraphenebase\base58.py", line 49, in __init__
    raise ValueError("Error loading Base58 object")
ValueError: Error loading Base58 object
2021-09-07T13:50:42+0300 | INFO | Podping startup sequence initiated, please stand by, full bozo checks in operation...
2021-09-07T13:50:42+0300 | INFO | Testing Account Resource Credits - before 100.00%

However if one gives the password on the command line it works but fails again with zmq error later:

PS C:\Users\graem> podping --hive-account podping.win --hive-posting-key xxxxxxx server
2021-09-07T13:55:21+0300 | INFO | podping 1.0.0b0 starting up in server mode
2021-09-07T13:55:22+0300 | INFO | Podping startup sequence initiated, please stand by, full bozo checks in operation...
2021-09-07T13:55:22+0300 | INFO | Testing Account Resource Credits - before 100.00%
2021-09-07T13:55:22+0300 | INFO | Transaction sent: 36086acd17688444f6a2ed3c241f5bad305a9a20 - JSON size: 203
2021-09-07T13:55:22+0300 | INFO | Testing Account Resource Credits.... 5s
2021-09-07T13:55:37+0300 | INFO | Testing Account Resource Credits - after 99.65%
2021-09-07T13:55:37+0300 | INFO | Capacity for further podpings : 285.3
2021-09-07T13:55:38+0300 | INFO | Transaction sent: ae912c9fbbd3df006b1d823c9f0ee5000827d204 - JSON size: 231
2021-09-07T13:55:38+0300 | INFO | Startup of Podping status: SUCCESS! Hit the BOOST Button.
2021-09-07T13:55:38+0300 | INFO | Hive account: @podping.win
2021-09-07T13:55:38+0300 | INFO | Running ZeroMQ server on 127.0.0.1:9999
2021-09-07T13:55:38+0300 | ERROR | Proactor event loop does not implement add_reader family of methods required for zmq. zmq will work with proactor if tornado >= 6.1 can be found. Use `asyncio.set_event_loop_policy(WindowsSelectorEventLoopPolicy())` or install 'tornado>=6.1' to avoid this error. occurred
Traceback (most recent call last):
  File "C:\Users\graem\AppData\Roaming\Python\Python39\site-packages\asgiref\sync.py", line 482, in thread_handler
    raise exc_info[1]
  File "C:\Users\graem\AppData\Roaming\Python\Python39\site-packages\asgiref\sync.py", line 482, in thread_handler
    raise exc_info[1]
  File "C:\Users\graem\AppData\Roaming\Python\Python39\site-packages\podping_hivewriter\cli\podping.py", line 194, in server
    loop = asyncio.get_running_loop()
RuntimeError: no running event loop

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\graem\AppData\Roaming\Python\Python39\site-packages\zmq\asyncio.py", line 49, in _get_selector_windows
    from tornado.platform.asyncio import AddThreadSelectorEventLoop
ModuleNotFoundError: No module named 'tornado'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\graem\AppData\Roaming\Python\Python39\site-packages\podping_hivewriter\podping_hivewriter.py", line 318, in _zmq_response_loop
    iri: str = await socket.recv_string()
  File "C:\Users\graem\AppData\Roaming\Python\Python39\site-packages\zmq\sugar\socket.py", line 736, in recv_string
    msg = self.recv(flags=flags)
  File "C:\Users\graem\AppData\Roaming\Python\Python39\site-packages\zmq\_future.py", line 238, in recv
    return self._add_recv_event('recv', dict(flags=flags, copy=copy, track=track))
  File "C:\Users\graem\AppData\Roaming\Python\Python39\site-packages\zmq\_future.py", line 405, in _add_recv_event
    self._add_io_state(POLLIN)
  File "C:\Users\graem\AppData\Roaming\Python\Python39\site-packages\zmq\_future.py", line 575, in _add_io_state
    self._update_handler(self._state)
  File "C:\Users\graem\AppData\Roaming\Python\Python39\site-packages\zmq\_future.py", line 591, in _update_handler
    self._get_loop()
  File "C:\Users\graem\AppData\Roaming\Python\Python39\site-packages\zmq\_future.py", line 38, in _get_loop
    self._init_io_state(self._current_loop)
  File "C:\Users\graem\AppData\Roaming\Python\Python39\site-packages\zmq\asyncio.py", line 137, in _init_io_state
    self._get_selector(io_loop).add_reader(
  File "C:\Users\graem\AppData\Roaming\Python\Python39\site-packages\zmq\asyncio.py", line 133, in _get_selector
    return _get_selector(io_loop)
  File "C:\Users\graem\AppData\Roaming\Python\Python39\site-packages\zmq\asyncio.py", line 51, in _get_selector_windows
    raise RuntimeError(
RuntimeError: Proactor event loop does not implement add_reader family of methods required for zmq. zmq will work with proactor if tornado >= 6.1 can be found. Use `asyncio.set_event_loop_policy(WindowsSelectorEventLoopPolicy())` or install 'tornado>=6.1' to avoid this error.
2021-09-07T13:55:38+0300 | INFO | Status - Hive Node: <Hive node=https://api.deathwing.me, nobroadcast=False> - Uptime: 0