chazkii / chromewhip

Scriptable Google Chrome™ as a HTTP service + asyncio driver
MIT License
118 stars 16 forks source link

No way to generate multiple events produced by a ws #10

Closed Tylerlhess closed 6 years ago

Tylerlhess commented 6 years ago

I may be missing the documentation or code section on this but there seemingly is no way to open a new tab and receive WS events indefinitely.

You can navigate to a page and await an individual response. You can navigate to a page and await some arbitrary event and then pull out the events that happened during that time. You seemingly can not setup a generator that will return network events or wait for the next event then return it.

Use case: I am scraping a streaming site for network events sent to an overlay. These are sent json encoded to a websocket opened on page load. I need to consume these events in realtime without having to reload the page after each event. I can do this with PyChromeDevTools but they lack multi-tab support.

Optimally, I would like a way to create an async generator that i could yield from the events or be able to yield from network events from all tabs while being able to add or remove tabs as different streams come and go.

Tylerlhess commented 6 years ago

I got this working but had to create an event loop and then put tasks into the loop for each tab that awaited the ws.recv() then handle the messages received that way. Thought this could likely be wrapped for future use to allow just awaiting the ws automatically.