Open fselmo opened 1 month ago
You hit on the two that I'm most interested in! As a user, I don't want to have to manage process_subscriptions()
.
Not specific to the Persistent Provider itself, but as far as overall UX - and to tack onto the mention of .subscribe()
, do you think it would be worthwhile to add optional kwargs to the method instead of (or in addition to) relying on LogsSubscriptionArg
?
async def subscribe(
self,
subscription_type: SubscriptionType, # Meh
subscription_arg: Optional[
Union[
LogsSubscriptionArg, # logs, optional filter params
bool, # newPendingTransactions, full_transactions
]
] = None,
callback: Optional[Callable] = None, # Sneaking that in here
*,
address: Optional[
Union[
Address,
ChecksumAddress,
ENS,
Sequence[Union[Address, ChecksumAddress, ENS]]
],
]= None,
topics: Optional[Sequence[Union[HexStr, Sequence[HexStr]]]] = None,
) -> HexStr:
This will certainly require some additional sanity-checking, and may even require separate methods for subscriptions between newHeads and logs (et al; which wouldn't be terrible, as I don't necessarily love subscription_type
), but it may be more expressive to have a defined set of arguments, while still allowing people to use pre-packaged log structures.
For me as web3.py user, my expectation was to have some class abstraction such as SubscriptionManager
that will allow to:
asyncio.CancelledError
is thrown.subscribe()
method would provide an Optional callback function to be called when a message is received from within the given subscription.subscribe()
, maybe SubscriptionManager
might provide a general callback function in the __init__()
.I think it would be great if the logic and client can be separated, both to future-proof in case of any changes in the underlying libraries that is out of the control of the maintainers here the interface here can be maintained. It also would allow for better integration of both other async libraries (trio, curio, even gevent even though it's not technically async) but also different clients (httpx being the most prominent, also trio-websocket or any AnyIO based client). People have hacked together their own implementations of their stack (myself included), but what that turns into is that I end up using multiple web3 clients which is fine for personal use (even if it may involve transactions on the level of a small country's GDP in terms of values transacted) but is not ideal for people who perhaps aren't or haven't been writing code to interact with web3 nodes constantly and the abstraction is not merely shortcuts but maybe the primary way to learn how one interacts with web3 writ large programmatically.
This might be a slightly longer-term project and cover more than just PersistentConnectionProvider - although that is part of it for sure, but luckily libraries like AnyIO would help quite a bit, and there isn't really a deadline per se. I don't know how many people use other clients and implementations for most of the stack, but I suspect that httpx at least has a fair number of users, which felt unlikely as recent as the first year of COVID. I think web3 adoption is only going to increase and a library that is able to reduce friction for both maintainers/contributors and users in the long run would be helpful for a lot of people.
The newer
PersistentConnectionProvider
implementations, mostly theWebSocketProvider
, seem to be getting a lot of use and a lot of good feedback is coming back from users, whether through issues or Discord. It's time to start thinking about some good UX abstractions that can help facilitate subscription management, re-connection logic, and anything else that might make sense to bake into the library.Some ideas that have been brewing:
request_processor._request_information_cache
) and re-subscribing with new subscription ids when the re-connection is triggered.w3.eth.subscribe
method perhaps? Likely managed within aSubscriptionManager
class strapped to persistent connection providers.If you have UX improvements, pain points, or any comments on anything discussed in this issue, please feel free to join the discussion below!