search

paritytech / substrate

Substrate: The platform for blockchain innovators
Apache License 2.0
8.39k stars 2.65k forks source link

storage keys paged and query storage at not implemented in light client #6884

Open dvc94ch opened 4 years ago

dvc94ch commented 4 years ago

I can emulate query storage at by fetching each key individually, but I don't see how I can substitute storage keys paged.

kianenigma commented 4 years ago

I was informed in https://github.com/paritytech/substrate/pull/6847 through reviews that it is intentional to not implement some of the heavy ones for the light clients. Not sure if it is the case here, just pointing out.

dvc94ch commented 4 years ago

Well, we can't use the light client in this case, which is a shame

bkchr commented 4 years ago

Hmm, the question is why do you not send the request directly to the full node?

Basically we could forward any request, but this increases the latency of the request.

dvc94ch commented 4 years ago

Well we're using the light client so all our requests are serialized to a string and passed to the light client task. We don't do any networking, so forwarding the request is fine. I don't see why we'd have more latency.

bkchr commented 4 years ago

Because you first do a RPC request to the light client which than forwards the request to a full node. While when directly connecting to a full node, you only need to send the request to one peer (less latency).

However, I'm not entirely sure what you strategy is for light clients and RPC requests.

@tomusdrw @svyatonik can someone of you shed some light on this? Do we want to support for RPC requests in the future for light clients?

dvc94ch commented 4 years ago

Just to explain why I opened this issue and what my expectations were:

When using the light client the expectation is that I do not need to trust the full node as the light client verifies the responses, so just forwarding the requests to a full node is not really what I'm after.

storage keys paged

hopefully the light client can make sure that the keys are produced in a deterministic order, making sure that the full node can not selectively ignore keys or reorder them and that the returned keys are part of the storage root.

query storage at

should make sure that the right value for the key and that the full node can't just return garbage for example

tomusdrw commented 4 years ago

I don't think we have a way to generate efficient proofs for a range of storage keys, so the RPC would essentially end up the same as querying keys one-by-one. I assume that RPCs will be supported selectively (perhaps we should document it properly somewhere?) and the decisions will be driven by:

I think up until when we come up with some micro-payment (or at least ranking) scheme to avoid light client leeching on full nodes we should be very careful with implementing particular RPCs for light nodes.

svyatonik commented 4 years ago

I'd add that "query storage" part may be implemented (quite) efficiently using changes tries. But since they're not enabled (yet?), then the only possible impl would be to query storage one-by-one.

dvc94ch commented 4 years ago

Since we are hoping to use the substrate networking stack as a core component of our client https://github.com/paritytech/substrate/pull/7133, it no longer makes sense for us to support rpc requests to a full node at all anymore. Is there a way I can help move this issue forward?