[350 SigmaUSD] Node discovery via node's REST API

[greenhat]

Implementation details

Take a list of nodes (seeds) and ask them for their known peers via /peers/all REST API endpoint. Repeat the same operation (ask for their known peers) with received nodes skipping the ones we've already asked, keeping a limit of simultaneous requests. Test if the REST API is available on the fetched nodes via /info endpoint on port 9053. Filter out nodes that do not have REST API(connection refused, timeout, etc.) Stop when there are no nodes left to ask or a given timeout is reached. Return the list of discovered nodes without the seeds.

Rust API

async fn peer_discovery(seeds: NonEmptyVec<Url>, max_parallel_requests: BoundedU16<1, u16:MAX>, 
    timeout: Duration) -> Result<Vec<Url>, PeerDiscoveryError>

Introduce type NonEmptyVec<T> = BoundedVec<T, 1, usize::MAX>; in bounded-vec crate. Testing should be done by running a peer discovery on the public nodes list(seeds) shipped with the node - https://github.com/ergoplatform/ergo/blob/f38f3a6d0ab8849871adc54010846c5afb101ff4/src/main/resources/mainnet.conf#L49-L64 using port 9053 (default for REST API).

Wasm API

async fn peer_discovery(seeds: Box<[web_sys::Url]>, max_parallel_reqs: u16,
    timeout_sec: u32) -> Result<Box<[web_sys::Url]>, JsValue>

Swift API

func peerDiscovery(seeds: [URL], maxParallelReqs:UInt16, timeoutSec: UInt32, 
    closure: @escaping (Result<[URL], Error>) -> Void) throws -> RequestHandle

The underlying C FFI layer should use string for URL representation, passing the array of strings as a pointer and a count.

/nipopow/proof REST API endpoint support

Implement ergo_rest::api::node::get_nipopow_proof_by_header_id using /nipopow/proof/{m}/{k}/{headerId} REST API endpoint.

Add it to the Wasm and C/Swift API. See existing node::get_info code for examples.

[kettlebell]

Going to work on this.

[kettlebell]

I've got working code that scans over the peers, and it looks like only 2 seeds have an open REST API: "213.239.193.208:9030" and "159.65.11.55:9030". Not a single non-seed peer is responsive on port 9053. I've manually checked things with curl. Is this normal?

[greenhat]

I've got working code that scans over the peers, and it looks like only 2 seeds have an open REST API: "213.239.193.208:9030" and "159.65.11.55:9030". Not a single non-seed peer is responsive on port 9053. I've manually checked things with curl. Is this normal?

Well, that does not sound good. Are you sure about port 9030, cause I used "213.239.193.208:9053" in tests. There is a project I saw in our discord that gathers some stats about the network and I asked them to count the nodes with open REST API and I believe the number they gave me was 17 nodes. I wonder how did they found them.

[MrStahlfelge]

I am working on the same problem and opened an improvement suggestion for the node: ergoplatform/ergo#1652

Please check it and state your remarks.

[kettlebell]

Well, that does not sound good. Are you sure about port 9030, cause I used "213.239.193.208:9053" in tests.

I made sure to use port 9053 for all the requests. I copied pasted those IP addresses from the seed list.

[kettlebell]

I searched around Discord and found this thread: https://discord.com/channels/668903786361651200/668930902751182913/920043012384497696

It sounds like they used the handshake protocol to find the nodes. Also @greenhat, is https://ergo.nodespyder.io/ the project you saw on discord?

[greenhat]

I searched around Discord and found this thread: https://discord.com/channels/668903786361651200/668930902751182913/920043012384497696

Here is that answer I mention above - https://discord.com/channels/668903786361651200/669989266478202917/925702327103553566 It's from the same andee you found.

It sounds like they used the handshake protocol to find the nodes.

That's plausible. I mean, I'm pretty sure they did P2P peer discovery and then filtered nodes with open REST API.

Also @greenhat, is https://ergo.nodespyder.io/ the project you saw on discord?

Yes, I asked them originally, but I'm not recalling I got the answer.

[kettlebell]

@greenhat just want to discuss the issue with reqwest on WASM not having timeouts as mentioned here: https://github.com/ergoplatform/sigma-rust/blob/d2fd2bd0884eda81f167dc60feb33132e9d96f96/ergo-rest/src/api/node.rs#L27-L32

There is a PR on the reqwest repo to implement timeout that hasn't been looked for almost a year: https://github.com/seanmonstar/reqwest/pull/1274. I'm no expert, but it looks to me that reqwest on WASM is mostly a wrapper around web APIs and the PR just linked pretty much makes sense to even me.

Now peer-discovery wouldn't work at all without accounting for timeouts. What I think we could do is create a reqwest crate in sigma-rust which passes through (hyper-based) reqwest for non-WASM32, and vendor all the WASM-specific code together with the timeout-PR applied on top (it's not a massive amount of code: https://github.com/seanmonstar/reqwest/tree/master/src/wasm). It's not pretty, but what do you think?

[greenhat]

@greenhat just want to discuss the issue with reqwest on WASM not having timeouts as mentioned here:

https://github.com/ergoplatform/sigma-rust/blob/d2fd2bd0884eda81f167dc60feb33132e9d96f96/ergo-rest/src/api/node.rs#L27-L32

There is a PR on the reqwest repo to implement timeout that hasn't been looked for almost a year: seanmonstar/reqwest#1274. I'm no expert, but it looks to me that reqwest on WASM is mostly a wrapper around web APIs and the PR just linked pretty much makes sense to even me.

I came to the same conclusion. I saw this PR and had hoped that it'll be merged sooner rather than later.

Now peer-discovery wouldn't work at all without accounting for timeouts. What I think we could do is create a reqwest crate in sigma-rust which passes through (hyper-based) reqwest for non-WASM32, and vendor all the WASM-specific code together with the timeout-PR applied on top (it's not a massive amount of code: https://github.com/seanmonstar/reqwest/tree/master/src/wasm). It's not pretty, but what do you think?

Whoa! I missed that timeout support is vital for peer-discovery. I totally agree with you, and I appreciate you're taking care of this.

[kettlebell]

I've implemented everything except for the swift binding of peerDiscovery. I'm running into an issue though. It seems that the swift array [URL] can't be made to conform to the FromRawPtr protocol (which is needed for wrapClosure: https://github.com/ergoplatform/sigma-rust/blob/ec018ab808bcad6f87c02a677a9bfec0d8d6d39a/bindings/ergo-lib-ios/Sources/ErgoLib/NodeInfo.swift#L28-L30

because I can't find any way to initialize such an array from an UnsafeRawPointer. Any ideas on a fix?

[greenhat]

I've implemented everything except for the swift binding of peerDiscovery. I'm running into an issue though. It seems that the swift array [URL] can't be made to conform to the FromRawPtr protocol (which is needed for wrapClosure:

https://github.com/ergoplatform/sigma-rust/blob/ec018ab808bcad6f87c02a677a9bfec0d8d6d39a/bindings/ergo-lib-ios/Sources/ErgoLib/NodeInfo.swift#L28-L30

because I can't find any way to initialize such an array from an UnsafeRawPointer. Any ideas on a fix?

Great job! Unfortunately, I cannot find a way to to pass [URL] array with the current wrapClosure implementation. Maybe a custom wrapClosureVecUrl(_ closure: @escaping (Result<[URL], Error>) -> Void) -> CompletionCallback would work. As a fallback plan I suggest a "traditional" wrapper type to hold an array (like DataInputs).

[greenhat]

The bounty is sent. Thank you!