litentry / litentry-parachain

The Litentry parachain
https://docs.litentry.com
GNU General Public License v3.0
62 stars 20 forks source link

Parallelise the http requests when building assertions #1468

Closed Kailai-Wang closed 1 year ago

Kailai-Wang commented 1 year ago

Context

We are currently doing it sequentially. Imagine the user linked 20 identities and requested a VC while we have a very bad network - this could easily be a problem and block the following requests.

A simple solution is to send the identities in a vector all in one go. Should we go for it? Please bear in mind that we need to "measure before optimise" to make sure we don't optimise things prematurely: we should be able to answer the questions: how long does it take to generate a VC with 20+ identities? What about 50 identities? What if under a throttled network?


:heavy_check_mark: Please set appropriate labels and assignees if applicable.

jingleizhang commented 1 year ago

ToDo:

[ ] change the single query to multi query each time [ ] need to check with TDF about their maximum vector length for one input parameter

outofxxx commented 1 year ago

I wrote some test cases about create_identity and request_vc, and the problems I found so far are as follows:

  1. Create 3/5/10/20/30 identities, based on these identities to request_vc A4, it will cause graphql panic, as follows:

[0m[2023-03-22T02:36:01Z ERROR lc_assertion_build::a4] [BuildAssertion] A4, Request, RequestError("HttpReqError(IO(Os { code: 11, kind: WouldBlock, message: \"Resource deadlock avoided\" }))")

  1. Found a todo!() problem of SubstrateNetwork, which will cause system panic, this question has been told to @BillyWooo

thread '<unnamed>' panicked at 'not yet implemented', /home/ztgx/codespace/litentry/litentry-parachain/tee-worker/litentry/core/data-providers/src/graphql.rs:68:42 fatal runtime error: failed to initiate panic, error 5

Test cases are here.

jingleizhang commented 1 year ago

https://github.com/litentry/litentry-parachain/blob/c133592fd5d78c875863f14620887df17361c816/tee-worker/litentry/core/data-providers/src/graphql.rs#L62-L69

The Rococo network should be added.

Kailai-Wang commented 1 year ago

https://github.com/litentry/litentry-parachain/blob/c133592fd5d78c875863f14620887df17361c816/tee-worker/litentry/core/data-providers/src/graphql.rs#L62-L69

The Rococo network should be added.

Thanks! I'm adding this in my latest PR already - but we still need support from TDF colleagues. See #1469

outofxxx commented 1 year ago

The addresses has currently no limits.