Closed fryorcraken closed 1 year ago
Issue encountered: a wasm function cannot load a wasm payload of more than 4.0KB. Worked around by loading the payload in JS and passing it to the wasm API.
The PoC is local JS code in one file. Next steps:
It was discussed whether a browser can handle 60-100MB in memory. From testing, this is possible.
However, downloading 60MB/100MB would be a UX issue:
Hence, we may need to have strategies in place such as:
Do note that for now the Merkle tree is few kB so it's not an issue just yet.
It was discussed that the size of the tree comes to the fact that we need to be able to delete members.
Another note is that to download the Merkle tree from the smart contract, the stack involves several components:
We may face issue with any of this step due to data size limits or timeout due to the length of time taken to download the data.
@richard-ramos did some build experiments with zerokit (go-waku currently uses kilic/rln) and the static library results are 100s of MB large: https://github.com/vacp2p/zerokit/issues/37.
This is not ideal for mobile environment (large app) and can be blocker for Waku React Native (uses go-waku) as there are timeout and size limits to upload packages to npmjs.com (that's how waku react native is currently distributed).
@staheri14 mentioned that there are two versions of the zerokit library in the repo, with one being lighter.
@staheri14 mentioned that there are two versions of the zerokit library in the repo, with one being lighter.
Not sure about this tbh, my point was about the tree size, which may/may not have an impact on the lib size, @s1fr0 can confirm better.
Related to the tree size, fyi, we have published a forum post covering some initial analysis of the storage overhead of the membership Merkle tree https://forum.vac.dev/t/waku-rln-relay-evaluating-storage-overhead-of-membership-merkle-tree/151 Feel free to comment and share concerns
compiling zerokit in wasm
Why are we doing this? Seems like a wasted effort for no reason. Why are we not using the existing JS implementation of RLN that works? cc @staheri14
:( I did not know about the existence of https://github.com/njofce/zk-chat/blob/main/zk-chat-client-lib/
In regards of using zerokit (wasm) vs the JS library used by zk-chat
In summary, we don't know if it's possible to reach an ideal library size for the web, and how effort would be involved (e.g. Forking dependencies, upstream feature flags, more library split, etc).
But, zk-kit does not provide the circom circuit, so we would still need to pass ours or find theirs. The developers using zk-kit are supposed to provide the
Using zerokit does involve some risk and unknown unknowns around being able to reduce library/circuit size to an idea web app size.
However, the risk still partially exists with zk-kit.
Added maintainability risks on using a 3rd's party library makes zerokit a safer choice.
Are there other libraries available?
Other project: https://github.com/Rate-Limiting-Nullifier/zk-keeper
uses the same library than zk-chat actually:
"@zk-kit/identity": "^1.4.1",
"@zk-kit/protocols": "^1.8.2",
"circomlibjs": "^0.0.8",
Not sure why they would need circomlibjs
. Circuit generation on the fly? Maybe something to check out @richard-ramos as circom circuit size is one of the challenges.
Edit: circomlibjs
is actually 9MB so not really an option for a webapp (makes sense in an extension).
The other interesting part is whether the extension is generic enough so it could be use for RLN.
@fryorcraken Thanks for the breakdown from your POV! This is exactly the kind of discussion we should've had at the beginning of this work, but glad we are doing it properly now! Would like to hear @richard-ramos and @staheri14 thoughts on this too.
But, zk-kit does not provide the circom circuit, so we would still need to pass ours or find theirs.
We are using the same and they are here: https://github.com/privacy-scaling-explorations/rln
Uncertainty if there are differences with zerokit and how would we manage them
My intuition is that they are not that big (especially compared to initial WASM blocker, which might be addressed now?), but I could be wrong and to give a qualified answer you'd have to look at RLN Relay API, MT algo etc and do a more thorough diff, something I think @richard-ramos @staheri14 @staheri14 are able to do.
The risk around circom size remain. Also, if their circom circuit is smaller then can we replicate the technique to make it smaller on our circuit?
That's not a risk relevant to zk-kit, it is the same for both.
https://github.com/njofce/zk-chat uses a client/server model (see zk-chat-server-lib), whie we would use the @zk-kit libraries, it does mean that there some unknowns unknowns here (does some of the code work only in NodeJS? etc) as they do not run the JS code only in the browser.
While this is true and it is a different model, IIRC there's no Node specific code that does the core RLN operations we care about. Could be wrong though, so requires looking a bit more.
Long term I agree we probably want to use Zerokit for everything, if possible. The most immediate goal now is to get this working end-to-end for testnet2 and Devcon, so the question right now is what would be the fastest reasonable path towards that? What do you think @richard-ramos
It was non-trivial to find the commit version of the zk-kit repo in which the rln module existed (it is now deleted from the zk-kit master branch), but finally found it:
My thoughts and findings so far: Their public and private inputs for the rln proofs LOOK SIMILAR to the zero-kit (but not killic library). zk-kit/rln seems to support secret sharing, and slashing, and also utilize a rlnIdentifier for secret sharing, which matches the vac/zero-kit. But we need to have the circuit file to be sure about the underlying computations.
bn128
.generateMerkleProof
function call and the actual implementation). For EACH rln witness generation, the entire tree needs to be reconstructed from the set of leaves
and then the MerkleProof
object is extracted from that tree. It is perceivable that this would slow down the proof generation a lot. In zero-kit, the tree is constructed once and is used for the subsequent rln witness generations. IncrementalMerkleTree
class in the same repo to construct the tree once and then use createProof
function to populate the required MerkleProof
object. In a nutshell, it means the tree gets created once and used for all the rln witness generations, similar to zero-kit. IncrementalMerkleTree
class is implemented in such a way that it holds the entire tree, similar to zero-kit, hence supports both member addition and deletion. ATM, there is no absolute answer re whether the two libs are the same or not, the more time we invest the more certainty we get about this. I am still investigating.
cc: @fryorcraken @richard-ramos @oskarth
Apparently, the rln relevant functionality is moved from zk-kit repo to this repo https://github.com/Rate-Limiting-Nullifier/rlnjs/
Note that https://github.com/waku-org/js-rln/issues/1#issuecomment-1246067128 was written with @richard-ramos so I believe we are on the same page
Will cancel this meeting to attend Monday/Tuesday Vac PM instead
Regarding zerokit risks: The main risk the bundle size, see https://github.com/waku-org/js-rln/issues/1#issuecomment-1246067128 for breakdown
Hence, for now let's keep focus on zerokit. No need to spend more time investigating difference with rlnjs (@richard-ramos, do you agree?)
rln-js update:
web-chat
Encoder
interface in RLN to append the proof to the proto message.Then, @fryorcraken will clone web-chat
and start integration of Web wallet.
Risks: web-chat uses a React framework for the chat component, uncertain how to fit new buttons/options with this framework.
Currently thinking about a sidebar with the following options:
@oskarth: The most immediate goal now is to get this working end-to-end for testnet2 and Devcon, so the question right now is what would be the fastest reasonable path towards that? What do you think @richard-ramos
After discussing this with @fryorcraken, we concluded that continuing the zerokit/wasm approach is the path we should take, since in practice most of the work required in js-rln for usage in js-waku is complete (besides bugs that might be reported in the PR and the size of zerokit output .wasm file).
When starting on export/import, best to talk with zk-keepers team.
FYI @oskarth @staheri14 I am not sure we'll have time to integrate it in web-chat.
Instead, we are focusing on examples.waku.org/rln-js/
This will still demonstrate sending messages with proof, receiving messages with proof and verifying them
It's a bit more raw. The upside is that it's just javascript and HTML in a single page, hence more readable to any developer than if it was a React app
Devcon deliverable (and more!) achieved:
Need to tidy up the issue, close it and open new issues for follow up if needed.
I suggest to close this for now:
New issue to be opened once https://github.com/vacp2p/rfc/issues/543 is ready
Interface exposed to check proof. Other API improvement can be tracked as we push for adoption and look at improving devex
For now light client protocols are recommended from the browser, we can leave relay validation to nwaku nodes
Slashing is not ready, new issue can be open if and wehn we want to be able to slash from the browser.
Meta issue: https://github.com/vacp2p/research/issues/129
Breakdown
We want (1) , (2) and (3) for DevCon. Order on the other user stories in loose.
1. As a user, I want to join the group
2. As a user, I want to send a message that includes a proof
Would probably use Light Push (but does not really matter)
relay.send
orlightpush.push
is used.3. As a end user, I want to use RLN in the web
4. As a rln-js user, I want to my RLN credentials to be securely saved in the browser storage
[ ] ~Credentials are saved in the browser~
[x] Credentials derived from Ethereum Wallet signature
This is a nice feature for web-chat users
Also an example for developers
See how eth-pm saves encryption key
5. As a rln-js user, I want to be able to export and import my RLN credentials
6. As a user, I want my node to check proofs on messages
The message could be received via store, filter or relay.
Note: This could be pass as a generalized "validation callback" method to filter and store. Maybe some helper to avoid too much code.
7. As a user, I want Waku Relay to check proofs on messages and drop them if invalid
Useful once Relay scalability #905 is ready. Preferably use pubsub validator.
8. As a user, I want to be able to slash spammers
Review if slashing makes sense in the browser once #905 is progressed.
Other requirements
tsc
orparcel
orvite
<script>
tag):rollup
orvite
prettier
+eslint
mocha
/chai
is used for js-waku, ok to consider more modern alternative, consideraegir
karma
, ok to consider better alternative oraegir
(js-waku uses webpack forkarma
, best to tryrollup
or whichever bundler is used for prod code).