Open MaksymZavershynskyi opened 4 years ago
Just found we have some initial work on this: https://github.com/nearprotocol/rust-ethash
Curious considerations of using Rust vs Javascript? The benefit of using JS is that will allow to have pure functions directly in the browser. Not sure what is the performance hit obviously here. With Rust, we can compile to WASM and introduce into browser as well, but definitely more work.
I suggest rust, because the go implementation took about 4 seconds to proof one block, with js that's going to be much longer.
With Rust, we can compile to WASM and introduce into browser as well, but definitely more work.
I think this case is simpler, because it's almost a single pure function to import
@ilblackdragon I have added Rust vs JS pros and cos into the description.
I think we should definitely do rust-ethash based on pros and cons above
Here is another reason to start replacing components with Rust. Conversation is taken from Slack:
@Kouprin (wasn't able to run e2e tests on his machine)
I passed e2e test locally after downgrading yarn from 1.22.4 to 1.22.0. We need to understand why yarn version affects proofs calculation. I also found that sol-contracts bin-files are different with yarn 1.22.4. This may help to investigate. For now, we should add to README requirement to use yarn 1.22.0. To do that, it's needed to execute yarn set version 1.22.0.
@nearmax
Sad, yarn on my machine is 1.22.4 and it passes. It is not the first bug we had with npm and yarn. For example some of our stuff does not work with node version > 13. This is a pretty good reason for migrating our code to a language with more deterministic package management, like Rust. Fortunately we can do it incrementally by compiling Rust components with Wasm.
Also a lesson i learnt, yarn probably has bug in handle git dependency cache, to ensure it's correctly resolve git dependencies, you need yarn cache clean (I think there was conversation with Chad, Mike and me on slack that finally found this). npm, AFAIK is more reliable because it doesn't have cache :D Also, it's important to commit and check yarn.lock isn't changed after every commit (it's set for rainbow-bridge-sol on ci) rainbow-bridge-lib and rainbow-bridge-cli does commit yarn.lock, but doesn't check yarn.lock stay unchanged on ci
On Sat, Sep 5, 2020 at 12:45 PM Maksym Zavershynskyi < notifications@github.com> wrote:
Here is another reason to start replacing components with Rust. Conversation is taken from Slack:
@Kouprin https://github.com/Kouprin (wasn't able to run e2e tests on his machine)
I passed e2e test locally after downgrading yarn from 1.22.4 to 1.22.0. We need to understand why yarn version affects proofs calculation. I also found that sol-contracts bin-files are different with yarn 1.22.4. This may help to investigate. For now, we should add to README requirement to use yarn 1.22.0. To do that, it's needed to execute yarn set version 1.22.0.
@nearmax https://github.com/nearmax
Sad, yarn on my machine is 1.22.4 and it passes. It is not the first bug we had with npm and yarn. For example some of our stuff does not work with node version > 13. This is a pretty good reason for migrating our code to a language with more deterministic package management, like Rust. Fortunately we can do it incrementally by compiling Rust components with Wasm.
â You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/near/rainbow-bridge-cli/issues/150#issuecomment-687654395, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADFFFCALMNRWVXFT6T25VGTSEKIN7ANCNFSM4OSLQ4JQ .
To really own this code we need to understand it deeply, which means we would need to do a lot of in-depth reading of it, including adding lots of our own tests. Since there is a good chance the entire rainbow bridge CLI might be rewritten into Rust for security reasons, we might as well rewrite ethashproof in Rust first. In the process, we might greatly benefit from reusing existing Ethereum Rust utils, like Parity utils that might entirely replace github.com/tranvictor/ethutils which might make the code smaller and easier to verify/read. Also, we would need to write the spec of the ethashproof library in the process, because we would benefit from having a detailed spec of the bridge, overall. knowledge that many people at NEAR should have.
Which security reasons do you think Rust is better than Go? It seems Go is "secure" enough so the official ethereum client is written with...
On the other hand, ethashproof
package doesn't need my ethutils
package at all, you can use directly ethclient
from go-ethereum
package. I was using it in the commandline because i was lazy and was used to my own library.
Which security reasons do you think Rust is better than Go? It seems Go is "secure" enough so the official ethereum client is written with...
Rust prevents torn reads/writes which is very desirable for cryptographic tools. Since we might need to rewrite it from the scratch anyway and we have most of the code in Rust it makes sense to rewrite it into Rust to keep the number of languages that we need to maintain small. Also, we will be able to link it against our other Rust code if we ever need to.
Is there somewhere to apply for grants for this work? Someone to reach out to, my email is drew@commonwealth.im
@nearmax are we going to grant a bounty on this issue? This seems to out of our priority for a while
Hello, we @edgeware-builders rewrote this library in Rust. It is extensible if you want to help out. We are also interested if this fulfills any bounty since we are trying to find grants to support this work.
https://github.com/rust-ethereum/ethash/pull/11
Is there any interest in providing support? Nonetheless, we hope that this is useful for you all. We use a different merkle tree to build DAG roots but similarly have generated 10 years of roots into the future if it helps anyone out.
@djsatok this might be useful for that issue as well.
@drewstone , hey there! I was literally going through the rainbow-bridge issues today and looking into them. Let's connect via e-mail.
https://github.com/nearprotocol/ethashproof is a fork of a fork of https://github.com/tranvictor/ethashproof While the original project is a great, according to the original author it was in PoC stage for a long time already. Also, it uses utils written by the same author github.com/tranvictor/ethutils to work with Ethereum-specific primitives.
To really own this code we need to understand it deeply, which means we would need to do a lot of in-depth reading of it, including adding lots of our own tests. Since there is a good chance the entire rainbow bridge CLI might be rewritten into Rust for security reasons, we might as well rewrite ethashproof in Rust first. In the process, we might greatly benefit from reusing existing Ethereum Rust utils, like Parity utils that might entirely replace github.com/tranvictor/ethutils which might make the code smaller and easier to verify/read. Also, we would need to write the spec of the ethashproof library in the process, because we would benefit from having a detailed spec of the bridge, overall.
Update 2020-08-30
Pros and Cos of Rust instead of JS
Pros
Result
, while JS have implicit through exception mechanism. Because we are using some JS libraries current bridge code is riddled with the following pattern, which is very unsafe because of the retries and generic try-catch mechanisms.Cons