ethers-io / ethers.js

Complete Ethereum library and wallet implementation in JavaScript.
https://ethers.org/
MIT License
7.93k stars 1.84k forks source link

React Native: Solutions for slow crypto-operations (`Wallet.createRandom()`) #2250

Closed mrousavy closed 1 year ago

mrousavy commented 2 years ago

A lot of issues have been opened about React Native being slow. For example, creating a wallet takes 33 seconds on an iPhone 11 Pro - I don't even want to find out how slow that is on an older Android device.

This code:

import 'react-native-get-random-values'
import '@ethersproject/shims'
import { ethers } from 'ethers'

  const now = performance.now()
  Logger.log('💰 Creating new Wallet...')
  const wallet = ethers.Wallet.createRandom()
  const end = performance.now()
  Logger.log(
    `💰 New wallet created! Took ${end - now}ms, Phrase: ${wallet.mnemonic.phrase}`,
  )

Takes half a minute:

[16:49:19.253]  💰 Creating new Wallet...
[16:49:52.812]  💰 New wallet created! Took 33559.440958321095ms, Phrase: ...

See https://github.com/facebook/hermes/issues/626

I'm wondering if there is any workaround to make this code faster, as it's definitely not a solution to let the user wait for 5 minutes until he can use the app.

I can create a library for the crypto polyfills that uses a pure C/C++ implementation accessed synchronously through JSI for maximum performance, but I'd need help to find out what algorithms are slow and where I can swap them.

mrousavy commented 2 years ago

For example, for atob and btoa this JSI library by @craftzdog is 4x faster than base64-js: https://github.com/craftzdog/react-native-quick-base64

mrousavy commented 2 years ago

I'd love to make the pbkdf2 function asynchronous (run the algorithm on a separate Thread, return a Promise) to not block the entire JS interaction (pressing buttons, updating loading percentage, etc.)

nvm it only takes 1ms if written in C++, no need to make it async.

mrousavy commented 2 years ago

I did it.

@zemse @mirceanis @ricmoo how do I swap the pbkdf2 function?

The one from this library takes 14 seconds:

ether's pbkdf2 took 14279.91887497902ms

and my pure C++ JSI implementation takes 0.001 second:

C++ pbkdf2 took 1.4742500185966492ms

..so it's almost 10.000x faster.

I'd love to make that open source if there's a way to swap the implementations.

mrousavy commented 2 years ago

So I patched @ethersproject/pbkdf2/lib/browser-pbkdf2.js to use my custom C++ implementation instead of the JS one, and here's the results:

Before

💰 Importing wallet from phrase: ....
💰 Successfully imported wallet in 45114.42308330536ms!

After

💰 Importing wallet from phrase: ....
💰 Successfully imported wallet in 1120.2051666378975ms!

...so I brought it down from 45 seconds to 1 second, a 45x performance improvement.

My patch ```patch diff --git a/node_modules/@ethersproject/pbkdf2/lib/browser-pbkdf2.js b/node_modules/@ethersproject/pbkdf2/lib/browser-pbkdf2.js index 45c6f03..41e4bf6 100644 --- a/node_modules/@ethersproject/pbkdf2/lib/browser-pbkdf2.js +++ b/node_modules/@ethersproject/pbkdf2/lib/browser-pbkdf2.js @@ -1,47 +1,10 @@ "use strict"; Object.defineProperty(exports, "__esModule", { value: true }); exports.pbkdf2 = void 0; -var bytes_1 = require("@ethersproject/bytes"); -var sha2_1 = require("@ethersproject/sha2"); + function pbkdf2(password, salt, iterations, keylen, hashAlgorithm) { - password = (0, bytes_1.arrayify)(password); - salt = (0, bytes_1.arrayify)(salt); - var hLen; - var l = 1; - var DK = new Uint8Array(keylen); - var block1 = new Uint8Array(salt.length + 4); - block1.set(salt); - //salt.copy(block1, 0, 0, salt.length) - var r; - var T; - for (var i = 1; i <= l; i++) { - //block1.writeUInt32BE(i, salt.length) - block1[salt.length] = (i >> 24) & 0xff; - block1[salt.length + 1] = (i >> 16) & 0xff; - block1[salt.length + 2] = (i >> 8) & 0xff; - block1[salt.length + 3] = i & 0xff; - //let U = createHmac(password).update(block1).digest(); - var U = (0, bytes_1.arrayify)((0, sha2_1.computeHmac)(hashAlgorithm, password, block1)); - if (!hLen) { - hLen = U.length; - T = new Uint8Array(hLen); - l = Math.ceil(keylen / hLen); - r = keylen - (l - 1) * hLen; - } - //U.copy(T, 0, 0, hLen) - T.set(U); - for (var j = 1; j < iterations; j++) { - //U = createHmac(password).update(U).digest(); - U = (0, bytes_1.arrayify)((0, sha2_1.computeHmac)(hashAlgorithm, password, U)); - for (var k = 0; k < hLen; k++) - T[k] ^= U[k]; - } - var destPos = (i - 1) * hLen; - var len = (i === l ? r : hLen); - //T.copy(DK, destPos, 0, len) - DK.set((0, bytes_1.arrayify)(T).slice(0, len), destPos); - } - return (0, bytes_1.hexlify)(DK); + // uses my C++ implementation. + return global.pbkdf2(password.buffer, salt.buffer, iterations, keylen, hashAlgorithm) } exports.pbkdf2 = pbkdf2; //# sourceMappingURL=browser-pbkdf2.js.map ```

I still wonder why it takes a full second to import the wallet from the passphrase, I am even using the native C++ atob, btoa and Buffer implementations, will debug further - but at least now the app is useable.

zemse commented 2 years ago

Can you try using resolutions in package.json to override the pbkdf2 subpackage? (I'm currently afk otherwise would include an example)

mrousavy commented 2 years ago

I'll try to play with resolutions too, right now I have used patch-package to locally patch it - quick and dirty but it works.

Since getting the wallet still takes more than a second on a relatively new iPhone, I'm not done with optimizing yet. Unfortunately I'm having trouble getting the flamegraph profiler working in React Native, do you maybe know off the top of your head which other functions are causing those performance issues that are worth looking into? I can also implement those in C++

ricmoo commented 2 years ago

Just as an FYI, I've been adding an option to v6 to register custom implementations for certain crypto methods, so this should be easier to swap out in v6.

mrousavy commented 2 years ago

Do you have an ETA for v6?

sandys commented 2 years ago

@ricmoo this is interesting for us too. actually we have been tracking this using another issue - https://github.com/ethers-io/ethers.js/issues/1503

im betting both will get fixed by the new swapouts.

ricmoo commented 2 years ago

I'm working on v6 as fast as I can and hope to get a beta out before the end of January. There are a few priority changes needed for another minor bump though, which might bump that a bit.

Cakesoft-Sameer commented 2 years ago

@mrousavy this patch seems to be the end to my quest.But still unable to get it working throwing "undefined" while using it.Could you shed some light on the usage

maykonmichel commented 2 years ago

There is something that we can do to help you release v6?

andra961 commented 2 years ago

@mrousavy this patch seems to be the end to my quest.But still unable to get it working throwing "undefined" while using it.Could you shed some light on the usage

I know nothing about jsi but i think you still need to have the module itself installed in the global object somehow. @mrousavy can you please share the pbkdf2 c++ module too so we can try it out?(if i missed it or i misunderstood something please correct me)

Thinhnguyen185 commented 2 years ago

@mrousavy hmm, patch seem don't used, global always "undefined", peformace when run createRandom() on andoird very bad

mrousavy commented 2 years ago

Well obviously you still need the native function, otherwise the patch won't work. That's the whole idea, moving it to native. 😅

The code is closed source for a client of mine (at Margelo), but we will release something that speeds up this operation (and all other crypto operations) by magnitudes soon.

@ricmoo I think I mentioned this before, but we're essentially bringing crypto to React Native with C++ implementations instead of JS browser-polyfills, so this will be a huge performance improvement 🔥 I'll DM you on Twitter once we are ready to test the integration with ethers

linxianxi commented 2 years ago

@ricmoo When will V6 be released? My application is about to go into production, so I may have to switch to web3.js

ricmoo commented 2 years ago

There is a v6 beta available now, but I haven’t got any meaningful documentation together yet.

If the new Wallet(utils.randomBytes(32)) method doesn’t work for you (which should be instant, even on react-native), you could probably trivially just implement HDNode using native libraries, or there are probably already existing libraries for that part designed for RN.

But if you go the Web3.js route, I’d be curious how easy it is to switch between the two.

linxianxi commented 2 years ago

There is a v6 beta available now, but I haven’t got any meaningful documentation together yet.

If the new Wallet(utils.randomBytes(32)) method doesn’t work for you (which should be instant, even on react-native), you could probably trivially just implement HDNode using native libraries, or there are probably already existing libraries for that part designed for RN.

But if you go the Web3.js route, I’d be curious how easy it is to switch between the two.

new Wallet(utils.randomBytes(32)) does not generate mnemonic words.web3.js integration with react-native is more difficult and has no documentation. If you can tell us how to use it in V6 here.Thanks.

sandys commented 2 years ago

Same here. I would prefer to use v6 beta and put my app in production. Rather than switch between libraries.

Anyway u could give us a complete example on how to use v6 ?

On Thu, 14 Apr, 2022, 17:35 linxianxi, @.***> wrote:

There is a v6 beta available now, but I haven’t got any meaningful documentation together yet.

If the new Wallet(utils.randomBytes(32)) method doesn’t work for you (which should be instant, even on react-native), you could probably trivially just implement HDNode using native libraries, or there are probably already existing libraries for that part designed for RN.

But if you go the Web3.js route, I’d be curious how easy it is to switch between the two.

new Wallet(utils.randomBytes(32)) does not generate mnemonic words.web3.js integration with react-native is more difficult and has no documentation. If you can tell us how to use it in V6 here.Thanks.

— Reply to this email directly, view it on GitHub https://github.com/ethers-io/ethers.js/issues/2250#issuecomment-1099046177, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAASYUZR5PUHRK34JGCTYJDVE7YIVANCNFSM5HA54GVA . You are receiving this because you commented.Message ID: <ethers-io/ethers .@.***>

zemse commented 2 years ago

Anyway u could give us a complete example on how to use v6 ?

Until the official docs are up here's an example to use custom pbkdf2 implementation.

IMP: This code example needs a fast pbkdf2 implementation in RN. Just copy pasting this below snippet would NOT WORK.

import { Wallet } from "ethers";
import { pbkdf2 } from "@ethersproject/crypto";

// IMP: this code WON'T WORK by simple copy paste in React Native, you need to replace the following with a
// working implementation of pbkdf2 in RN.
import { pbkdf2Sync } from "crypto"; // fast implementation in your enviornment

/**
 * @param {*} password Uint8Array
 * @param {*} salt Uint8Array
 * @param {*} iterations number
 * @param {*} keylen number
 * @param {*} algo "sha256" | "sha512"
 * @returns BytesLike
 */
function customPbkdf2(password, salt, iterations, keylen, algo) {
  console.log("custom pbkdf2");
  return pbkdf2Sync(password, salt, iterations, keylen, algo);
}

// registers custom implementation for pbkdf2
pbkdf2.register(customPbkdf2);

console.log("creating wallet");
const wallet = Wallet.createRandom();
console.log(wallet.address);

I would prefer to use v6 beta and put my app in production.

Note that v6 is in very early beta as mentioned by ricmoo.

linxianxi commented 2 years ago

Anyway u could give us a complete example on how to use v6 ?

Until the official docs are up here's an example to use custom p

import { Wallet } from "ethers";
import { pbkdf2 } from "@ethersproject/crypto";

import { pbkdf2Sync } from "crypto"; // fast implementation in your enviornment

/**
 * @param {*} password Uint8Array
 * @param {*} salt Uint8Array
 * @param {*} iterations number
 * @param {*} keylen number
 * @param {*} algo "sha256" | "sha512"
 * @returns BytesLike
 */
function customPbkdf2(password, salt, iterations, keylen, algo) {
  console.log("custom pbkdf2");
  return pbkdf2Sync(password, salt, iterations, keylen, algo);
}

// registers custom implementation for pbkdf2
pbkdf2.register(customPbkdf2);

console.log("creating wallet");
const wallet = Wallet.createRandom();
console.log(wallet.address);

I would prefer to use v6 beta and put my app in production.

Note that v6 is in very early beta as mentioned by ricmoo.

This will report an error,did you try it on RN?

zemse commented 2 years ago

Which line does it error on?

import { pbkdf2Sync } from "crypto"; // fast implementation in your enviornment

function customPbkdf2(password, salt, iterations, keylen, algo) {
  console.log("custom pbkdf2");
  return pbkdf2Sync(password, salt, iterations, keylen, algo);
}

You need to replace the above code with a pbkdf2 method that works nicely in RN.

andra961 commented 2 years ago

Which line does it error on?

import { pbkdf2Sync } from "crypto"; // fast implementation in your enviornment

function customPbkdf2(password, salt, iterations, keylen, algo) {
  console.log("custom pbkdf2");
  return pbkdf2Sync(password, salt, iterations, keylen, algo);
}

You need to replace the above code with a pbkdf2 method that works nicely in RN.

Pardon me where can i get the v6 code to try this out?

ricmoo commented 2 years ago

@andra961 it’s the v6-beta branch of this repo, or npm install ethers@beta. There is no documentation yet.

zemse commented 2 years ago

@mrousavy Do you have the pure C++ JSI implementation mentioned in the reply above available for public? The example here is for swapping the pbkdf2 ethers would use.

andra961 commented 2 years ago

@mrousavy Do you have the pure C++ JSI implementation mentioned in the reply above available for public? The example here is for swapping the pbkdf2 ethers would use.

He already said it's closed source

Thinhnguyen185 commented 2 years ago

I find solution for slow when create random wallet ` import hdkey from 'ethereumjs-wallet/dist.browser/hdkey'; import bip39 from 'bip39'; import {Contract, ethers, Wallet} from 'ethers'; const getPrivateKeyFromSeed = (mnemonic: string) => { const root = hdkey.fromMasterSeed(bip39.mnemonicToSeed(mnemonic)); const derivedNode = root.derivePath(ethers.utils.defaultPath); const privateKey = derivedNode.getWallet().getPrivateKeyString(); return privateKey; };

const createWallet = () => { let entropy: Uint8Array = randomBytes(16); const mnemonic = entropyToMnemonic(entropy); const privateKey = getPrivateKeyFromSeed(mnemonic);

return { wallet: new ethers.Wallet(privateKey), mnemonic, }; }; `

andra961 commented 2 years ago

I find solution for slow when create random wallet ` import hdkey from 'ethereumjs-wallet/dist.browser/hdkey'; import bip39 from 'bip39'; import {Contract, ethers, Wallet} from 'ethers'; const getPrivateKeyFromSeed = (mnemonic: string) => { const root = hdkey.fromMasterSeed(bip39.mnemonicToSeed(mnemonic)); const derivedNode = root.derivePath(ethers.utils.defaultPath); const privateKey = derivedNode.getWallet().getPrivateKeyString(); return privateKey; };

const createWallet = () => { let entropy: Uint8Array = randomBytes(16); const mnemonic = entropyToMnemonic(entropy); const privateKey = getPrivateKeyFromSeed(mnemonic);

return { wallet: new ethers.Wallet(privateKey), mnemonic, }; }; `

How can you make bip39 work on react native?

Thinhnguyen185 commented 2 years ago

I find solution for slow when create random wallet import hdkey from 'ethereumjs-wallet/dist.browser/hdkey'; import bip39 from 'bip39'; import {Contract, ethers, Wallet} from 'ethers'; const getPrivateKeyFromSeed = (mnemonic: string) => { const root = hdkey.fromMasterSeed(bip39.mnemonicToSeed(mnemonic)); const derivedNode = root.derivePath(ethers.utils.defaultPath); const privateKey = derivedNode.getWallet().getPrivateKeyString(); return privateKey; }; const createWallet = () => { let entropy: Uint8Array = randomBytes(16); const mnemonic = entropyToMnemonic(entropy); const privateKey = getPrivateKeyFromSeed(mnemonic); return { wallet: new ethers.Wallet(privateKey), mnemonic, }; };

How can you make bip39 work on react native?

https://www.npmjs.com/package/bip39 this work on RN

andra961 commented 2 years ago

I find solution for slow when create random wallet import hdkey from 'ethereumjs-wallet/dist.browser/hdkey'; import bip39 from 'bip39'; import {Contract, ethers, Wallet} from 'ethers'; const getPrivateKeyFromSeed = (mnemonic: string) => { const root = hdkey.fromMasterSeed(bip39.mnemonicToSeed(mnemonic)); const derivedNode = root.derivePath(ethers.utils.defaultPath); const privateKey = derivedNode.getWallet().getPrivateKeyString(); return privateKey; }; const createWallet = () => { let entropy: Uint8Array = randomBytes(16); const mnemonic = entropyToMnemonic(entropy); const privateKey = getPrivateKeyFromSeed(mnemonic); return { wallet: new ethers.Wallet(privateKey), mnemonic, }; };

How can you make bip39 work on react native?

https://www.npmjs.com/package/bip39 this work on RN

I tried and it didn't. Pretty sure it requires shims to work. Are you using something like rn-nodeify?

linxianxi commented 2 years ago

It took me one day to find a solution!

npm i events assert bip39 buffer hdkey crypto-browserify stream-browserify
npm i -D @types/hdkey    // if your project uses typescript

metro.config.js

module.exports = {
  // add resolver extraNodeModules, like webpack alias
  resolver: {
    extraNodeModules: {
      crypto: require.resolve('crypto-browserify'),
      stream: require.resolve('stream-browserify'),
    },
  },
  ...
};

index.js

// add this line
global.Buffer = require('buffer').Buffer;

create wallet

const bip39 = require('bip39');
import HDKey from 'hdkey';

const createWallet = () => {
  const entropy = ethers.utils.randomBytes(16);
  const mnemonic = ethers.utils.entropyToMnemonic(entropy);
  const hdkey = HDKey.fromMasterSeed(bip39.mnemonicToSeedSync(mnemonic));
  const derivedNode = hdkey.derive(ethers.utils.defaultPath);
  const privateKey = ethers.utils.hexlify(derivedNode.privateKey);
  const { address } = new ethers.Wallet(privateKey);
  return {
    address,
    mnemonic,
    privateKey
  }
}
heroims commented 2 years ago

My current solution is to directly use react-native-scrypt and react-native-aes-crypto to replace scrypt's method and pbkdf2's method. I modified a few js with patches in a less elegant way. if you need it you can download it here.

sandys commented 2 years ago

My current solution is to directly use react-native-scrypt and react-native-aes-crypto to replace scrypt's method and pbkdf2's method. I modified a few js with patches in a less elegant way. if you need it you can download it here.

this is pretty cool! is it possible for you to structure ur code as a separate library that overrides the ethers and other libraries ? like so https://www.tutorialkart.com/nodejs/override-function-of-a-node-js-module/

any project will set an override so that your library patches are taken - https://www.stefanjudis.com/today-i-learned/how-to-override-your-dependencys-dependencies/

this will make ur library very quick to adopt.

adelbeke commented 2 years ago

Did someone succeeded to use Marc's solution with his custom C++ ?

mrousavy commented 2 years ago

Yes, I did.

I'm working on releasing the package as open-source today or tomorrow, stay tuned. Follow me on Twitter for the announcement

mrousavy commented 2 years ago

Hey - we've just published the fully native C++ library for Crypto. 🎉🥳

As I said before, doing crypto in JS simply won't work for mobile apps (React Native) since JS is too slow in such cases. That's why libraries like BN.js or crypto-browserify are a huge bottleneck in React Native applications. Instead, we at Margelo implemented those functions fully natively in C++ to offer huge speedups (in BN.js we measured up to 300x improvements and in QuickCrypto up to 58x improvements!)

As of today, I recommend to install both react-native-quick-crypto and react-native-bignumber in your projects and use them as drop-in replacements for free app speedups!

Install:

yarn add react-native-bignumber
yarn add react-native-quick-crypto

Then add them to your babel.config.js (see the READMEs for details).

This makes ethers.js run smoothly on React Native! Would be cool if we could add that to the README or docs of ethers.js? cc @ricmoo

adelbeke commented 2 years ago

@mrousavy Amazing work !

We'll use it on Ko :)

mrousavy commented 2 years ago

@ricmoo ethers.js currently doesn't use crypto (nor react-native-quick-crypto) if it is available, not sure where the try/import/catch is happening (or if you guys are doing that), but just wanted to make sure that y'all are aware of this.

I had to manually patch node_modules/@ethersproject/pbkdf2/... to use our crypto.pbkdf2Sync function (which is implemented natively in react-native-quick-crypto) to improve the performance for wallet creation.

const start = performance.now()
const wallet = ethers.Wallet.createRandom()
const end = performance.now()
console.log(`Creating a Wallet took ${end - start} ms.`)

Before patching @ethersproject/pbkdf2 (with your JS-based implementation):

Creating a Wallet took 16862 ms

After patching @ethersproject/pbkdf2 (with our native C++/JSI react-native-quick-crypto implementation):

Creating a Wallet took 289 ms

Here's the patch I used:

@ethersproject+pbkdf2+5.6.1.patch ```diff diff --git a/node_modules/@ethersproject/pbkdf2/lib.esm/pbkdf2.js b/node_modules/@ethersproject/pbkdf2/lib.esm/pbkdf2.js index e211793..06a80af 100644 --- a/node_modules/@ethersproject/pbkdf2/lib.esm/pbkdf2.js +++ b/node_modules/@ethersproject/pbkdf2/lib.esm/pbkdf2.js @@ -1,44 +1,4 @@ "use strict"; -import { arrayify, hexlify } from "@ethersproject/bytes"; -import { computeHmac } from "@ethersproject/sha2"; -export function pbkdf2(password, salt, iterations, keylen, hashAlgorithm) { - password = arrayify(password); - salt = arrayify(salt); - let hLen; - let l = 1; - const DK = new Uint8Array(keylen); - const block1 = new Uint8Array(salt.length + 4); - block1.set(salt); - //salt.copy(block1, 0, 0, salt.length) - let r; - let T; - for (let i = 1; i <= l; i++) { - //block1.writeUInt32BE(i, salt.length) - block1[salt.length] = (i >> 24) & 0xff; - block1[salt.length + 1] = (i >> 16) & 0xff; - block1[salt.length + 2] = (i >> 8) & 0xff; - block1[salt.length + 3] = i & 0xff; - //let U = createHmac(password).update(block1).digest(); - let U = arrayify(computeHmac(hashAlgorithm, password, block1)); - if (!hLen) { - hLen = U.length; - T = new Uint8Array(hLen); - l = Math.ceil(keylen / hLen); - r = keylen - (l - 1) * hLen; - } - //U.copy(T, 0, 0, hLen) - T.set(U); - for (let j = 1; j < iterations; j++) { - //U = createHmac(password).update(U).digest(); - U = arrayify(computeHmac(hashAlgorithm, password, U)); - for (let k = 0; k < hLen; k++) - T[k] ^= U[k]; - } - const destPos = (i - 1) * hLen; - const len = (i === l ? r : hLen); - //T.copy(DK, destPos, 0, len) - DK.set(arrayify(T).slice(0, len), destPos); - } - return hexlify(DK); -} +var crypto = require("crypto"); +exports.pbkdf2 = crypto.pbkdf2Sync; //# sourceMappingURL=pbkdf2.js.map diff --git a/node_modules/@ethersproject/pbkdf2/lib/browser-pbkdf2.js b/node_modules/@ethersproject/pbkdf2/lib/browser-pbkdf2.js index 45c6f03..f2c584d 100644 --- a/node_modules/@ethersproject/pbkdf2/lib/browser-pbkdf2.js +++ b/node_modules/@ethersproject/pbkdf2/lib/browser-pbkdf2.js @@ -1,47 +1,6 @@ "use strict"; Object.defineProperty(exports, "__esModule", { value: true }); exports.pbkdf2 = void 0; -var bytes_1 = require("@ethersproject/bytes"); -var sha2_1 = require("@ethersproject/sha2"); -function pbkdf2(password, salt, iterations, keylen, hashAlgorithm) { - password = (0, bytes_1.arrayify)(password); - salt = (0, bytes_1.arrayify)(salt); - var hLen; - var l = 1; - var DK = new Uint8Array(keylen); - var block1 = new Uint8Array(salt.length + 4); - block1.set(salt); - //salt.copy(block1, 0, 0, salt.length) - var r; - var T; - for (var i = 1; i <= l; i++) { - //block1.writeUInt32BE(i, salt.length) - block1[salt.length] = (i >> 24) & 0xff; - block1[salt.length + 1] = (i >> 16) & 0xff; - block1[salt.length + 2] = (i >> 8) & 0xff; - block1[salt.length + 3] = i & 0xff; - //let U = createHmac(password).update(block1).digest(); - var U = (0, bytes_1.arrayify)((0, sha2_1.computeHmac)(hashAlgorithm, password, block1)); - if (!hLen) { - hLen = U.length; - T = new Uint8Array(hLen); - l = Math.ceil(keylen / hLen); - r = keylen - (l - 1) * hLen; - } - //U.copy(T, 0, 0, hLen) - T.set(U); - for (var j = 1; j < iterations; j++) { - //U = createHmac(password).update(U).digest(); - U = (0, bytes_1.arrayify)((0, sha2_1.computeHmac)(hashAlgorithm, password, U)); - for (var k = 0; k < hLen; k++) - T[k] ^= U[k]; - } - var destPos = (i - 1) * hLen; - var len = (i === l ? r : hLen); - //T.copy(DK, destPos, 0, len) - DK.set((0, bytes_1.arrayify)(T).slice(0, len), destPos); - } - return (0, bytes_1.hexlify)(DK); -} -exports.pbkdf2 = pbkdf2; +var crypto = require("crypto"); +exports.pbkdf2 = crypto.pbkdf2Sync; //# sourceMappingURL=browser-pbkdf2.js.map ```

You can DM me on Twitter if you have any questions or want to collab on getting this integrated with ethers.js. :)

anarkrypto commented 2 years ago

@mrousavy I couldn't get it to work with Expo.

There was no change in performance.

This implementation only works with prebuild ?

sonnyvesali commented 2 years ago

@anarkrypto I was using an expo managed workflow and apply the @ethersproject/project/pbkdf2 patch applied above. However I found that it won't work in the managed workflow, and you'll have to eject to bare workflow if you want it to work due to the native code modification that will happen under the hood. It's not enough to apply the changes in the babel.config.js you have to patch the pbkdf2 in order to ensure that ethers works performantly when creating new wallets. Hope this helps. I can confirm that the patch does work and lowers wait times from ~6-10 seconds [depending on the device, beforehand it was 6 seconds on new iPhones like the iPhone 13 pro, and 10-12 seconds on iPhone 7s, all the way to sub to single second waits for all phone's on all builds for new wallet creation in production builds for iOS, I haven't built it yet for android but I assume that it works as well, but maybe some build.gradle changes will have to be made.

Note: I got the error that it doesn't work in Expo Go and that's exactly the point remember this is drop in native code, you're going to have to use less convenient local building tools like the react-native cli or expo dev-client builds from now on to build the app for emulators locally, and the nice things that come with testing expo go on a real phone on the same network also goes away, hope this helps 😄

sonnyvesali commented 2 years ago

@anarkrypto send me an dm on twitter w/ a link to your expo codebase or if it's closed source invite me as a contributor to your org and I can show you what it looks like to get it working.

bdtren commented 2 years ago

If you just need bip39 then https://www.npmjs.com/package/@mfellner/react-native-bip39 is a good solution.

But for my case, I also need ed25519-hd-key, ... so upgrade to RN 0.70+ and ready to face a lot of deprecated errors is my only solution for now.

margox commented 1 year ago

Step 1: create a patch js file with the following content:

const crypto = require("react-native-quick-crypto")
exports.pbkdf2 = crypto.pbkdf2Sync

Step 2: update the module-resolver config in babel.config.js:

{
   alias: {
      // other alias ....
      '@ethersproject/pbkdf2': 'path/to/your/patch.js',
    }
}

Step 3: npm start --reset-cache

FrozenPyrozen commented 1 year ago

You could try to disable hermes for android and it would work faster:

In my case: const seed = await bip39.mnemonicToSeed(mnemonic); tooks 1500ms with hermes and without hermes 336ms

ricmoo commented 1 year ago

This is available in v6, using the .register functions on each of the hash, hmac, etc. functions.

For example:

ethers.keccak256.register((data: Uint8Array) => {
  yourCustomLibHere.hash(data);
});

See the src.ts/crypto/ files for more details. I'll be adding more docs for RN, Expo, etc. to help too.

Thanks! :)

dexterslabor commented 1 year ago

@mrousavy amazing work! Improvement was from 4108ms down to 210ms replacing @ethersproject/pbkdf2/lib/browser-pbkdf2 with your implementation of pbkdf2Sync 🥳

martin-opensky commented 1 year ago

Thank you to everyone in this thread and particularly @ricmoo and @mrousavy for their work with these libraries.

Using the thread above I was able to get Ethers v6.6 and react-native-quick-crypto (RNQC) working with RN. Without using RNQC Ethers v6.6 was able to create a wallet in around 1300ms which was a big improvement from Ethers v5.7 (around 5500ms). However when using RNQC, Ethers V6.6 creates a new wallet in 300ms dropping to 50ms for any additional requests.

I will include my setup for anyone wishing to work with Ethers v6:

  1. Follow the installation steps as mentioned in the RNQC repo including updating the babel.config.js file: https://github.com/margelo/react-native-quick-crypto
  2. You no longer need to use "react-native-get-random-values" as mentioned in the Ethers v5 docs. Instead use getRandomValues from RNQC before importing the @ethersproject/shims. I created a file called 'ethers-setup.ts' with the following:
import { getRandomValues } from 'react-native-quick-crypto';
global.getRandomValues = getRandomValues;

export * from '@ethersproject/shims';

Then imported 'ethers-setup' into the App.tsx file.

  1. For me, the patch mentioned by @margox only works with Ethers v5. For v6 I utilised the new approach outlined by @ricmoo to register pbkdf2. The code I have looks like this:
import * as ethers from 'ethers';
import { pbkdf2Sync } from 'react-native-quick-crypto';

ethers.pbkdf2.register(
  (
    password: Uint8Array,
    salt: Uint8Array,
    iterations: number,
    keylen: number,
    algo: 'sha256' | 'sha512'
    // eslint-disable-next-line max-params
  ) => {
    return ethers.hexlify(pbkdf2Sync(password, salt, iterations, keylen, algo));
  }
);

export * from 'ethers';

I now import this into my project instead of the usual import { ethers } from 'ethers';

mrousavy commented 1 year ago

Awesome stuff @martin-opensky - glad to see you can benefit from those speedups.

I believe there's also other functions than pbkdf2 that can be replaced and should be much faster - @ricmoo do you have any more info on that?

martin-opensky commented 1 year ago

Yeah great point @mrousavy. I am intrigued to see whether the encrypt wallet method could be improved as it takes 30 seconds+ during testing. It seems this process relies on scrypt but I struggled to get the RNQC library to work with ethers, using the ethers.scrypt.register approach.

I did have a question @mrousavy if you could confirm the security of the native implementations? Ethers uses noble-hashes which has been audited. Can you think of any security issues that could arise by replacing these functions with native implementations you have written?