Recently in one of my many side projects, I wanted to create a simple native app working for iOS and Android which would be able to generate private key from Mnemonics and then sign some transactions.
To get started easily, I chose the Expo Framework and I wasn't really not expecting that it would get so complex.
Naive approach
After a short analysis, I expected the following dependencies:
- Ethers to create and manage wallets.
- Expo as app framework.
After reading the documentation of Ethers, it turns out that I need a bit more dependencies because of some security issue (more here):
// Import the crypto getRandomValues shim (**BEFORE** the shims)
import "react-native-get-random-values"
// Import the the ethers shims (**BEFORE** ethers)
import "@ethersproject/shims"
// Import the ethers library
import { ethers } from "ethers";
As I'm coding in TDD with Typescript, after installing all the dependencies everything worked fine and fast so I thought that the prototype was perfectly working (at least as good as my green tests).
The code was super simple:
const wallet : Wallet = Wallet.fromMnemonic("<memonics>")
Problem
I tried it on my phone!
Expo is really awesome because you can test your app on your phone with the Expo app by just scanning the QR code appearing in your terminal.
After the scan, I realised that the creation of the private key from the Mnemonics was taking more than 1 minute depending on the phone (for both Android and iOS)!
This was really not acceptable for my use case and needed to be fixed.
Moreover, the issue is known and there is a discussion which can be found there. However, after few hours of tries, I decided to go for another way.
Seed for the solution
While searching for the solution I was always thinking: "it's not possible, I cannot be the only person having this problem". And after few hours of search, I found this file in the Metamask repo:
diff --git a/node_modules/bip39/index.js b/node_modules/bip39/index.js
index efed68c..d487227 100644
--- a/node_modules/bip39/index.js
+++ b/node_modules/bip39/index.js
@@ -4,6 +4,8 @@ var _pbkdf2 = require('pbkdf2')
var pbkdf2 = _pbkdf2.pbkdf2Sync
var pbkdf2Async = _pbkdf2.pbkdf2
var randomBytes = require('randombytes')
+const { NativeModules } = require('react-native')
+const Aes = NativeModules.Aes
// use unorm until String.prototype.normalize gets better browser support
var unorm = require('unorm')
@@ -49,11 +51,15 @@ function salt (password) {
return 'mnemonic' + (password || '')
}
-function mnemonicToSeed (mnemonic, password) {
+function mnemonicToSeed(mnemonic, password) {
var mnemonicBuffer = Buffer.from(unorm.nfkd(mnemonic), 'utf8')
var saltBuffer = Buffer.from(salt(unorm.nfkd(password)), 'utf8')
-
- return pbkdf2(mnemonicBuffer, saltBuffer, 2048, 64, 'sha512')
+ // For chrome environments use the javascript version of pbkdf2
+ if (__DEV__ && (!!global.__REMOTEDEV__ || (global.location && global.location.pathname.includes('/debugger-ui'))))
+ return pbkdf2(mnemonicBuffer, saltBuffer, 2048, 64, 'sha512')
+ const seed = Aes.pbkdf2Sync(mnemonicBuffer.toString('utf8'), saltBuffer.toString('utf8'), 2048, 512);
+ const seedBuffer = global.Buffer.from(seed, "hex")
+ return seedBuffer
}
function mnemonicToSeedHex (mnemonic, password) {
It told me 2 things:
- Bip39 was not compatible with React Native
- There is a new concept of patch in the JS ecosystem that I wasn't aware of!
Patching
The theory defines patching as:
"Patching is the process of applying a small piece of code or software update to a larger piece of software or code in order to fix a bug, improve performance, or add a new feature."
The way I was proceeding to fix libraries, before hitting that problem was: clone the repo, do some changes, push the changes to git, use the git version instead of the NPM one.
It turned out that a package called patch-package
is doing the same but much easier for developers!
Patch-Package
First you need to have a look at the doc.
You can install it via:
yarn add patch-package postinstall-postinstall
OR
npm i patch-package
Afterwards adjust your package.json
to add:
"scripts": {
+ "postinstall": "patch-package"
}
Now, you can do your changes, and that's so convenient: you go directly in your node-modules
folder and you change the annoying few lines causing the issue.
Once you're done, you just call:
yarn patch-package <package-name> // in my case bip39
It will compare your modified version with the original version of the library and it will create a diff-file containing your changes in the patches
folder.
This patch needs to be added in your git repo and thus will work with all the collaborators directly (the patches are even applied on Github Actions in my case).
Solution
After I understood how this new weapon was working, I just tried to use Bip39
and ethereumjs
along with ethers
to make my use-case work.
At the end of the patching journey the createWallet
method looks like this.
import { Wallet } from "ethers";
import * as Bip39 from "bip39";
import { hdkey } from "ethereumjs-wallet";
export const createWallet = async (mnemonics: string): Promise<Wallet> => {
const seed = await Bip39.mnemonicToSeed(mnemonics);
const hdNode = hdkey.fromMasterSeed(seed);
const node = hdNode.derivePath("m/44'/60'/0'/0/0");
return new Wallet(node.getWallet().getPrivateKey().toString("hex"));
};
This code runs in under a second with Expo on every device that I could test with (including Moto G, iPhone 6, iPhone 8, iOS Simulator...).
In order for this code to work on Expo, I needed to install few dependencies (Buffer
and events
) and wrote a bunch of patches.
If that's useful to you, here are the patches:
diff --git a/node_modules/bip39/src/index.js b/node_modules/bip39/src/index.js
index 91d1a72..f71d8c6 100644
--- a/node_modules/bip39/src/index.js
+++ b/node_modules/bip39/src/index.js
@@ -1,9 +1,12 @@
"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
+global.Buffer = global.Buffer || require('buffer').Buffer
const createHash = require("create-hash");
const pbkdf2_1 = require("pbkdf2");
const randomBytes = require("randombytes");
const _wordlists_1 = require("./_wordlists");
+const { NativeModules } = require('react-native')
+const Aes = NativeModules.Aes
let DEFAULT_WORDLIST = _wordlists_1._default;
const INVALID_MNEMONIC = 'Invalid mnemonic';
const INVALID_ENTROPY = 'Invalid entropy';
@@ -52,7 +55,12 @@ function salt(password) {
function mnemonicToSeedSync(mnemonic, password) {
const mnemonicBuffer = Buffer.from(normalize(mnemonic), 'utf8');
const saltBuffer = Buffer.from(salt(normalize(password)), 'utf8');
- return pbkdf2_1.pbkdf2Sync(mnemonicBuffer, saltBuffer, 2048, 64, 'sha512');
+ // For chrome environments use the javascript version of pbkdf2
+ if (__DEV__ && (!!global.__REMOTEDEV__ || (global.location && global.location.pathname.includes('/debugger-ui'))))
+ return pbkdf2_1.pbkdf2Sync(mnemonicBuffer, saltBuffer, 2048, 64, 'sha512')
+ const seed = Aes.pbkdf2Sync(mnemonicBuffer.toString('utf8'), saltBuffer.toString('utf8'), 2048, 512);
+ const seedBuffer = global.Buffer.from(seed, "hex")
+ return seedBuffer
}
exports.mnemonicToSeedSync = mnemonicToSeedSync;
function mnemonicToSeed(mnemonic, password) {
diff --git a/node_modules/cipher-base/index.js b/node_modules/cipher-base/index.js
index 6728005..16426bf 100644
--- a/node_modules/cipher-base/index.js
+++ b/node_modules/cipher-base/index.js
@@ -1,5 +1,5 @@
var Buffer = require('safe-buffer').Buffer
-var Transform = require('stream').Transform
+var Transform = require('readable-stream').Transform
var StringDecoder = require('string_decoder').StringDecoder
var inherits = require('inherits')
diff --git a/node_modules/ethereumjs-wallet/dist.browser/index.js b/node_modules/ethereumjs-wallet/dist.browser/index.js
index ef4695c..5d89e22 100644
--- a/node_modules/ethereumjs-wallet/dist.browser/index.js
+++ b/node_modules/ethereumjs-wallet/dist.browser/index.js
@@ -70,7 +70,7 @@ var __importDefault = (this && this.__importDefault) || function (mod) {
};
Object.defineProperty(exports, "__esModule", { value: true });
exports.thirdparty = exports.hdkey = void 0;
-var crypto = __importStar(require("crypto"));
+var crypto = __importStar(require("crypto-js"));
var ethereumjs_util_1 = require("ethereumjs-util");
var scrypt_js_1 = require("scrypt-js");
var hdkey_1 = require("./hdkey");
diff --git a/node_modules/ethereumjs-wallet/dist.browser/thirdparty.js b/node_modules/ethereumjs-wallet/dist.browser/thirdparty.js
index 4d7f582..b66a5d8 100644
--- a/node_modules/ethereumjs-wallet/dist.browser/thirdparty.js
+++ b/node_modules/ethereumjs-wallet/dist.browser/thirdparty.js
@@ -59,7 +59,7 @@ var __importDefault = (this && this.__importDefault) || function (mod) {
};
Object.defineProperty(exports, "__esModule", { value: true });
exports.fromQuorumWallet = exports.fromKryptoKit = exports.fromEtherCamp = exports.fromEtherWallet = void 0;
-var crypto = __importStar(require("crypto"));
+var crypto = __importStar(require("crypto-js"));
var ethereumjs_util_1 = require("ethereumjs-util");
var scrypt_js_1 = require("scrypt-js");
var index_1 = __importDefault(require("./index"));
diff --git a/node_modules/ethereumjs-wallet/src/index.ts b/node_modules/ethereumjs-wallet/src/index.ts
index 421cd71..33534c3 100644
--- a/node_modules/ethereumjs-wallet/src/index.ts
+++ b/node_modules/ethereumjs-wallet/src/index.ts
@@ -1,4 +1,4 @@
-import * as crypto from 'crypto'
+import * as crypto from 'crypto-js'
import {
BN,
keccak256,
Conclusion
Patching similarly to forking is not a great solution but it can get useful when you work on prototypes or on something very experimental technologies or codebases evolving quickly.
I'm happy that I found this inexpensive way to try fixes and I think that this is super useful in the React Native / Expo context because the React Native API is slightly different from the Node API which is the golden standard of most of the libraries.