ethereum / remix-project

Remix is a browser-based compiler and IDE that enables users to build Ethereum contracts with Solidity language and to debug transactions.
https://remix-ide.readthedocs.io
MIT License
2.45k stars 943 forks source link

Remix crashes/hangs in browser with long iterations which store many variables #2447

Open dzimbeck opened 2 years ago

dzimbeck commented 2 years ago

When running tests on a system that uses a lot of iterations and arrays with lets say around 30-120 elements which are sent to storage, Remix JavaScript VM may run for a few tests but will eventually crash.

In Firefox for example it may say the browser is being "unresponsive" and ask to "wait". If you click wait, sometimes you get lucky and it will continue and complete it's tasks. Other times it fails outright and will stay pending for example.

transact to BAYR.transfer pending ... (indefinitely)

And it will just sit like that forever until I refresh the browser and have to start all over again from the beginning setting up all the test accounts and variables. This issue MAY be a duplicate as I see a few other people complain of similar issues for years but I didn't find any similar issues on github. Still, I have definitely yet to see it resolved.

In Chrome similar failures can happen, it's really all the same. I understand browsers may not be able to handle the tests, however I'm on a pretty fast computer with more than enough memory.

To replicate, set up a function that commits so a couple arrays of various sizes and iterate a lot and see how many times you can run it before it crashes the browser.

What perhaps makes this the most frustrating is it's not really that much data. Even though the stack is maxed out and I use a lot of structs to get around stack being too deep, the issue would persist I think even with a test contract that you keep pushing to see what the breaking point is. On Chrome it says "out of memory"

Most people will say "just use testnet or ganache or truffle or whatever"... but the point is not to have a broken product and Remix JavaScript VM is almost broken for complex interconnected contracts. It barely gets through a few tests and eventually just can't handle any more data and fails. I think a guideline in the GUI even on how much memory should be needed for tests and how to increase it or a way to detect if it's pushing browsers to the limit or solving the issue with it running out of memory outright. I can and will eventually use testnet but there is a great speed and convenience to using the JavaScript VM. So please keep this issue open until it's resolved. Thank you.

qwerty472123 commented 2 years ago

Same question.

I firstly think this is a weakness of ethereumjs. But I tested that and think that performance is acceptable for me.

And then I think it caused by remix's dependency not up-to-date. It still use library like BN.js, not the native BigInt.

Finally, I found that remix always record all memory, stacks and do a lot after each opcode.

https://github.com/ethereum/remix-project/blob/e5a113bdb0f4d09add3ffdceb69f21207c2fd41f/libs/remix-simulator/src/VmProxy.ts#L93

I know that is useful for Debugger plugin. But I think that is not required for most of cases.(debug by events and revert string is ok for me)

So, in my opinion, remix should provide us a button to temporary disable the tracing.

Before the official solution, I use a Tampermonkey script to do that, and the speed is improved a lot.

// ==UserScript==
// @name         Remix Speed up
// @match        http://localhost:8080/*
// @match        http://remix.ethereum.org/*
// @match        https://remix.ethereum.org/*
// @grant        unsafeWindow
// @run-at       document-start
// ==/UserScript==
// change the match to the url for remix what you are using.

// If the hook is OK, Run `switcher.on()` on DevTools Console to enable tracing, `switcher.off()` to disable. the default is disabled
let switcher = {};
unsafeWindow.switcher = switcher;
let isOK = false;
let isON = false;

unsafeWindow.webpackJsonp = [];
let oldPush = unsafeWindow.webpackJsonp.push;
const newPush = function (...data) {
    for (const v of data) {
        if (isOK) break;
        for (const name of Object.keys(v[1])) {
            if (isOK) break;
            const ori = v[1][name];
            v[1][name] = function (module, __webpack_exports__, __webpack_require__, ...args) {
                const ret = ori.apply(this, [module, __webpack_exports__, __webpack_require__, ...args]);
                if (__webpack_exports__.VmProxy) {
                    isOK = true;
                    const oldVM = __webpack_exports__.VmProxy.prototype.setVM;
                    __webpack_exports__.VmProxy.prototype.setVM = function (vm, ...args) {
                        const oldVMOn = vm.on;
                        vm.on = function (type, listener) {
                            if (type === "step") {
                                switcher.on = () => {
                                    isON = true;
                                    return Object.getPrototypeOf(vm).on.call(vm, type, listener);
                                };
                                switcher.off = () => {
                                    isON = false;
                                    return Object.getPrototypeOf(vm).off.call(vm, type, listener);
                                };
                                if (!isON) return vm;
                            }
                            return Object.getPrototypeOf(vm).on.call(vm, type, listener);
                        };
                        const ret = oldVM.apply(this, [vm, ...args]);
                        delete vm.on;
                        return ret;
                    }
                }
                return ret;
            };
            v[1][name].toString = () => ori.toString();
        }
    }
    return oldPush.apply(this, data);
};
let retPush = oldPush;

Object.defineProperty(unsafeWindow.webpackJsonp, 'push', {
    enumerable: true, configurable: true,
    get: () => isOK ? oldPush : retPush,
    set: val => {
        oldPush = val;
        retPush = newPush;
        return val;
    }
});

Futhermore, techniques like key-frame used in MP4 video compress can be used in such scenario.

dzimbeck commented 2 years ago

Thanks for the response! Yes since I'm also testing the contract on Georli, I can post it here. Obviously my contract is extremely complicated. But I want to note that this contract works fine in Goerli but will absolutely crash JavaScript VM. There are advantages to using the JavaScript VM, maybe a person wants to test quick, doesn't want to hassle with getting testnet coins set up or something plus they don't want to drag down the testnet.

github.com/dzimbeck/bitbay-solidity

To set this contract up, deploy BitBay, deploy liquidity pool, deploy all 3 contracts for BAY, BAYR, BAYF and then hook proxies so set BAY,BAYR,BAYF as "true" for a proxy in BitBay(giving them permission to access and make calls), then make liquidity pool also hooked to BitBay main by setting it and then set the proxy for liquidity pool to BitBay so they can call each other.

Then change supply to whatever you want when it's live, then mint coins by either manually supplying array of 38 values or alternative hook in the admin contract to call mint which gives the coins evenly. The reason this contract is complex is because basic sending doesn't send a single value it sends an array of anywhere from 30-38 values because the supply is dynamic(you need to remember what moves from liquid to reserve and what the supply was when it moved or else a true shift in equity is physically impossible). So "simple" transactions are really heavy and do require loops. There is no other way.

I have seen other people post on this issue and even "less complex" contracts will crash as long as they start storing enough data or start to rack up calls to the stack. What's weird is it doesn't crash on the first transaction but it takes a couple transactions to test. Technically for testnet I have 38 values but for remix it was even crashing with 15 values (10 major sections, 5 minor sections) so reduction of the amount of data only barely delayed the problem. And this contract has no bugs as testnet shows it is in fact working.

Anyways, this process works on Goerli 0xD7567f0841Aae04833C8c004e8685cBF9F1a6b11 BAYL 0xe491CC5b00e24fE53cCA89adaC79f5D7FdA80a82 BAYR 0x3c4Fa78cA311FF83A5EdF60140B0b45e8c39eBf9 BAYF (these point to the main data contract which has everything linked)

To reproduce, simply deploy to VM instead of testnet. What will happen is after a couple transactions because VM has to iterate and deal with tons of uints (typical sending is about a half a million to a million gas!) and then after about 5 or so transactions to various users it will absolutely crash. On chrome it runs out of memory on firefox it just hangs. There could be a memory leak or something. Like you said, it could be an issue with tracing. To be honest I don't know why so I opened this issue.

@qwerty472123 Thanks for posting that script, if I ever get the free time I will totally see if that improves the problem.

Still I think this issue should stay open because it is indeed a bug. If they use an indexed database it should be able to in theory support larger amounts of data in javascript.