cloudflare worker fails with some parameters that ethers.js harcode

[wighawag]

cloudflare worker mimic the behavior of a service worker, as such it is like a limited browser environment.

unfortunately it does not implement all parameters of the fetch api making it fails with ethers.js due to these lines:

https://github.com/ethers-io/ethers.js/blob/ce8f1e4015c0f27bf178238770b1325136e3351a/packages/web/src.ts/browser-geturl.ts#L17-L21

While the redirect field works, the other do not and I get a respective error for each:

The 'credentials' field on 'RequestInitializerDict' is not implemented.

The 'cache' field on 'RequestInitializerDict' is not implemented.

The 'mode' field on 'RequestInitializerDict' is not implemented.

The 'referrer' field on 'RequestInitializerDict' is not implemented.

Would be great if we could override the default behavior here.

[ricmoo]

Ugh. :(

The problem is allowing them to be overrides would balloon the size of the code, as those would all need non-default behaviours implemented in the node getUrl functions.

They are the default values, so I am inclined to remove them and let the defaults take over, but that may break something else. I need to investigate this further.

Ideally, Cloudflare would ignore values if they are the default. :s

[ricmoo]

I'm adding a new field to the options (which will be ignored in Node) called skipFetchSetup to accommodate this. It must be set to true, in which case these parameters will not be set explicitly and will rely on the defaults. I don't know how this will affect the CORS for things like connecting to INFURA, since the same-origin is the default, but my guess is since Cloudflare Workers don't support it, they don't enforce it anyways.

This will be in the next minor bump, which should be coming out soon. I've completed 22 of the 30 issues I'm rolling into this release, so it's getting close. :)

[TimTinkers]

Hi @ricmoo I see that commit 6582ede added support for Cloudflare Workers. Unfortunately I'm still experiencing issues attempting to use the InfuraProvider.getWebSocketProvider() in a durable object (Cloudflare's solution for stateful Workers).

Including the following code at any point in the durable object will cause the request to fail.

const ethereumNode = ethers.providers.InfuraProvider.getWebSocketProvider(
  'mainnet', {
    projectId: '...'
  }
);

This throws the error Error: Failed to construct 'WebSocket': the constructor is not implemented. I haven't done any further debugging to try and find the cause.

I am on ethers v5.5.1 and am using esbuild to bundle for execution on the Worker.

[TimTinkers]

I did further digging. Looks like I found my error here, which ultimately comes from ethers using ws for its WebSocket implementation; ws doesn't work in the browser and hence won't work in Workers either.

[ricmoo]

You might need to adjust your bundler settings? The module version of packages will use the browser-ws file, which will expose the WebSocket directly instead.

I forgot the expose the skip flag higher up the stack though, so it may still not work with http. I need to rectify that this week.

[ricmoo]

(I’m frantically trying to get a v6 beta out, which will resolve a lot of these bundler issues and create much smaller code foot prints; sorry I haven’t been as responsive to issues lately as I try to get this ready)

[TimTinkers]

Thanks for the quick reply! Been following a lot of your other issues here; I'm very excited for v6 with the tree shaking improvements.

This could very well be an issue with my bundler; I'm quite a novice when it comes to that. I will go look into the esbuild settings in more detail. Right now I'm just using the itty-durable example settings unaltered.

Another thought I had: can I perhaps use the browser version of ethers (from https://cdn.ethers.io/lib/ethers-5.5.1.umd.min.js) directly? Would you know how I could try including a UMD script in an ESM module? I am at quite the loss for figuring out how to try including that in my project/bundler. It's certainly not going to be as easy as import { ethers } from 'ethers'...

[TimTinkers]

@ricmoo so I played around today with the ESM build of Ethers on a direct browser page

    <script type="module">
      import { ethers } from "./ethers-esm.js";
      ...

and the InfuraProvider.getWebSocketProvider(...) worked a charm there. Compared the source of that ESM script to my esbuild output and it looks like the same script is being sent to the Cloudflare Worker. Which is leaving me confused once again because the Worker is supposed to be just a normal browser process. I'd assume that if it worked in the browser it'd work on Cloudflare. Still at a loss. :(

Side note: I think CORS on the CDN is broken again so I had to ESM script locally on my testing server like this.

[TimTinkers]

@ricmoo I think I identified the issue and posted about it separately in #2237. It might be safe to close this and some other issues that were being caused specifically by the Fetch API changes.

[DanielAGW]

Hi @ricmoo ! I really appreciate your work and totally fell in love with Ethers. Any idea when v6 beta (or even alpha :smile: ) could be released, addressing this Cloudflare Worker issue? We have created a simple Ethereum/Polygon API intended for Cloudflare workers and we would love to try it out outside local environments. We are using JsonRpcProvider with Moralis JSON RPC nodes. Any estimations would be highly appreciated. Thanks!

[wighawag]

@DanielAGW

as a workaround I perform a post-process step after build :

    let content = fs.readFileSync('dist/index.mjs').toString();
    content = content.replace('mode: "cors"', '//mode: "cors"');
    content = content.replace('cache: "no-cache"', '//cache: "no-cache"');
    content = content.replace('credentials: "same-origin"', '//credentials: "same-origin"');
    content = content.replace('referrer: "client"', '//referrer: "client"');
    fs.writeFileSync('dist/index.mjs', content);

Note: using esbuild to generate dist/index.mjs

[DanielAGW]

@wighawag Wow, thank you! I am using webpack and this is my webpack config, based on your suggestion:

module.exports = {
    target: "webworker",
    entry: "./src/index.js",
    mode: "production",
    module: {
        rules: [
            {
                test: /\.(mjs|js|jsx)$/,
                exclude: /node_modules/,
                loader: "babel-loader",
                options: {
                    presets: [
                        "@babel/preset-env",
                        {
                            plugins: [
                                "@babel/plugin-proposal-class-properties"
                            ]
                        }
                    ]
                },
            },
            {
                test: /\.js$/,
                loader: 'string-replace-loader',
                options: {
                    multiple: [
                        { search: 'request.mode = "cors";', replace: '/* request.mode = "cors"; */' },
                        { search: 'request.cache = "no-cache";', replace: '/* request.cache = "no-cache"; */' },
                        { search: 'request.credentials = "same-origin";', replace: '/* request.credentials = "same-origin"; */' },
                        { search: 'request.referrer = "client";', replace: '/* request.referrer = "client"; */' }
                    ]
                }
            }
        ],
    }
};

I needed to install string-replace-loader webpack loader and everything worked out of the box, amazing.

Thank you @wighawag and thank you @ricmoo for the amazing package.

[clbrge]

@ricmoo your PR #1886 that added skipFetchSetup is a good workaround, but it doesn't seem accessible from connectionInfo. Do I miss something ? Shouldn't we have a line

options.skipFetchSetup = !!connection.skipFetchSetup;

in web/src/index to propagate the option?

[ricmoo]

Yeah, I messed up an didn’t expose it high enough up. I might have to make another minor bump to expose it. I was hoping to get v6 out sooner too though. Let me look into this.

[syffs]

@ricmoo your PR #1886 that added skipFetchSetup is a good workaround, but it doesn't seem accessible from connectionInfo. Do I miss something ? Shouldn't we have a line

options.skipFetchSetup = !!connection.skipFetchSetup;

in web/src/index to propagate the option?

@ricmoo I understand that the workaround is there but unusable from any provider connection ? Any ETA ?

Thanks !

[odyslam]

This is great news! I am working on a smol tool to easily resolve ENS addresses via cloudflare workers, so really looking forward to unblocking this :)

[nwhite89]

@wighawag Wow, thank you! I am using webpack and this is my webpack config, based on your suggestion:

module.exports = {
    target: "webworker",
    entry: "./src/index.js",
    mode: "production",
    module: {
        rules: [
            {
                test: /\.(mjs|js|jsx)$/,
                exclude: /node_modules/,
                loader: "babel-loader",
                options: {
                    presets: [
                        "@babel/preset-env",
                        {
                            plugins: [
                                "@babel/plugin-proposal-class-properties"
                            ]
                        }
                    ]
                },
            },
            {
                test: /\.js$/,
                loader: 'string-replace-loader',
                options: {
                    multiple: [
                        { search: 'request.mode = "cors";', replace: '/* request.mode = "cors"; */' },
                        { search: 'request.cache = "no-cache";', replace: '/* request.cache = "no-cache"; */' },
                        { search: 'request.credentials = "same-origin";', replace: '/* request.credentials = "same-origin"; */' },
                        { search: 'request.referrer = "client";', replace: '/* request.referrer = "client"; */' }
                    ]
                }
            }
        ],
    }
};

I needed to install string-replace-loader webpack loader and everything worked out of the box, amazing.

Thank you @wighawag and thank you @ricmoo for the amazing package.

Hi @DanielAGW

Tried looking at doing this but I still get the same errors just wondering how you're using ethers I've got

import { ethers } from 'ethers';

provider = ethers.getDefaultProvider(https://bsc-dataseed.binance.org:443);

with the response of

Error: could not detect network (event="noNetwork", code=NETWORK_ERROR, version=providers/5.5.3)
    at Logger.makeError (worker.js:7514:23)
    at Logger.throwError (worker.js:7523:20)
    at JsonRpcProvider.<anonymous> (worker.js:12370:27)
    at Generator.throw (<anonymous>)
    at rejected (worker.js:12019:65) at line 7513, col 21

Thanks

[odyslam]

I think that it has been fixed by @ricmoo but not included in an NPM version.

[fsjuhl]

@wighawag Wow, thank you! I am using webpack and this is my webpack config, based on your suggestion:

module.exports = {
    target: "webworker",
    entry: "./src/index.js",
    mode: "production",
    module: {
        rules: [
            {
                test: /\.(mjs|js|jsx)$/,
                exclude: /node_modules/,
                loader: "babel-loader",
                options: {
                    presets: [
                        "@babel/preset-env",
                        {
                            plugins: [
                                "@babel/plugin-proposal-class-properties"
                            ]
                        }
                    ]
                },
            },
            {
                test: /\.js$/,
                loader: 'string-replace-loader',
                options: {
                    multiple: [
                        { search: 'request.mode = "cors";', replace: '/* request.mode = "cors"; */' },
                        { search: 'request.cache = "no-cache";', replace: '/* request.cache = "no-cache"; */' },
                        { search: 'request.credentials = "same-origin";', replace: '/* request.credentials = "same-origin"; */' },
                        { search: 'request.referrer = "client";', replace: '/* request.referrer = "client"; */' }
                    ]
                }
            }
        ],
    }
};

I needed to install string-replace-loader webpack loader and everything worked out of the box, amazing.

Thank you @wighawag and thank you @ricmoo for the amazing package.

Thank you so much! This fixed it for me!

[ricmoo]

This is now available in v5.6. I've tried it out and am loving Cloudflare Workers.

To use it, connect to your provider setting the skipFetchSetup flag:

const provider = new StaticJsonRpcProvider({
 url: URL,
 skipFetchSetup: true
});

Let me know if you have any issues!

[ricmoo]

This was added, so I'll close it. Please re-open or create a new issue if there are any problems.

Thanks! :)