Support HTTP_PROXY and NO_PROXY environment variables

[thda]

Hi,

building the example pages fails behind a corporate proxy.

Fetching d3-dsv fails and running npm add d3-dsv does not fix.

I cannot post the stack trace (corporate policies) but it looks like fetch uses undici which does not handle https_proxy and no_proxy env variables.

Thanks

[mbostock]

Ref. https://github.com/nodejs/undici/issues/1650 Ref. https://github.com/nodejs/node/issues/8381

[thda]

Thanks for acknowledging the feature request.

In the meantime is it possible to bypass the dynamic install done using jsdelivr and rely only on node_modules ?

Some corporate environments provide vetted mirrors of npm and disallow the use of proxys in build environments.

[Fil]

In the meantime is it possible to bypass the dynamic install done using jsdelivr and rely only on node_modules ?

The next release will integrate this feature that was just merged this week https://github.com/observablehq/framework/pull/1156

[mbostock]

Also, even if you use node_modules for explicit imports, you would have to fastidiously avoid using any implicit imports, as all implicit imports are npm: imports that are downloaded from jsDelivr.

It should be possible for us to add a proxy config option, and then respect that option when fetching from jsDelivr. I’m looking into it now. The main difficulty is the plumbing (we make the request within populateNpmCache and we don’t have access to the config there, but we could pass it in).

[mbostock]

You could try installing Undici and adding the following to your observablehq.config.js:

import {ProxyAgent, setGlobalDispatcher} from "undici";

const proxyAgent = new ProxyAgent({uri: process.env.https_proxy});
setGlobalDispatcher(proxyAgent);

Or, if your proxy has a username and password and uses basic auth:

import {ProxyAgent, setGlobalDispatcher} from "undici";

const proxyUrl = new URL(process.env.https_proxy);
const token = `Basic ${btoa(`${proxyUrl.username}:${proxyUrl.password}`)}`;
const proxyAgent = new ProxyAgent({uri: proxyUrl.protocol + proxyUrl.host, token});
setGlobalDispatcher(proxyAgent);

This would require you to set the https_proxy environment variable and pass it through to the observable command, say using dotenv. You could also just hard-code the process.env.https_proxy with your proxy to see if it works.

[muuankarski]

Install undici and adding the content of upper code block to observablehq.config.js with hard coded proxy address did the trick and I can finally use versions above 1.0.0!

[thda]

Thanks, the provided code fixed it for me too. I’ll try the implicit import route too once released.

[gdelfino]

You could try installing Undici and adding the following to your observablehq.config.js:

import {ProxyAgent, setGlobalDispatcher} from "undici";

const proxyAgent = new ProxyAgent({uri: process.env.https_proxy});
setGlobalDispatcher(proxyAgent);

This worked for me after some changes. In my case, there is not only a corporate proxy but also a custom CA. For now I just disabled the check, but ideally I should somehow configure it to use my custom CA certificate:

import {ProxyAgent, setGlobalDispatcher} from "undici";

const proxyAgent = new ProxyAgent( {uri: process.env.https_proxy, 
                                    requestTls:{rejectUnauthorized:false}} );
setGlobalDispatcher(proxyAgent);

[eosfor]

in case of ZScaler or any other thing like that the ProxyAgent might not work, as it is not a real proxy. So wat you can also do is to set rejectUnauthorized: false in observablehq.config.js