replace `request` with `fetch` (can be done with Ai)

[coolaj86]

As a matter of cleanup, we'd like to drop the @root/request dependency. Here's the list of files that still use it:

This is a task that will need human verification, but can reliably be done via LLM (ChatGPT 4o or o1, Claude, etc) by pasting in the block of code that uses request and giving a promp similar to the following:

provide the equivalent code using fetch and make note of any changes that may have nuance differences

grep -F 'request(' | cut -d: -f1 | sort -u

flutter/releases.js
go/releases.js
gpg/releases.js
iterm2/releases.js
macos/releases.js
mariadb/releases.js
node/releases.js
terraform/releases.js
zig/releases.js

Example Prompt Sequence

1. Original Prompt

provide the equivalent code using fetch and make note of any changes that may have nuance differences

let resp = await request({
    url: `https://storage.googleapis.com/flutter_infra_release/releases/releases_${osname}.json`,
    json: true,
});

Reply from ChatGPT 4o (NOT Correct)

Note: This is NOT correct. It ignores headers and

let response = await fetch(`https://storage.googleapis.com/flutter_infra_release/releases/releases_${osname}.json`);
let resp = await response.json();

if (!response.ok) {
    throw new Error(`HTTP error! Status: ${response.status}`);
}

2. Second Prompt

note: we provide a larger block with more context for how resp is used, and clarify that request is @root/request:

Provide the equivalent code to the use of the @root/request below (similar to node request with promise support) using fetch. Make note of any changes that may have nuance differences between the two:

    let resp = await request({
      url: `https://storage.googleapis.com/flutter_infra_release/releases/releases_${osname}.json`,
      json: true,
    });

    let osBaseUrl = resp.body.base_url;
    let osReleases = resp.body.releases;

Reply from ChatGPT 4o (Correct, and had good comments)

let response = await fetch(`https://storage.googleapis.com/flutter_infra_release/releases/releases_${osname}.json`);

if (!response.ok) {
    throw new Error(`HTTP error! Status: ${response.status}`);
}

let resp = await response.json();

let osBaseUrl = resp.base_url;
let osReleases = resp.releases;

3. A Different Approach

This is the documentation for @root/request:

• https://www.npmjs.com/package/@root/request
• https://github.com/therootcompany/request.js

Write a single request function that wraps fetch in a way that works similarly to @root/request in this scenario:

let resp = await request({
    url: `https://storage.googleapis.com/flutter_infra_release/releases/releases_${osname}.json`,
    json: true,
});

console.log(resp.body.base_url);
console.log(resp.headers['Content-Type']);
console.log(resp.ok);
console.log(resp.statusCode)

GPT 4o Response (Good)

async function request(options) {
    const { url, json } = options;

    const response = await fetch(url, {
        method: 'GET',
        headers: {
            'Accept': 'application/json',
        },
    });

    const headers = {};
    response.headers.forEach((value, key) => {
        headers[key] = value;
    });

    const body = json ? await response.json() : await response.text();

    return {
        body,
        headers,
        ok: response.ok,
        statusCode: response.status,
    };
}

// Example usage
let resp = await request({
    url: `https://storage.googleapis.com/flutter_infra_release/releases/releases_${osname}.json`,
    json: true,
});

console.log(resp.body.base_url);
console.log(resp.headers['content-type']);
console.log(resp.ok);
console.log(resp.statusCode);

4. On Second Thought...

The LLM business is taking longer than just doing it the normal way.

Probably just better to use normal intelligence on this one, with cues from how fetch is already being used.

async function fetchJSON({ url }) {
    let resp = await fetch(url, {
        method: 'GET',
        headers: { 'Accept': 'application/json' },
    });

    let headers = Object.fromEntries(resp.headers);
    let body = await response.json();
    let result = {
        ok: resp.ok,
        statusCode: response.status,
        headers: headers,
        body: body,
    };
    return result;
}

// Example usage
let resp = await fetchJSON({
    url: `https://storage.googleapis.com/flutter_infra_release/releases/releases_${osname}.json`,
});

console.log(resp.body.base_url);
console.log(resp.headers['content-type']);
console.log(resp.ok);
console.log(resp.statusCode);

rg -A 10 ' fetch\('
julia/releases.js
20:  let resp = await fetch(
21-    'https://julialang-s3.julialang.org/bin/versions.json',
22-    {
23-      headers: {
24-        Accept: 'application/json',
25-      },
26-    },
27-  );
28-  let buildsByVersion = await resp.json();
29-
30-  /*

_common/githubish.js
47:  let resp = await fetch(url, opts);
48-  if (!resp.ok) {
49-    let headers = Array.from(resp.headers);
50-    console.error('Bad Resp Headers:', headers);
51-    let text = await resp.text();
52-    console.error('Bad Resp Body:', text);
53-    let msg = `failed to fetch releases from '${baseurl}' with user '${username}'`;
54-    throw new Error(msg);
55-  }
56-
57-  let respText = await resp.text();

_common/githubish-source.js
47:  let resp = await fetch(url, opts);
48-  if (!resp.ok) {
49-    let headers = Array.from(resp.headers);
50-    console.error('Bad Resp Headers:', headers);
51-    let text = await resp.text();
52-    console.error('Bad Resp Body:', text);
53-    let msg = `failed to fetch releases from '${baseurl}' with user '${username}'`;
54-    throw new Error(msg);
55-  }
56-
57-  let respText = await resp.text();
--
119:  let resp = await fetch(dist.download, {
120-    method: 'HEAD',
121-    redirect: 'follow',
122-    signal: abortCtrl.signal,
123-  });
124-  let headers = Object.fromEntries(resp.headers);
125-
126-  // Workaround for bug where METHOD changes to GET
127-  abortCtrl.abort();
128-  await resp.text().catch(function (err) {
129-    if (err.name !== 'AbortError') {

[MichalTirpak]

Hello, is this issue still active, if so could i work on it?

[coolaj86]

@MichalTirpak Yes. Go ahead. I just updated the list.

Use this branch rather than main as the basis for your work: https://github.com/webinstall/webi-installers/pull/909 (or if that's merged by the time you get to it, just use main) (that's now in main)