AlttiRi
commented
3 years ago By the way, this guy tried to do GM_fetch over GM_xmlhttpRequest:
https://github.com/mitchellmebane/GM_fetch
He uses window.fetch polyfill as a base.
But as I said earlier it's impossible to implement streaming for fetch with XMLHttpRequest. So, it's just a wrapper for more convenient using like Axios that has no benefits of the streaming.
Here is my simplified wrapper:
The code
```js // Using: const response = await fetch(url); const {status, statusText} = response; const lastModifiedSeconds = response.headers.get("last-modified"); const blob = await response.blob(); // The simplified `fetch` — wrapper for `GM.xmlHttpReques` async function fetch(url) { return new Promise((resolve, reject) => { const blobPromise = new Promise((resolve, reject) => { GM.xmlHttpRequest({ method: "get", url, responseType: "blob", onload: async (response) => resolve(response.response), onerror: reject, onreadystatechange: onHeadersReceived }); }); blobPromise.catch(reject); function onHeadersReceived(response) { const { readyState, responseHeaders, status, statusText } = response; if (readyState === 2) { // HEADERS_RECEIVED const headers = parseHeaders(responseHeaders); resolve({ headers, status, statusText, arrayBuffer: () => blobPromise.then(blob => blob.arrayBuffer()), blob: () => blobPromise, json: () => blobPromise.then(async (blob) => JSON.parse(await blob.text())), text: () => blobPromise.then(blob => blob.text()), }); } } }); } function parseHeaders(headersString) { class Headers { get(key) { return this[key.toLowerCase()]; } } const headers = new Headers(); for (const line of headersString.split("\n")) { const [key, ...valueParts] = line.split(":"); // last-modified: Fri, 21 May 2021 14:46:56 GMT if (key) { headers[key.trim().toLowerCase()] = valueParts.join(":").trim(); } } return headers; } ```BTW, my special version of fetch resolves correctly on HEADERS_RECEIVED event, and then response resolves on load event, while fetch.js polyfill's fetch resolves only on load event, what's not OK.
derjanb
tophf
TwoLeaves
Implement
GM.fetch/GM_FetchI have already talked about
GM.xmlhttpRequestandfetchhere: https://github.com/Tampermonkey/tampermonkey/issues/1050 But it makes sense to write this separate issue, since there was talking aboutfetch: trueoption ofGM.xmlhttpRequest, but this issue is about a new function.Tampermonkey currently has
fetch: trueas a Beta feature, but that looks no very useful feature since it usesfetchjust only in the background script, the other moments ofGM.xmlhttpRequestwork in the usual way.I suggest to add a new
GM.fetchfunction that will be written from zero, has the same API asfetchand use the benefist of streaming, which impossible to do withXMLHttpRequest.Compatible with Fetch API
Like
GM.xmlhttpRequestit will allow to bypass CORS limitation, but API ofGM.fetchshould be compatible with the common Fetch API.One of annoying things of
GM.xmlhttpRequestwhen you use it the first times is that while it names likeXMLHttpRequest, but API is not the same. API ofGM.fetchshould be compatible with the common Fetch API. It will make writing of script to easier since you do not use some other API, and you can reuse the existing code that was written forfetch.So, using any existing code that written for
fetchshould work withGM.fetchtransparently. For example wrappingReadableStream(response) for progress indication:The example
```js let response = await fetch(url); response = loadingProxyResponse(response, onLoadingProgressLog); // ... // just logs the loading progress function onLoadingProgressLog(receivedLength, contentLength) { const percentage = receivedLength / contentLength; const text = (percentage * 100 + "").substring(0, 4) + " %"; console.log(text); } function loadingProxyResponse(response, onProgress) { const contentLength = parseInt(response.headers.get("Content-Length")); let receivedLength = 0; const reader = response.body.getReader(); const readableStream = new ReadableStream({ async start(controller) { while (true) { const {done, value} = await reader.read(); // value is Uint8Array if (done) { break; } receivedLength += value.length; try { onProgress(receivedLength, contentLength); } catch (e) { console.error("onProgress error:", e); } controller.enqueue(value); } controller.close(); reader.releaseLock(); }, cancel() { reader.cancel(); } }); return new Response(readableStream, {headers: response.headers}); } ```Streaming
Okay, compatibility with Fetch API is good, but it not the main thing that I want. You can write some wrapper for
GM.xmlhttpRequestthat will look likefetcheven now. But with just a wrapping you can't do the main feature offetch— it's streaming.The streaming is the core difference between
XMLHttpRequestandfetch. When you download some data withXMLHttpRequestthe entire data stores in the memory until the request is finished and data is handled. In case of a userscript manager extension the consuming of memory is double. Since the data stores both in content/web script and in the background script. But when you usefetchyou can consume data from the stream as soon as the data received and if you do not need the data anymore the garbage collector can destroy them.In case of TamperMonkey you do not need to store the downloaded data in the background script's context. You just pass them to context script as they are received and GC collect them. So it already notable reduce memory consuming — two times. Also with streaming you can use use the such approach: https://github.com/jimmywarting/StreamSaver.js In this case there no any washing of the memory for storing data at all when you download files. Memory consuming is O(1) for any file of any size.
So this will make userscripts more memory optimised. Notable optimised.
Also I think it would more easier to implement (and support it later) a new function from zero, than trying to add the
fetch's functional to the existingGM.xmlhttpRequestas a mode (fetch: true) which you currently do. The same thing would actual for other userscript managers.