[FireMonkey] responseType in GM.xmlHttpRequest and GM.fetch

[ghpzin]

Need some help to figure out how reponseType is supposed to work for GM.xmlHttpRequest and GM.fetch.

On current FireMonkey version (2.59) example userscript:

example userscript

``` // ==UserScript== // @name test_xhr_fetch // @match *://example.com/* // ==/UserScript== console.log('START USERSCRIPT --- ' + GM_info.script.name); GM.fetch( 'https://cdn.sstatic.net/Sites/stackoverflow/Img/favicon.ico', {responseType: 'blob'}) .then(response=>{console.log("GM.fetch",response);}) .catch(error => console.error(error.message)); console.log('XHR start'); GM.xmlHttpRequest({ url:'https://cdn.sstatic.net/Sites/stackoverflow/Img/favicon.ico', responseType: 'blob', onload: function(response){console.log("onload",response);}, onerror: function(response){console.log("onerror",response);}, onabort: function(response){console.log("onabort",response);}, ontimeout: function(response){console.log("ontimeout",response);}, timeout: 1000 }) .then(() => {console.log('XHR end');}); ```

Output in Browser Toolbox console (console inside webpage doesn't write any related errors):

19:22:15.187 START USERSCRIPT --- test_xhr_fetch [test_xhr_fetch.user.js:6:9](user-script:FireMonkey/test_xhr_fetch/test_xhr_fetch.user.js)
19:22:15.198 XHR start [test_xhr_fetch.user.js:23:9](user-script:FireMonkey/test_xhr_fetch/test_xhr_fetch.user.js)
19:22:15.619 Uncaught DOMException: XMLHttpRequest.responseText getter: responseText is only available if responseType is '' or 'text'. [background.js:1004](moz-extension://6bf6d54b-42ea-4c97-a472-c43e21a1f123/content/background.js)
19:22:15.631 GM.fetch 
Object { headers: {…}, bodyUsed: false, ok: true, redirected: false, status: 200, statusText: "OK", type: "basic", url: "https://cdn.sstatic.net/Sites/stackoverflow/Img/favicon.ico", blob: Blob }
[test_xhr_fetch.user.js:11:28](user-script:FireMonkey/test_xhr_fetch/test_xhr_fetch.user.js)

The problem seems to be with provided responseType, but blob works with GM.fetch.

For GM.xmlHttpRequest and that url responseType options that work (console message is written with onload and response object):

• nothing (no provided responseType)
• '' (empty responseType)

Don't work:

• 'arraybuffer'
• 'blob'
• 'document'
• 'json'

'GM.xmlHttpRequest' Browser Toolbox console error for: 'arraybuffer', 'blob', 'document' and 'json'

``` Uncaught DOMException: XMLHttpRequest.responseText getter: responseText is only available if responseType is '' or 'text'. [background.js:1004](moz-extension://6bf6d54b-42ea-4c97-a472-c43e21a1f123/content/background.js) makeResponse moz-extension://6bf6d54b-42ea-4c97-a472-c43e21a1f123/content/background.js:1004 onload moz-extension://6bf6d54b-42ea-4c97-a472-c43e21a1f123/content/background.js:989 (Async: EventHandlerNonNull) xmlHttpRequest moz-extension://6bf6d54b-42ea-4c97-a472-c43e21a1f123/content/background.js:989 xmlHttpRequest moz-extension://6bf6d54b-42ea-4c97-a472-c43e21a1f123/content/background.js:979 process moz-extension://6bf6d54b-42ea-4c97-a472-c43e21a1f123/content/background.js:930 API moz-extension://6bf6d54b-42ea-4c97-a472-c43e21a1f123/content/background.js:811 apply self-hosted:2429 raw resource://gre/modules/ExtensionCommon.jsm:2516 wrapResponse resource://gre/modules/ExtensionChild.jsm:225 responses resource://gre/modules/ExtensionChild.jsm:194 map self-hosted:204 emit resource://gre/modules/ExtensionChild.jsm:194 recvRuntimeMessage resource://gre/modules/ExtensionChild.jsm:376 recvRuntimeMessage self-hosted:1273 _recv resource://gre/modules/ConduitsChild.jsm:82 receiveMessage resource://gre/modules/ConduitsChild.jsm:188 (Async: JSActor query) _send resource://gre/modules/ConduitsChild.jsm:65 _send resource://gre/modules/ConduitsParent.jsm:290 promises resource://gre/modules/ConduitsParent.jsm:329 map self-hosted:204 _cast resource://gre/modules/ConduitsParent.jsm:329 _cast self-hosted:1323 recvRuntimeMessage resource://gre/modules/ExtensionParent.jsm:362 AsyncFunctionNext self-hosted:783 (Async: async) recvRuntimeMessage self-hosted:1273 _recv resource://gre/modules/ConduitsChild.jsm:82 receiveMessage resource://gre/modules/ConduitsParent.jsm:450 (Async: JSActor query) _send resource://gre/modules/ConduitsChild.jsm:65 _send resource://gre/modules/ConduitsChild.jsm:115 _send self-hosted:1385 sendRuntimeMessage resource://gre/modules/ExtensionChild.jsm:347 sendMessage chrome://extensions/content/child/ext-runtime.js:73 callAsyncFunction resource://gre/modules/ExtensionCommon.jsm:1063 callAsyncFunction resource://gre/modules/ExtensionChild.jsm:730 callAndLog resource://gre/modules/ExtensionChild.jsm:701 callAsyncFunction resource://gre/modules/ExtensionChild.jsm:729 stub resource://gre/modules/Schemas.jsm:2859 xmlHttpRequest moz-extension://6bf6d54b-42ea-4c97-a472-c43e21a1f123/content/api.js:301 AsyncFunctionNext self-hosted:783 (Async: async) wrapFunction chrome://extensions/content/child/ext-userScripts-content.js:243 user-script:FireMonkey/test_xhr_fetch/test_xhr_fetch.user.js:24 inject resource://gre/modules/ExtensionContent.jsm:714 injectInto resource://gre/modules/ExtensionContent.jsm:468 AsyncFunctionNext self-hosted:783 (Async: async) loadContentScript resource://gre/modules/ExtensionProcessScript.jsm:397 ```

• 'text'

'GM.xmlHttpRequest' Browser Toolbox console error for 'text'

``` Uncaught DOMException: XMLHttpRequest.responseXML getter: responseXML is only available if responseType is '' or 'document'. [background.js:1009](moz-extension://6bf6d54b-42ea-4c97-a472-c43e21a1f123/content/background.js) makeResponse moz-extension://6bf6d54b-42ea-4c97-a472-c43e21a1f123/content/background.js:1009 onload moz-extension://6bf6d54b-42ea-4c97-a472-c43e21a1f123/content/background.js:989 (Async: EventHandlerNonNull) xmlHttpRequest moz-extension://6bf6d54b-42ea-4c97-a472-c43e21a1f123/content/background.js:989 xmlHttpRequest moz-extension://6bf6d54b-42ea-4c97-a472-c43e21a1f123/content/background.js:979 process moz-extension://6bf6d54b-42ea-4c97-a472-c43e21a1f123/content/background.js:930 API moz-extension://6bf6d54b-42ea-4c97-a472-c43e21a1f123/content/background.js:811 apply self-hosted:2429 raw resource://gre/modules/ExtensionCommon.jsm:2516 wrapResponse resource://gre/modules/ExtensionChild.jsm:225 responses resource://gre/modules/ExtensionChild.jsm:194 map self-hosted:204 emit resource://gre/modules/ExtensionChild.jsm:194 recvRuntimeMessage resource://gre/modules/ExtensionChild.jsm:376 recvRuntimeMessage self-hosted:1273 _recv resource://gre/modules/ConduitsChild.jsm:82 receiveMessage resource://gre/modules/ConduitsChild.jsm:188 (Async: JSActor query) _send resource://gre/modules/ConduitsChild.jsm:65 _send resource://gre/modules/ConduitsParent.jsm:290 promises resource://gre/modules/ConduitsParent.jsm:329 map self-hosted:204 _cast resource://gre/modules/ConduitsParent.jsm:329 _cast self-hosted:1323 recvRuntimeMessage resource://gre/modules/ExtensionParent.jsm:362 InterpretGeneratorResume self-hosted:1611 AsyncFunctionNext self-hosted:783 (Async: async) recvRuntimeMessage self-hosted:1273 _recv resource://gre/modules/ConduitsChild.jsm:82 receiveMessage resource://gre/modules/ConduitsParent.jsm:450 (Async: JSActor query) _send resource://gre/modules/ConduitsChild.jsm:65 _send resource://gre/modules/ConduitsChild.jsm:115 _send self-hosted:1385 sendRuntimeMessage resource://gre/modules/ExtensionChild.jsm:347 sendMessage chrome://extensions/content/child/ext-runtime.js:73 callAsyncFunction resource://gre/modules/ExtensionCommon.jsm:1063 callAsyncFunction resource://gre/modules/ExtensionChild.jsm:730 callAndLog resource://gre/modules/ExtensionChild.jsm:701 callAsyncFunction resource://gre/modules/ExtensionChild.jsm:729 stub resource://gre/modules/Schemas.jsm:2859 xmlHttpRequest moz-extension://6bf6d54b-42ea-4c97-a472-c43e21a1f123/content/api.js:301 AsyncFunctionNext self-hosted:783 (Async: async) wrapFunction chrome://extensions/content/child/ext-userScripts-content.js:243 user-script:FireMonkey/test_xhr_fetch/test_xhr_fetch.user.js:24 inject resource://gre/modules/ExtensionContent.jsm:714 injectInto resource://gre/modules/ExtensionContent.jsm:468 AsyncFunctionNext self-hosted:783 (Async: async) loadContentScript resource://gre/modules/ExtensionProcessScript.jsm:397 ```

Similar issues with GM.fetch and that url, responseType options that work:

• nothing (no provided responseType)
• 'text'
• 'blob'
• 'arrayBuffer' Don't work:
• 'json'
• 'formData' Both have same looking errors, inside webpage console (it's not from .catch as commenting it out still writes this error):

  An unexpected apiScript error occurred [test_xhr_fetch.user.js:12:27](user-script:FireMonkey/test_xhr_fetch/test_xhr_fetch.user.js)
  <anonymous> user-script:FireMonkey/test_xhr_fetch/test_xhr_fetch.user.js:12
  (Async: promise callback)
  <anonymous> user-script:FireMonkey/test_xhr_fetch/test_xhr_fetch.user.js:12

  Error in Browser Toolbox console:

  'GM.fetch' Browser Toolbox console error for 'json' and 'formData'

An unexpected apiScript error occurred for '"FireMonkey" (ID: firemonkey@eros.man, moz-extension://6bf6d54b-42ea-4c97-a472-c43e21a1f123/)': Error: message is not defined :: Async*fetch@moz-extension://6bf6d54b-42ea-4c97-a472-c43e21a1f123/content/api.js:270:46
Async*@user-script:FireMonkey/test_xhr_fetch/test_xhr_fetch.user.js:8:4
[ext-userScripts-content.js:359](chrome://extensions/content/child/ext-userScripts-content.js)
    handleAPIScriptError chrome://extensions/content/child/ext-userScripts-content.js:359
    wrapFunction chrome://extensions/content/child/ext-userScripts-content.js:256
    AsyncFunctionThrow self-hosted:787
    (Async: async)
    wrapFunction chrome://extensions/content/child/ext-userScripts-content.js:250
    <anonymous> user-script:FireMonkey/test_xhr_fetch/test_xhr_fetch.user.js:8
    inject resource://gre/modules/ExtensionContent.jsm:714
    injectInto resource://gre/modules/ExtensionContent.jsm:468
    AsyncFunctionNext self-hosted:783
    (Async: async)
    loadContentScript resource://gre/modules/ExtensionProcessScript.jsm:397

I could assume that if you provided wrong responseType for either of these functions it could throw exception or fail in some way, but it's probably not intended that it fails silently and doesn't get caught on onerror callback or throws exception you can catch from inside userscript. Thought maybe I am doing something wrong with it.

[erosman]

To start with, due to the isolation of userScript context, objects that can not be cloned can not passed.

AFAIK, formData is one of those that may not work due to above limitation. The blob might also be the same but I haven't worked on it to see how it works out.

I have reworked the response in some of the others to make them work.

GM.xmlHttpRequest is basically the same as xmlHttpRequest but sent from background script to avoid CORS issues. responseType will defaulted to text. (Firemonkey Help -> xmlHttpRequest))

Similarly, GM.fetch is basically the same as fetch but sent from background script to avoid CORS issues. However, it is not possible to send the response object back so I had to add responseType in order to handle the response in the background before sending it back. (FireMonkey Help -> fetch)

I could assume that if you provided wrong responseType for either of these functions it could throw exception or fail in some way, but it's probably not intended that it fails silently and doesn't get caught on onerror callback or throws exception you can catch from inside userscript.

Didn't you get anything in the FireMonkey Log?

[ghpzin]

Thank you for quick answer.

Didn't you get anything in the FireMonkey Log?

Nothing related to the script in question, just from other scripts.

Overall my question was mostly the usability of GM.xmlHttpRequest with responseType set to blob. Is it normal for it to fail that way or maybe I am doing something it's not intended to do.

GM.xmlHttpRequest is basically the same as xmlHttpRequest but sent from background script to avoid CORS issues.

This is one of the thing that I am still confused about as - if you run this:

let xhr = new XMLHttpRequest();
xhr.open("GET",'https://www.example.com');
xhr.responseType = 'blob';
xhr.addEventListener("load", function(){console.log('simple xhr:',this.response);});
xhr.send();

on https://www.example.com page, it works.

Thought this one fails:

GM.xmlHttpRequest({
  url:'https://www.example.com',
  responseType: 'blob',
  onload: function(response){console.log("onload",response);},
  onerror: function(response){console.log("onerror",response);},
  onabort: function(response){console.log("onabort",response);},
  ontimeout: function(response){console.log("ontimeout",response);},
  timeout: 1000
});

with: Uncaught DOMException: XMLHttpRequest.responseText getter: responseText is only available if responseType is '' or 'text'.

If it's a limitation with 'the isolation of userScript context' as you mentioned, then I guess I should only use GM.fetch if I need blob reponse. And it's okay to close the issue.

[erosman]

That might be a bug... let me test it and get back to you

[erosman]

It is a bug (oversight)...sorry about that... fixed for v2.60

[erosman]

Don't work: 'json' 'formData'

json works when I test it.

[ghpzin]

Don't work: 'json' 'formData'

json works when I test it.

Just tried it with a proper json response and GM.fetch returns response as expected. So this works:

GM.fetch(
  "https://duckduckgo.com/country.json",
  {responseType: 'json'})
  .then(response=>{console.log("GM.fetch",response);})
  .catch(error => console.error(error.message));

I guess it only fails that way if it's not a proper json reponse and you set responseType: 'json'. This fails:

GM.fetch(
  "https://duckduckgo.com",
  {responseType: 'json'})
  .then(response=>{console.log("GM.fetch",response);})
  .catch(error => console.error(error.message));

With this error in page console:

An unexpected apiScript error occurred [test_xhr_fetch.user.js:36:27](user-script:FireMonkey/test_xhr_fetch/test_xhr_fetch.user.js)
    <anonymous> user-script:FireMonkey/test_xhr_fetch/test_xhr_fetch.user.js:36
    (Async: promise callback)
    <anonymous> user-script:FireMonkey/test_xhr_fetch/test_xhr_fetch.user.js:36

And this one in Browser Toolbox console:

An unexpected apiScript error occurred for '"FireMonkey" (ID: firemonkey@eros.man, moz-extension://6bf6d54b-42ea-4c97-a472-c43e21a1f123/)': Error: message is not defined :: Async*fetch@moz-extension://6bf6d54b-42ea-4c97-a472-c43e21a1f123/content/api.js:270:46
Async*@user-script:FireMonkey/test_xhr_fetch/test_xhr_fetch.user.js:32:4
[ext-userScripts-content.js:359](chrome://extensions/content/child/ext-userScripts-content.js)
    handleAPIScriptError chrome://extensions/content/child/ext-userScripts-content.js:359
    wrapFunction chrome://extensions/content/child/ext-userScripts-content.js:256
    InterpretGeneratorResume self-hosted:1893
    AsyncFunctionThrow self-hosted:885
    (Async: async)
    wrapFunction chrome://extensions/content/child/ext-userScripts-content.js:250
    <anonymous> user-script:FireMonkey/test_xhr_fetch/test_xhr_fetch.user.js:32
    inject resource://gre/modules/ExtensionContent.jsm:714
    injectInto resource://gre/modules/ExtensionContent.jsm:468
    AsyncFunctionNext self-hosted:881
    (Async: async)
    loadContentScript resource://gre/modules/ExtensionProcessScript.jsm:397

If I change responseType in second request to: text, blob or arrayBuffer, then it works. formData also fails with An unexpected apiScript error occurred in second request, thought I haven't tried if it works as expected on proper formData reponse.

[erosman]

I guess it only fails that way if it's not a proper json response

That is understandable as JSON.parse() fails.

[ghpzin]

For that case I think my expectation would be for that error to be catchable or have some other way to check that it happened from the inside of written userscript. As currently it's not that easy to tell that returned response wasn't of the specified type and parse failed (other than apiScript error from console while running it on the page). If it's hard to change, I think some documentation warning about the possibility of that error while specifying wrong responseType for GM.fetch could be good too.

[erosman]

I can add a note to the Help. I will also see how the code can be improved.

[erosman]

I have updated the GM.fetch for v2.60

Now in case of the aforementioned parse errors:

• Errors gets logged in FireMonkey log
• Return value would be the error message

GM.fetch(
  "https://duckduckgo.com/country.json",
  {responseType: 'json'})
  .then(response => console.log("GM.fetch",response);})
  .catch(error => console.error(error.message));

// GM.fetch JSON.parse: unexpected character at line 1 column 1 of the JSON data

[erosman]

v2.60 uploaded Let me know how it goes.

[ghpzin]

Thank you. For v2.60 my initial use case (using GM.xmlHttpRequest with responseType:"blob") seems to be fixed. Also tested GM.fetch with responseType:"json" with non-json response and it returns string with value "JSON.parse: unexpected character at line 1 column 1 of the JSON data" (instead of object type that returns with proper response), so it seems to be pretty easy to catch from userscript.

[erosman]

The error should also log to the Log.

[ghpzin]

Yes it does log in FireMonkey log: fetch https://www.example.com/ ➜ JSON.parse: unexpected character at line 1 column 1 of the JSON data

The only issue I could see with v2.60 is that GM.fetch returns string with error for these cases instead of promise reject with error message inside (so you can .catch() it). Thought not sure if it's better that way.

[erosman]

I tried to reject but it wasn't passed to the userscript. There are multiple stages involved and that reject comes from try...catch. I will try again to see if I can convert the error back to a Promise.

There is another issue with returning a Promise.reject(). What happens when userscript uses await?

const response = GM.fetch('https://duckduckgo.com/', {responseType: 'json'});

[ghpzin]

I think if it returns promise reject the regular way, it would behave the same way as regular fetch. So you have to catch exception from it with .catch()/try{} catch(){} or it becomes uncaught exception

Examples

For example, when trying to fetch wrong url: ```js async function test() { let awaitFetch = await fetch('https://www.example.com12345') .catch(error => console.error("fetch .catch:", error)); console.log("awaitFetch", awaitFetch); } test(); ``` log: ``` fetch .catch: TypeError: NetworkError when attempting to fetch resource. awaitFetch undefined ``` Or when you manually return `Promise.reject` for some reason: ```js async function test2() { let awaitFetch = await fetch('https://www.example.com/404') .then(response => { console.log("fetch response:", response); if (!response.ok) { return Promise.reject("fetch error, response.ok=false"); } return response; }) .catch(error => console.error("fetch .catch:", error)); console.log("awaitFetch", awaitFetch); } test2(); ``` log: ``` fetch response: Response { type: "basic", url: "https://www.example.com/404", redirected: false, status: 404, ok: false, statusText: "Not Found", headers: Headers(12), body: ReadableStream, bodyUsed: false } fetch .catch: fetch error, response.ok=false awaitFetch undefined ```

Overall about how GM.fetch behaves with responseType, wouldn't it be easier to just copy behavior from regular fetch as much as possible? So don't specify responseType, don't return json and similar properties in response, just functions to parse response. Not sure if it's possible due to it being a firefox extension, but in regular js I would just return object with function that returns promise of parsed response, so if user wants to parse it, it's on him to call response.json(), write into .catch() other options (like calling .text() instead if this one failed), etc. Iirc that would be similar to how response from fetch looks like:

Example

```js function testFetch() { return new Promise((resolve, reject) => { try { let response = { _text: 'd1asd123', text: function() { return this._text; }, json: function() { return new Promise((resolve, reject) => { let json; try { json = JSON.parse(this._text); resolve(json); } catch (e) { reject({ response: this, error: e }); } }); } }; resolve(response); } catch (e) { console.error("extension: reject", e); reject(e); } }); } testFetch() .then(response => { console.log("user: response", response); console.log("user: response.json()"); return response.json(); }) .catch((e) => { console.log("user: .catch", e); }); ``` log: ``` user: response Object { _text: "d1asd123", text: text(), json: json()} user: response.json() user: .catch Object { response: {…}, error: SyntaxError } ```

[erosman]

Overall about how GM.fetch behaves with responseType, wouldn't it be easier to just copy behavior from regular fetch as much as possible?

Sadly, it is not possible. There are 3 separate isolated layers involved, userscript in userScript context, API script in content context & background script in browser context.

Process of GM.fetch is following:

• GM.fetch is in the API script
• GM.fetch sends an async message to background script to perform fetch
• Background script performs standard fetch with the given parameters
• It is not possible to send back the fetch response from background script to the API script as it causes error
• An object is created in the background script and sent back to the API script
• API script then clones that object and sends it to the userscript

[ghpzin]

Then I am not sure if changing current behavior from v2.60 to an error message in reject would be better or worse. In practice I don't think it would change anything as if I understand correctly you won't be able to change it so that it's possible to get both response and error message on this kind of error.

[erosman]

I will try and see what can be done.