Unable to use a module javascript file as a dependency - wrong MIME type error

[daattali]

In a widget's YAML file, when defining a JavaScript dependency, it doesn't seem to be possible to use a javascript file that's a module. The resulting page has a javascript console error message:

Failed to load module script: The server responded with a non-JavaScript MIME type of "text/plain". Strict MIME type checking is enforced for module scripts per HTML spec.

For a reproducible example:

1. Use scaffoldWidget() to create a plain htmlwidget

2. Create the following javascript file as inst/htmlwidgets/test.js:

   export function foo() {
     alert("bar");
   }

3. Modify the default yaml file to:

   dependencies:
     - name: test
       version: 1.0.0
       src: htmlwidgets
       script:
         - src: test.js
           type: module

[timelyportfolio]

@daattali I am nearly positive that this will not be possible (see article) since the viewer does not use a server. To make this work, I believe we would need either

1. build server in R with something like servr (not sure servr will serve modules correctly though) or through terminal (with another language, such as node http-server). In RStudio, viewer (see code) might be adapted to serve text/javascript correctly.
2. build step using node and webpack, rollup, or esbuild to bundle the dependencies. packr by @johncoene might be a good reference here. Other packages that incorporate a build step are leaflet, reactR, and g2r.

[daattali]

I wasn't expecting this to work in the RStudio Viewer as a standalone, but when run inside a shiny app there is a server created. I don't fully understand why this can't work within a shiny app - are you saying it won't work in shiny as well?

[timelyportfolio]

@daattali Shiny context I think should allow an easier fix. javascript files are delivered as "application/javascript" (see line and function definition). However, the html standard and the modules article both say we should use text/javascript.

mime::guess_type allows a mime_extra argument that will override the defaults but unfortunately we do not have access to that through shiny::getContentType. Here is an example.

> mime::guess_type("js.js",mime_extra = c("js" = "text/javascript"))
[1] "text/javascript"
> mime::guess_type("js.js")
[1] "application/javascript"

mime has some prebuilt mimeexttra. As a hack and test, I thought we could overwrite mimeextra, but this did not work so I am missing something. I also tried to fork mime and add text/javascript, but this also did not work.

library(shiny)

# overwrite mime:::mimeextra
# based on https://stat.ethz.ch/pipermail/r-help/2008-August/171321.html
hacked_mimeextra <- c(
  "js" = "text/javascript",
  mime:::mimeextra
)
m <- getNamespace("mime")
## attempt to replace package version with hacked version ...
unlockBinding("mimeextra", m)
assignInNamespace("mimeextra", hacked_mimeextra, ns="mime", envir=m)
assign("mimeextra", hacked_mimeextra, envir=m)
lockBinding("mimeextra",m)
rm(hacked_mimeextra) ## get rid of global copy

# minimal shiny app to test js delivered with `text/javascript`
shinyApp(ui = htmlwidgets::createWidget("widget", x=list()), server = function(input,output,session){})

I believe this leaves us with changing the handler somewhere in the shiny-middleware, but I am out of time for the moment and have little experience here. Perhaps someone could jump in here.

[timelyportfolio]

@daattali attaching this https://lea.verou.me/2020/05/todays-javascript-from-an-outsiders-perspective/ not for solution but for commiseration.

[timelyportfolio]

@wch or @schloerke might be able to confirm but the application/javascript might be coming from httpuv line. In this case, all my prior discussion about middleware might be meaningless.

A quick test with a forked httpuv seems to indicate this is where the change should be made https://github.com/timelyportfolio/httpuv/commit/ca409733e393a58e34c4e364e3a0d5407fc24b46.

[daattali]

Thanks for the article share, as someone who left the JS world around 2013, it was great to see I'm not the problem with all the problems I run into :)

You're right that the discrepancy between "text/javascript" vs "application/javascript" MIME types is confusing and there doesn't seem to be a standard unfortunately.

However, it's not actually as big of a problem as I thought, and that isn't the cause of the problem I was seeing.

In my original bug report, I left out the last step of how to actually run the widget and see the error. I was trying to run the widget by calling the widget's function from the command line - which as you noted does not involve a server. The error happens in that case, but I don't expect it to work because modules require a server.

I made a mistake: if you run the htmlwidget within a shiny app, it does work (even though the javascript is being served as the "wrong" mime type). However, it only works when the htmlwidget's output is defined in the UI and doesn't work when it's in a renderUI - but this is because of a different issue https://github.com/rstudio/shiny/issues/3345

So bottom line: the wrong mime type for JS modules is indeed being used, but htmlwidgets do work out of the box in shiny.

For completion: The following code does work when run after the steps described in the original post:

library(shiny)
ui <- fluidPage(
  fooOutput("test")
)
server <- function(input, output, session) {
  output$test <- renderFoo(foo("test")) 
}
shinyApp(ui, server)

But this doesn't work:

library(shiny)
ui <- fluidPage(
  uiOutput("out")
)
server <- function(input, output, session) {
  output$out <- renderUI({ fooOutput("test") })
  output$test <- renderFoo(foo("test")) 
}
shinyApp(ui, server)

[daattali]

I'll leave this issue open just so that if someone from RStudio wants to discuss whether these files should be served as text/javascript or application/javascript, you have a context here. Feel free to close.

[schloerke]

So this is a fun one!

---

In favor of text/javascript:

• https://html.spec.whatwg.org/#application/javascript

In favor of application/javascript:

• https://www.w3schools.com/tags/att_script_type.asp
  • which leads to https://www.iana.org/assignments/media-types/application/javascript
    • IANA also says https://www.iana.org/assignments/media-types/text/javascript is "OBSOLETE"

These two sources are at odds.

It seems like WHATWG is trying to push for text/javascript as all browsers support it even though they acknowledge that it is obsolete. https://html.spec.whatwg.org/multipage/infrastructure.html#dependencies

This article (https://2ality.com/2011/08/javascript-media-type.html) does a good explanation on why WHATWG wants to use text/javascript.

Moral of the story: if you want IE8 to work DO NOT use "application/javascript" for your JS files' MIME type

IE8 has been end of life for a long time now (2016-01-16). So I do not think that is a fair consideration anymore.

---

My interpretation of text/* vs application/* is that text/* is for people to read and application/* is for machines to work with.

For purposes of shiny, if a JavaScript file is provided, it is a fair assumption that the JavaScript file should be executed... giving more ammo towards using application/javascript.

---

mime R package (which plumber uses) has application/javascript.

---

Given that WHATWG is only using text/javascript to be able to support very legacy browsers, I'd like to stick to application/javascript for purposes within the Shiny ecosystem.

[daattali]

Mozilla is also saying JS modules require the "text" version https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Modules

Not to refute anything you said, just to add another very reputable data point that makes this debate completely frustrating

Edit: they actually claim "text" for all javascript files https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/MIME_types/Common_types

[schloerke]

https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/MIME_types/Common_types references a proposal: https://datatracker.ietf.org/doc/draft-ietf-dispatch-javascript-mjs/ (plain text: https://www.ietf.org/archive/id/draft-ietf-dispatch-javascript-mjs-08.txt)

If the proposal becomes accepted, then

• text/javascript will move from OBSOLETE to COMMON
• application/javascript will move from COMMON to OBSOLETE

This would put IANA and WHATWG on the same page.

I'm guessing we'll eventually be moving back to text/javascript. (We'll need to change a handful of R packages when proposal is accepted.)

[daattali]

Or you could wait long enough until "text" becomes obsolete again :)

Thanks for finding that.

[zsigmas]

Thanks everyone! This thread has been really helpful.

@daattali I was wondering if you could elaborate a bit into how you use the foo function exported in test.js

I have been following what you mention but I can't find a way to access that function from inside the htmlwidget. If I include an import statement in the htmlwidget code I get an error as an import cannot be part of script that is not type module, which is expected. Basically I can include the module as a dependency, but I cannot use the exports from that module in the htmlwidget code as I cannot import them.

Can you point me to any repo where do you have this implemented, or have any more hints that you could share?

Thanks in advance!

[daattali]

I'm sorry but I really don't remember any of this anymore. I tried finding the repo where I was doing this and could not find it.

[zsigmas]

Thanks a lot anyway @daattali !