inlining javascript vs using export

[tansaku]

Expected behavior

I would expect consistent behaviour from sprockets in production mode both on my local machine and on a cloud host like heroku

Actual behavior

For some reason, when in production mode on my local dev machine a js module with dependencies gets completely inlined along with its dependencies. However, the exact same setup on heroku inlines the dependency but then includes the following for the sub-dependencies:

export * from "topojson-client";
export * from "topojson-server";
export * from "topojson-simplify";

this leads to browser errors. More details at: https://stackoverflow.com/questions/56148334/rails-asset-pipeline-javascript-syntaxerror-unexpected-token-export

System configuration

• Sprockets version 3.7.2
• Ruby version 2.5.3

Example App

This is the PR for the branch that is being deployed to heroku and run locally and getting these disparate results:

https://github.com/AgileVentures/LocalSupport/pull/1069

I can try and set up a minimal app focused on precisely this issue, but was hoping just to get a pointer on what controls whether sprockets chooses to inline javascript dependencies vs using the export keyword ...

many thanks in advance

[tansaku]

I think I tracked down the place where js files are concatenated:

    # Internal: Accumulate asset source to buffer and append a trailing
    # semicolon if necessary.
    #
    # buf    - String buffer to append to
    # source - String source to append
    #
    # Returns buf String.
    def concat_javascript_sources(buf, source)

      if source.bytesize > 0
        buf << source

        # If the source contains non-ASCII characters, indexing on it becomes O(N).
        # This will lead to O(N^2) performance in string_end_with_semicolon?, so we should use 32 bit encoding to make sure indexing stays O(1)
        source = source.encode(Encoding::UTF_32LE) unless source.ascii_only?

        if !string_end_with_semicolon?(source)
          buf << ";\n"
        elsif source[source.size - 1].ord != 0x0A
          buf << "\n"
        end
      end

      buf
    end

in utils.rb, but for the life of me I can't work out why the concatenation process would be different on heroku rather than my local machine, unless heroku is setting some kind of sprocket directives ...

[schneems]

I replied in the Heroku ticket. I think the issue as described here is that sprockets does not know that the export * from "topojson-server"; is a special command and thinks it is regular JS. If you're not using webpacker then the files that are installed into node_modules might be es6, but they might also not have a .es6 file extension which sprockets needs to know to use babel to compile an es6 file.

I've honestly not seen node_modules used with sprockets, possibly due to this reason. I think that's the realm of one of the issues that webpack(er) is trying to solve.

[tansaku]

hi @schneems thanks so much for the feedback. I agree that the sprockets 3 we are using is treating that export command as regular js. I did try upgrading to sprockets 4 and renaming the file to use the .es6 extension, but that ballooned into a major refactoring of our front end.

What's confusing is the difference between how sprockets 3 (and whatever else is involved) behaves on heroku in production (i.e. node module dependencies added via export) and when I try and run in production locally and in develop (where the dependencies get inlined and topojson can be used just fine). Which makes me think that there's some setting in sprockets (or somewhere) that controls whether node module dependencies are inlined, or added via export. I did search through the sprockets code but could not work out where that happens exactly.

It sounds like with sprockets 3 you'd recommend using webpacker to try and sort this?

[schneems]

Here's a talk at RailsConf talking about how webpack(er) handles these export lines https://youtu.be/2v4ySqyua1s?t=848