feature: write all router paths to disk

[yoshuawuyts]

It might be cool if budo's routes were able to be persisted to disk. This would allow users to prototype using a live-reloading environment, and once they're happy with the results store the output to disk and deploy it to GH-pages.

Currently the following modules are available to do this:

• wayfarer - generic radix-trie router
• wayfarer-to-server - server-side wrapper for wayfarer adding HTTP method support
• wayfarer-to-fs - write wayfarer-to-server paths to disk

Usage looks like this:

const toServer = require('wayfarer-to-server')
const toFs = require('wayfarer-to-fs')
const wayfarer = require('wayfarer')

const router = toServer(wayfarer())
router.on('/', {
  get: (req, res) => fs.createReadStream(__dirname + '/index.html').pipe(res))
})

toFs(router, __dirname + '/dist', (err) => {
  if (err) throw err
})

What do you think of adding such functionality to budo?

[mattdesl]

How might this look with regard to budo? I'm trying to envision how a small GH pages site/demo like this could use wayfarer-to-server to avoid having to manually write the index.html and browserify + uglify the bundle.js (unless I'm mistaken about the goals of this?).

Can you give example of other routes that might be worth persisting?

[yoshuawuyts]

Ah yes, I should've been a bit more clear in that respect. What you're outlining is exactly the use case for this.

budo currently server index.html and bundle.js files, which is neat when prototyping. However, when you're done prototyping it'd be cool if those files could be written to disk and uploaded to GH-pages using the same tool (as opposed to pulling in grunt or something equally gross to do the job).

A simplified config could look like:

./router.js

const toServer = require('wayfarer-to-server')
const html = require('simple-html-index')
const wayfarer = require('wayfarer')
const watchify = require('watchify')
const uglify = require('uglifyify')

const router = toServer(wayfarer())
module.exports = router

router.on('/', {
  get: (req, res) => html().pipe(res))
})

const b = watchify('./index.js')
  .plugin(uglify())

router.on('/bundle.js', {
  get: (req, res) => b.bundle().pipe(res)
})

./to-fs.js

const toFs = require('wayfarer-to-fs')
const router = require('./router')
const path = require('path')

const directory = path.join(__dirname, 'dist')
const overrides = { '/': '/index.html' }

toFs(router, directory, overrides, (err) => {
  if (err) throw err
})

[mattdesl]

This looks good; the only thing is that typically you end up with different dev and prod configurations; like using installify during development or uglify only during production. Any ideas how that would be handled?

Also, I guess how would it manifest itself on the end-user? How would these two lines be replaced? As it states in uglifyify readme, you generally still want to run uglifyjs on the final bundle.

[yoshuawuyts]

budo allows flag passing to the underlying browserify instance. This could still be done when writing to disk:

  "scripts": {
    "start": "budo index.js:bundle.js --live -- -t babelify | garnish",
    "build": "budo --write index.js:bundle.js -t babelify -t uglifyify -o ./dist"
  }

Does this answer your question?

edit: yeah, this would mean that direct unix pipes don't work, but luckily practically all bundle actions are available as browserify transforms.

edit: oh, you're right about the uglifyify output. I usually don't minify (gzip works pretty well), but i think this is a pretty specific case. In this case it might be best to have a separate optimization step that is called after writing files to disk. e.g. one could do:

  "scripts": {
    "start": "budo index.js:bundle.js --live -- -t babelify | garnish",
    "build": "budo --write index.js:bundle.js -t babelify -t uglifyify -o ./dist && npm run optimize",
    "optimize": "uglify -s dist/bundle.js | sponge | dist/bundle.js
  }

I admit it's not ideal, but I think it's somewhat of a corner case. Alternatively we could use a flag to pipe some of the files through to stdout (which would mean extending wayfarer-to-fs):

$ budo --write --raw index.js -t babelify -t uglifyify -o ./dist | uglify > dist/bundle.js

but I think this approach creates more problems than it solves.

[mattdesl]

I think browserify options could still be passed after the full stop, and maybe the flag could look something like this:

budo index.js:bundle.js --write=./static/ -- -t babelify -t uglifyify

Could be named --write or --save.

I actually think this feature would be pretty nice.

[andyinabox]

+1!

[mattdesl]

More thoughts:

• An ideal build step should include some or all of uglify-js, envify (or loose-envify), bundle-collapser, factor-bundle, dead code elimination, etc. I'd rather just build a separate tool like budo-build than integrate all these opinions/bloat into budo itself.
• Adding browserify as a direct dependency to a module has some subtle issues:
  • The browserify dependency may not always dedupe to the same version of browserify used by watchify. This means you might end up with a slightly different dev + build bundle.
  • It introduces a maintenance burden; the module always lag behind latest version of browserify and will require me to constantly update the package.json.

The way I've solved this for my projects is to use quick-build which installs latest browserify and uglify-js and adds a build script to the package.json. This also integrates nicely with my ghpages script. All together, I can set up a demo in a couple commands, and then build + deploy it when necessary with another couple commands.

Anyways, this is still unresolved since I'm open to discussing it further, but hopefully we can find a way that isn't too complex/painful. :smile:

[timwis]

Pardon the necro but I just wanted to chime in to point out that it seems the hold up here is around optimization for production; I'd like to suggest that there's a need for output/export/write in a prototyping tool -- to share your prototype. Yes, you can just use browserify index.js -o bundle.js but what about the index.html file that budo generates? I usually use a dist directory so that I only deploy what needs to be deployed (bundle.js, index.html, styles.css).

Currently, my build script looks something like browserify index.js -o dist/bundle.js && cp index.html dist/ && cp styles.css dist/. It seems silly to use curl to pull these files into the dist directory when they could just be written to disk instead of served over an http server.

Generating a dist directory and sharing it around, for me at least, comes before optimization is necessary. I can always upgrade my build script later.

bankai build will help, but budo does have some pretty nice features :-/

[dy]

Can't wait for the feature, in the meantime https://github.com/dy/budo

[mattdesl]

FWIW I implemented this in canvas-sketch and it introduces a number of new considerations:

• Introduces opinions about compression: uglify? bundle collapser? envify? minify HTML? etc... Also new flags like --no-minify
• Writes multiple files (HTML + JS), so this can't easily rely on stdout
• Can run into collision if your output JS file (say index.js) matches your input file. Budo's existing --serve option could help avoid collision but perhaps it should be renamed to --js or --src for clarity.
• Ideally a flag that concats everything into a single inline HTML file would be nice. I use this to produce single-file static sketches that can be opened directly.
• In canvas-sketch you have to specify an input HTML file if you want something other than the default. In budo currently it uses the first HTML in your static directory — how is that handled during build? Budo's handling of HTML mixes src/dist, canvas-sketch clearly separates the source HTML from the dist HTML.
• If we are emitting HTML files, we'll either need to detect and transform the script tag (to rename its src according to the CLI flag, or to inline it) or support templating within the HTML. In canvas-sketch you can use {{entry}} which will be replaced by the build's JS/script tag. More opinions! :smile:

This feature might be a bit awkward if implemented in budo currently, as budo wasn't really designed with it in mind, and it really increases the opinions and surface area of budo.

I'm open to other creative solutions, though.

PS:

Forking budo is actually a fine solution, or even creating a new npm module on top of budo. Budo is pretty stable, and probably won't be receiving many more changes as I'd rather keep it bare-bones. The canvas-sketch-cli tool is built directly on top of budo so you can see how it's used to build more bespoke and reusable build chains.