search

sintaxi / harp

Static Web Server/Generator/Bundler
http://harpjs.com
5k stars 342 forks source link

Support arbitrary JSON files. #335

Closed zeke closed 4 years ago

zeke commented 10 years ago

_data.json is a great convention, but why not open it up to any JSON file? The path and filename could becomes the path to that object in the template. For example ./foo/bar.json with content { "baz": 1} would be available in the template at public.foo.bar.baz.

Crazy?

sintaxi commented 10 years ago

I like the idea though I should explain how we came to using _data.json.

Back story

With each requests that comes in, Harp walks the project tree to build the _public object that gets passed into the template. This does two important things...

  1. builds the _contents object which lists all the publicly accessible files in the tree
  2. builds the _data objects in each level of the tree based on the _data.json files.

The big advantage to limiting to only _data.json files is performance. Because we build data using only _data.json files, those are the only files we need to open when walking the project tree. Making this initial step much, much faster. FWIW - this is the same reason harp doesn't currently support front-matter (we would have to open every file in the project on every request).

note - when running in production this walking of the tree only happens when the server is fired up which is why it needs to be restarted if there are any changes to the project.

Moving forward

With some of the upcoming changes to Harp, we have an opportunity to improve on this without sacrificing performance. To achieve this we will have to split the work harp does into two stages. Things that happen when a project file changes, and things that happen when a request comes in.

Im suggesting that we have the building of the _public object happen in the background using some sort of watch utility. When the project changes, we expire the dirty parts of the cache and rebuild the _public object. And then when a request comes in we only need to render the requested file without needing to re-walk the project or rebuilding the world (as jekyll does it). This should set the stage for adding the following features without sacrificing performance...

In fact, our performance may even improve because harp will be doing less work on each request. It may also lay the framework to support other types of data files like you are suggesting. thoughts?

ir-g commented 9 years ago

@zeke @sintaxi

I would love to see support for any JSON file, and I also think that it could potentially also solve problems from #226 - where JSON data is somehow needed in JS, and with this, it could be ajax'd in. (JSON would still be public?)

One consideration, if worried about speed, it to set up support for a folder called _data and any data in that folder would be available within harp. Thoughts?

maxdeviant commented 9 years ago

:+1: