Feature request: asset management

cyberglot commented 4 years ago

I know elm-pages uses webpack, so it would be nice to have access (even if limited) to it. Currently, I have a set of specific fonts that I've bought from a designer studio, so I need to serve it myself. Right now, it's being moved to /dist rather than a more convenient directory like /dist/assets/fonts.

The easy fix is to expect the elm-pages user to add all assets (including subdirectories) to /assets and it later gets compiled to /dist/assets. A more elegant solution would be to have it configurable, but the former is good enough.

cyberglot commented 4 years ago

Again, I am happy to send a Pull Request if you think that makes sense.

dillonkearns commented 4 years ago

Would it give you the desired behavior if you saved your font assets in static/assets/fonts? If I'm not mistaken, I think those files would then get copied over to dist/assets/fonts.

cyberglot commented 4 years ago

I didn't know that would be possible, I am using elm-pages-starter and I don't recall seeing it. I will test it right now.

cyberglot commented 4 years ago

It does work but the fonts get duplicated to dist/ anyway, but they don't get loaded on the website. I guess CSS requires full URLs? I will investigate this further.

cyberglot commented 4 years ago

It does work for now: it writes the CSS to use the fonts on the root, but it would be nice to have it in a better place. It's not a priority for me now but I can send a pull request to fix it anyway.

dillonkearns commented 4 years ago

I don't think I'm understanding exactly the situation, could you clarify?

I'll try to write my assumptions, but I could be totally off so please correct my misunderstandings:

Copy my-font.woff to my-elm-pages-site/static/assets/fonts/my-font.woff
You see the font copied to my-elm-pages-site/dist/assets/fonts/my-font.woff after running elm-pages build
You're including the asset in your style.css file? Or maybe you're including the font some other way?

cyberglot commented 4 years ago

I am importing the css file on index.js. All fonts are in /static/assets/fonts. The fonts folder indeed gets built into /dist/assets/fonts, however, all font files are dumped in /dist as well. CSS file gets embedded into index.html and the font-face property updates the url to /dist.

^ within /dist

^ css generated

^ original css

cyberglot commented 4 years ago

I have updated the webpack configs to send it all to /assets/, including fonts, images and the sort. Also added woff and woff2 font types to it. As I am already working on this, should javascript and css be minified and gzip'd as well? I believe having css as an outside file would let browsers cache it better for sites with multiple pages.

dillonkearns commented 4 years ago

Ah, yes I see now in the webpack config that ttf files are included so it makes sense that it's grabbing that asset and putting it into the top level.

https://github.com/dillonkearns/elm-pages/blob/e0f3eea3ad27326b65cc368dc80652c2b850bbf2/generator/src/develop.js#L270

It seems like this rule is obsolete, actually, and confusing, because there's already a rule to copy files as-is from the static folder:

https://github.com/dillonkearns/elm-pages/blob/e0f3eea3ad27326b65cc368dc80652c2b850bbf2/generator/src/develop.js#L130-L138

Let's remove this file-loader rule entirely, then it won't mess with the file path as you have it in your static directory:

https://github.com/dillonkearns/elm-pages/blob/e0f3eea3ad27326b65cc368dc80652c2b850bbf2/generator/src/develop.js#L269-L273

Minifying/Gzipping static files

As I am already working on this, should javascript and css be minified and gzip'd as well? I believe having css as an outside file would let browsers cache it better for sites with multiple pages.

My inclination is to keep the static folder as a folder entirely free of processing. It seems like users should have a way to just place files in that folder and the framework not mess with them in any way, not even gzipping or minifying the code.

My goal is that, for all of the basic Elm stuff you do, like importing a basic JS file that you use to wire up some ports, including an NPM package, etc., that stuff should be a zero-config sort of thing that "just works," including JS minification and bundling. But the static folder could be used for anything, and I don't want to make any assumptions about how users are using that. For example, maybe they don't want some code that's served up there mangled for some reason. Or maybe they don't want to spend the extra processing time gzipping an asset.

My thinking is that it seems reasonable to allow users to do any pre-processing as needed on files in static, and just keep it really simple and do a direct copy of those assets.

`/static` and `/images` output folder

I have updated the webpack configs to send it all to /assets/, including fonts, images and the sort.

Could you explain your thought process for this? Are you trying to avoid collisions and keep the top-level namespace from being polluted? And in that case, could it make sense to use a name that's less likely to cause other collisions? For example, you could imagine wanting to have a user-facing static folder named /assets?

I don't have answers to those questions and I'm not really sure how to come to a decision on them, I'm just trying to get the brainstorming process going and would love to see prior art on it to get some insight from other projects that have probably gone through these options and worked through feedback from users.

Looking at https://www.gatsbyjs.org/ in the network tab, it seems like they generate those assets into the /static/ folder 🤔 But also, they have hashed filenames so they're not going to collide because of that, maybe that's their reasoning?

Would love to hear your thoughts on all that, I'm trying to process all that.

dillonkearns commented 4 years ago

And thanks for working on this and talking through these questions with me! 🙏

cyberglot commented 4 years ago

My inclination is to keep the static folder as a folder entirely free of processing. It seems like users should have a way to just place files in that folder and the framework not mess with them in any way, not even gzipping or minifying the code.

I think it is reasonable, but add an option of making everything production-ready? I would be a little bummed to have to post-process everything because configuring webpack/etc is a nightmare. I apologise, I saw the closure-compiler for js and assumed the idea was to make everything prod-ready. Alternatively, it could do literal nothing, except from compiling elm to js and copying files, and let the user add their own processing step/plugin.

dillonkearns commented 4 years ago

Hey @cyberglot, sorry for the slow response getting back here!

The way I'm thinking about it, the goal is to make all the core building blocks production-ready. So for example, someone was asking on Slack recently about using Tailwind CSS with elm-pages. I suggested setting up PostCSS and the Tailwind CLI to output a css file, and then import that CSS file from the index.js in the elm-pages project. Josh went with that approach and he was really happy with it!

The way I'm looking at it, CSS is a core building block. PostCSS, Tailwind, etc. are not. As long as elm-pages provides a way to bundle up CSS nicely, you can do whatever processing steps you need to, and then import the result with elm-pages. I like the simplicity of that approach because it keeps elm-pages from getting bloated and keeps it focused on the core building blocks of a static Elm app.

I would be a little bummed to have to post-process everything because configuring webpack/etc is a nightmare.

I agree that webpack is a nightmare to configure. But there's no way to handle every possible use case automatically (otherwise webpack would just do that). So that's why I like the idea of this breakdown:

If you import some Elm, JS, or CSS in your elm-pages app, it will "just work".
If you include a static asset, it will just be copied over (no special cases to think about in case you don't want a certain file processed or want it processed in a special way... you can completely control it because these files are untouched)
If you include an image in elm-pages, I'd like having well optimized images to be a really nice experience, and that's what the image optimization API will be for. My plan is for there to be a programmatic API for describing how to optimize your images from within your elm-pages code, similar to Gatsby's image API.

cyberglot commented 4 years ago

I have thought about that and I realised that we should maybe have a tiny config type in Elm to so people can choose what they want to do, and I absolutely agree it should deal only with Elm, Js and Css. So people can choose if they want to inline or generate external Css files, for instance.

It would be nice if we could compile the layouts to HTML, cuz sometimes my website gets parsing errors from markdown -- just non-deterministic response due to network failure or the like.

dillonkearns commented 4 years ago

Alright, I've clarified some thoughts around this in this issue: https://github.com/dillonkearns/elm-pages/issues/148.

And the result of that thinking is now available in the new beta build: https://github.com/dillonkearns/elm-pages/blob/master/docs/7.0.0-elm-package-upgrade-guide.md#2---beta-build-command.

I'll close this discussion so we can get these discussions in one central place. I'm at the stage where I need input about this new approach in the beta build. I would love to gather thoughts, concerns, and ideas in the #148 thread, so please chime in there if you have any thoughts!

dillonkearns / elm-pages