💡 RFC: content bundles aka assets collocated in source

innermatrix commented 2 years ago

Background & Motivation

I want the following two things:

If page X references asset Y, I want the ability to put X and Y next to each other in my source tree
I want the URL of Y to be pretty

Other site generators I've used this have this, although they call it different things. For example, Hugo calls it page bundles. The way it works is that

I can put page/index.md and page/asset.xyz next to each other in the source tree
When the source is built, they end up a site/page/ and site/page/asset.xyz URLs

This doesn't work in Astro right now because:

If I put an asset in /src/pages, next to the page source, then I get a compile error about file asset.xyz.js not being found.
If I put an asset in /src/elsewhere, then it's not next to the page source and it ends up with _astro in its URL
If I put an asset in /public, then it's not next to the page source, but it doesn't end up with _astro in its URL

I have also considered putting both the page markdown file and the asset in /public, but then I am polluting my built site with all those md files, and I still have to put something in src… which could be a catch-all /src/pages/[page].astro which would track down the markdown file and the assets, but at that point I basically have to implement my entire site in that one component. Anyway, it gets messy.

Proposed Solution

Possible solutions

My preferred solution is for astro to allow assets directly in /src/pages and for those assets to end up at a clean URL (without _astro).

Alternatives considered

See above re other things I tried

Risks, downsides, and/or tradeoffs

Increased complexity of finding files to compile in /src/pages
Increased complexity of static asset builder

Open Questions

Detailed Design

No response

Help make it happen!

[ ] I am willing to submit a PR to implement this change.
[ ] I am willing to submit a PR to implement this change, but would need some guidance.
[ ] I am not willing to submit a PR to implement this change.

RafidMuhymin commented 2 years ago

The current behavior of copying the src directory to _astro/src with transformations is like copying the public directory to the root directory without transformations. I agree with @innermatrix that this behavior should be changed in favor of the standard behavior in a future version.

aFuzzyBear commented 2 years ago

This issue has came up a couple of times in the support over the past few days, I do feel that is does warrant a discussion.

Lets consider the following layout diagrams for the src/pages/ :

src/
pages/
|---blog/
     |--img.png
     |--post.md
     |---[posts]/
           |---[...slug].astro
           |---first-post.md
           |---assets/
                |---first-post-diagram.jpg/
                |---first-post-hero.jpg
                |---{....images}
      ||   |--second-post.md
           |--second-post-hero.jpg
|---products/
     |---[...id].astro
     |---sales-page.astro
     |---sales-page-banner.jpg
     |--sales-page-functions.{ ts| js }
     ....
|--index.astro
|--about.astro
|--contact.astro

The above file arrangement, breaks the current model of transformations as mentioned by @RafidMuhymin in the earlier post. However it does so with the best intentions so to speak, this format of allowing for transformations of files to be scoped not based on parent directories ie:src and public but allow for directory-level scoping of associated files and assets,

This would allow for greater parity between the various headless and standard CMS tools along with providing the user greater freedom for setting up and maintaining their projects as they see fit,

I do think that this feature would help to augment and extend the power of Astro's file-based routing,

FredKSchott commented 2 years ago

I believe this was already agreed to in another RFC, that anything starting with _ would be ignored inside of pages/. I believe that that would serve your need here! But I cannot for the life of me find that RFC, so maybe I'm imagining it, or it was only mentioned as a part of a larger convo? If that's the case, then I think revisiting this again in our next RFC call to re-commit/re-confirm would be great!

innermatrix commented 2 years ago

@FredKSchott I am confused how ignoring things inside pages addresses this. Can you elaborate?

FredKSchott commented 2 years ago

Ah, good catch. That idea would only address half of the problem:

src/pages/about/index.astro        # public page
src/pages/about/_assets/cover.png  # private asset

The above has been discussed and I think generally has buy-in. But, there is no guarantee about where those private assets are hosted. Today, only public gives you guarantees around final URL.

The reason that src doesn't give final URL guarantees is important: Astro bundles your assets for production, which essentially involves moving and combining files. We can't tell you a file will live somewhere in production and build/bundle it. This is why public exists.

The other problem is that src/pages may include more than just .md and .astro files. For example, an accepted RFC lets you do src/pages/foo.png.js which lets you define a JS function that runs and returns an image. https://github.com/snowpackjs/astro/issues/1305

I still think this is worth a discussion, at least to get more eyes on it. Maybe src/pages/foo.png is okay to put in the pages directory and treat like public/foo.png? I'd be curious what other people think.

Thanks for starting this convo, @innermatrix !

innermatrix commented 2 years ago

Thanks for clarifying. Yeah, I get the bundled/not bundled distinction, but the decision to make that distinction at the top-level directory (public and src/pages) — as opposed to at a file-level — results in some uncomfortable tradeoffs on the authoring end (and feature/DX disparity with other tools aimed at the same audience). I do agree that the current model was a good MVP though.

matthewp commented 2 years ago

Accepted RFC (add link): /foo.png.js to generate content.
- Possible solution to allow more control.
/src/pages/product/_images/pants.png (allows to colocation). No RFC for this yet. Would need an RFC. Could help with the collocation problem, but not with the build.
Needs a proposal that answers questions on how you can control the build knowing that we might bundle files together.
- For example, can certain file types be treated more like they live in public/.

FredKSchott commented 2 years ago

Hey everyone! Our current RFC process is beginning to break down at this size, with over 50 open RFCs currently in the "discussing" stage. A growing community is a great problem to have, but our ability to give you RFC feedback has suffered as a result. In an effort to improve our RFC process, we are making some changes to better organize things.

From now on, all RFCs will live in a standalone repo: https://github.com/withastro/rfcs

This allows us to do three things: 1) Use threaded discussions for high-level ideas and improvements, without necessarily requiring an implementation for every idea. 2) Improve the quality of our RFC template and the speed/quality of all feedback. 3) Support inline comments and explicit approvals on RFCs, via a new Pull Request review process.

We hope that this new process leads to better RFC weekly calls and faster feedback on your RFCs from maintainers. More detail can be found in the new RFC repo README.

We can't automatically convert this issue to an RFC in the new repo because new RFC template is more detailed that this one. But, you can still continue this discussion in the new repo by creating a new Discussion in the RFC repo and copy-and-pasting this post (and any relevant follow-up comments) into it. Discussions are available for high-level ideas and suggestions without the requirement of a full implementation proposal.

Then, when you are ready to propose (or re-propose) an implementation for feedback and approval, you can create a new RFC using the new RFC template. More detail about how to do this can be found in the new RFC repo README.

Thanks for your patience as we attempt to improve things for both authors and reviewers. If you have any questions, don't hesitate to reach out on Discord. https://astro.build/chat

withastro / astro