Store Assets Relative to Content (Bundles)

marcojakob commented 6 years ago

Do you want to request a feature or report a bug?

Feature

What is the current behavior?

Currently, the Media Library is a single folder where every image is dumped. This doesn't allow assets to be organized, managed, and deleted properly.

What is the expected behavior?

There is work going on to add subfolders (see #1046) and provide further improvements to the Media Library (see #1321).

But I think we should go a step further than that and allow assets to be stored relative to its content (it's been asked for in various comments to media library issues). The paradigm for such asset handling is different, but I belive it is a very reasonable way to organize the content.

The Hugo SSG has been moving more and more towards this way of organizing its content and calls this Page Bundles (see Hugo Content Management with Page Bundles and Content Organization).

content/
├── posts
│   ├── my-post-1
│   │   ├── index.md
│   │   └── post-image.jpg
│   ├── my-post-2
│   │   ├── index.md
│   │   └── post-image.jpg
│   ├── _index.md
│   └── overview-image.jpg

In this example, there are two page bundles, one for /posts/my-post-1 and one for /posts/my-post-2. They each have their own assets stored in the same folder.

Then there is also a bundle for the overview site for posts (Hugo calls it a Branch Bundle).

Reason for Bundles

Entire content for one page with its assets is stored in one folder. It is obvious which assets (images and also other resources) belong to a page.
SSG's generally determine output placement based on the location of source files. So, we can just reference assets with a simple relative path or just the filename instead of using /assets/some-unique-filename.jpg or /assets/some/duplicated/path/to/image.jpg.
The markdown content with relative images is rendered in the GitHub ui.
Content editors don't need to browse through lots of unrelated assets.
Assets can be optimized (sized/cropped) for its specific content.
Assets can safely be deleted if a page is deleted. In a central assets folder we are never sure if some image is still referenced by some content.
In my mind, images for a blog post are an integral part of the content. So, bundling it into one folder just makes sense.
Content is also portable: I can zip markdown and images together and they work independent from the entire website.
This might also improve an editorial workflow as the reviewer can focus on reviewing one folder containing everything related to the content.

tech4him1 commented 6 years ago

Also see initial discussion in https://github.com/netlify/netlify-cms/issues/325.

CanRau commented 6 years ago

would love that, too, i'm not yet using netlify cms but am considering it. as i'm managing my assets in an assets directory relative to the post it's not easy to migrate. and by the way, to prevent duplicate assets for internationalized content i'm also storing all language versions in the same directory having one shared assets folder would love to use this structure with netlify cms 👍

alexito4 commented 6 years ago

Content editors don't need to browse through lots of unrelated assets

I would like to underline this. For people that uses Apps for markdown editing this is important. Usually these Apps can render images if they are in the same directory as .md file or with absolute path, which is not useful for later deployment. If the CMS supported this feature it means we can work on our posts in the same way from our local machines and from the CMS. ❤️

arturogh commented 5 years ago

I want this functionality and would like to lend a hand 😁. Design and possibly code help (front end).

erquhart commented 5 years ago

@arturogh that would be awesome - happy to support you in your efforts, too 👍🏼👍🏼

_{Sent with GitHawk}

alexandernanberg commented 5 years ago

Great! This is a feature I'd really want too so I'd be happy to help in any way 😄

arturogh commented 5 years ago

Do you guys have any design assets for Netlify CMS I could leverage? Someone working on the technical, backend side of this?

Also, a short 5-10 min chat would be great :).

erquhart commented 5 years ago

You can view assets in our design repository on the fabulous Abstract - here's the invitation link.

@Benaiah and I both work on the backend, and @talves also has a fair amount of familiarity. We generally field questions right here in the issue, or on the PR if you have one.

Finally, I'm happy to have a call with you to answer any questions, I'll email you with details.

CanRau commented 5 years ago

Any news on relative assets? Just to know where we are 🤓

arturogh commented 5 years ago

Really sorry...I still want to work on this but life got crazy. I'd like to help with design and front end possibly at some point.

ahoward commented 5 years ago

anyone interested should check out ro. it manages data just like that. we use it that way @ dojo4. https://github.com/ahoward/ro -- https://dojo4.com/blog/static-is-the-new-black

BrandonDusseau commented 5 years ago

It's poor form to advertise your own projects on other people's repos. Please don't.

ahoward commented 5 years ago

@BrandonDusseau i am 100% sure that the netlify team and i are on the same page with respect to having spent 1000s of hours thinking about static sites. we've been users of bitballoon - become netlify - since the beginning and promoting them every chance we get - and i feel very calm saying that their team absolutely holds the spirit of intellectual conversation around software dear. if i were to be actually selling, or even promoting, something i'd done i would agree with you. but this is just nerds sharing ideas. i held in the design principles of ro that content would render relatively on a variety of platforms, and i blogged about it, and i'm sharing it, in case the code helps someone on this thread or, who knows, stimulates, some ideas inside the netlify team. there isn't even aa valid license on that repo so, obviously, people can steal or modify anything they like.

i prefer the old internet with black screens to this new facebook style one. some people just want to write code.

talves commented 5 years ago

Ok, I am going to pull the maintainer card here. The following is not directed at any one person:

Before a conversation gets out of hand, make sure to adhere to the Code of Conduct Let's keep the conversation on track with the issue title ("Store Assets Relative to Content") as best possible.

Thanks.

ahoward commented 5 years ago

imma follow up and say that relative assets are a super noble goal, it is really hard. the main problem being that you have to pin a relationship ahead of time. that is to say "there are images associated, or 'under', this author". the nesting has obvious advantages as well - rendering in the native/default github ui is no small thing - it really, really, really make many things feel super natural. primarily because 'stuff renders'. this was something i was after with ro.... however. i think it's flawed: ultimately i think making assets 'absolute' and 'idempotent' is the best design decision cuz complexity. i anyone wants to examine the complexity of resolving, and portably storing, assets in relation to 'core' data objects ro is a great place to look. it sux. but it's also awesome because stuff renders in github - this should not be over looked. it. is. rad. however, absolute urls (cloudinary, uploadcare, etc) is the right answer.

i think anyone working on static sites can agree (but hey, maybe not) that 1) the entire goal of static sites is to create portable content, not just replace the db and 2) rendering 'normal' on the main platforms, with multiple input entry points , is the holy grail. 3) damn hard.

i think (not that anyone is asking) that two design principles could/would/should solve both problems:

1) assets are uuids. uuid'd urls, etc. but unique and url'able in an absolute sense 2) a 2 way link system, meaning, a piece of content should be able to 'render itself' in a 'relative sense' (meaning it knows the potential assets related to itself) but, also, that content is aware of 'the asset associated with me', because, as any real world website maintainer knows, that asset - the one that facebook will show, that twitter will preview, is everything.

some sound design principles can solve all problems: namely, that all urls in markdown/etc are absolute, prefixed with https, and idempotent.

i'm just sharing my battles. take or leave what you will but i think the sentiment in this issue is very important. and that the solutions are very, very hard.

ahoward commented 5 years ago

tl;dr; keep relationships intact, even in the sense of rendering natively on github, but do so not via nesting storage, but by accepting, and preferring, single, idempotent, urls for every asset ever chucked at the the cms (aka. absolute links for all assets referenced in content/markdown). my 2 cts. and i'm out.

erquhart commented 5 years ago

Thanks @ahoward - I know you've pushed hard for a number of things that still haven't come to fruition, glad to see you're still keeping an eye on the project. This feature is definitely high on the priority list, and I agree with the goal of things making sense right in GitHub as well. Thanks for the thoughtful points!

_{Sent with GitHawk}

erodrig commented 5 years ago

Any update on this, for me this feature is critical for any medium size site.

erquhart commented 5 years ago

Probably our next priority after we finish OSS Docs support (#2093).

_{Sent with GitHawk}

erquhart commented 5 years ago

Meant to add this after the last community meeting - here's the approach we discussed:

allow the collection slug to contain a slash
everything before the slash is processed as a directory name with variables

collections:
  - name: posts
    label: Posts
    create: true
    folder: content/posts
    slug: "{{title}}/index.md"
    fields:
      - { name: title, label: Title }
      - { name: body, label: Body, widget: markdown }

This approach only allows a single directory level to be created dynamically per entry, which should be sufficient.

Thoughts? Use cases this wouldn't work for?

robdodson commented 5 years ago

I believe that would work for our use case on web.dev :)

marcojakob commented 5 years ago

@erquhart Awesome. Looking forward to that!

In your exampe you are using the {{title}} as a directory name. This is great as a default when a new post is created. But what happens when you later want to change the directory name (independently from the title)? Is there a way to do that in the CMS?

I'm asking this question because in a setup with bundles, the directory name is very important. One shouldn't customize the resulting slug via a slug entry in the frontmatter since SSGs will only apply that to the generated html file but not to assets inside the directory. Thus all relative paths inside the markdown would fail.

Also, I wouldn't want to store the slug in the frontmatter anyways. This would be unnecessary duplication since that information is already in the directory or filename. And if you change one and not the other, things get very confusing in the repo.

robdodson commented 5 years ago

Ah that's a good point. We often have posts whose title does not exactly march their slugified directory name.

On Thu, Jun 20, 2019, 1:50 PM Marco Jakob notifications@github.com wrote:

@erquhart https://github.com/erquhart Awesome. Looking forward to that!

In your exampe you are using the {{title}} as a directory name. This is great as a default when a new post is created. But what happens when you later want to change the directory name (independently from the title)? Is there a way to do that in the CMS?

I'm asking this question because in a setup with bundles, the directory name is very important. One shouldn't customize the resulting slug via a slug entry in the frontmatter since SSGs will only apply that to the generated html file but not assets inside the directory. Thus all relative paths inside the markdown would fail.

Also, I wouldn't want to store the slug in the frontmatter anyways. This would be unnecessary duplication since that information is already in the directory or filename. And if you change one and not the other, things get very confusing in the repo.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/netlify/netlify-cms/issues/1472?email_source=notifications&email_token=AAIEKDLUCQSFJGO3A3TQAYTP3PUPZA5CNFSM4FHSVJTKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODYGSISA#issuecomment-504177736, or mute the thread https://github.com/notifications/unsubscribe-auth/AAIEKDMZBIZPWBJAUCRCHYDP3PUPZANCNFSM4FHSVJTA .

erquhart commented 5 years ago

The requirement to change a slug after initial entry creation is of broader scope than this issue, covered under #445. Let me know if I missed what you were trying to communicate.

marcojakob commented 5 years ago

Yes, #445 should cover the case of changing the slug. We would just need to make sure in #445 that anything that is done with sluggification must also work with folder names and not just with file names.

erquhart commented 5 years ago

For sure, this issue will be addressed before that one 👍

jballe commented 5 years ago

This is looking fine to create folder per post as is a needed first step, so thank you @erquhart. To make this really useful we want to use this folder for assets/images.

Can we add an image widget using that folder eg.

fields:
- label: "Featured Image"
  name: "thumbnail"
  widget: "image"
  config:
    media_folder: "."

(the media_folder property is at least not documented today so I guess it has to be developed) What about medias in the body/markdown widget using the current folder. How can we achieve that?

Still, thank you for your work, I really like netlify-cms :)

erquhart commented 5 years ago

@jballe you won't need to configure, we'll likely implement so that all images are stored with the entry you're editing, including those in markdown. Netlify CMS already has some internal asset proxying to facilitate this. We should have a switch on the media library for global vs. entry, but that will probably come later.

jballe commented 5 years ago

Super, @erquhart! I agree that if you want to create a folder per post then you probably also want to store assets in that folder. However I can imagine, that for some fields I want to reference the global medias, but a candidate for a later step :+1:

patdavid commented 5 years ago

I'm just checking in to see if there's been any movement on this? Is it available yet to use?

erquhart commented 5 years ago

This is the very next thing we're working on. Stay tuned.

numoonchld commented 5 years ago

Looking forward to hear more about this!

ahoward commented 5 years ago

__yes!

--

a
http://dojo4.com/team/ara-t-howard
https://www.instagram.com/drawohara
i sometimes check email once or twice per week so if you're really after me come by, call, or text.

On Thu, Sep 5, 2019 at 3:34 PM Shawn Erquhart notifications@github.com wrote:

This is the very next thing we're working on. Stay tuned.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/netlify/netlify-cms/issues/1472?email_source=notifications&email_token=AAABPTQDU5FH23WUIQRTUX3QIF3NJA5CNFSM4FHSVJTKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6A4JMQ#issuecomment-528598194, or mute the thread https://github.com/notifications/unsubscribe-auth/AAABPTQOAXFAJEZ7VGFW673QIF3NJANCNFSM4FHSVJTA .

mazdak78 commented 5 years ago

Oh, I am looking forward for this one too, highly needed!

CanRau commented 5 years ago

@mazdak78 Already in beta it seems https://www.netlifycms.org/docs/beta-features/#relative-image-paths 🎉

mazdak78 commented 5 years ago

@mazdak78 Already in beta it seems https://www.netlifycms.org/docs/beta-features/#relative-image-paths 🎉

@CanRau : Oh, thanks, but I was mostly referring to feature about slug: "{{title}}/index.md" mentioned by @erquhart on Jun 19. :)

jimmyangel commented 5 years ago

@erquhart Thank you for a super useful product! Question: are you guys working on the slug: "{{title}}/index.md" feature you mentioned on Jun 19, referred to by @mazdak78 above? If so, is there a rough ETA? -- I am getting started on a big project right now and we are making content organization decisions, and this feature would make it so much easier to have a cleaner set up!

tomrutgers commented 5 years ago

@jimmyangel https://github.com/netlify/netlify-cms/issues/513

amine250 commented 4 years ago

Meant to add this after the last community meeting - here's the approach we discussed:
* allow the collection `slug` to contain a slash

* everything before the slash is processed as a directory name with variables
collections:
  - name: posts
    label: Posts
    create: true
    folder: content/posts
    slug: "{{title}}/index.md"
    fields:
      - { name: title, label: Title }
      - { name: body, label: Body, widget: markdown }
This approach only allows a single directory level to be created dynamically per entry, which should be sufficient.

Thoughts? Use cases this wouldn't work for?

Hi @erquhart, do you have an ETA for this feature? Thanks.

jimmyangel commented 4 years ago

What about images? Can they be stored in the same location, i.e., content/posts/{{title}}

goulvenclech commented 4 years ago

Hi everyone ! Does anyone know if this feature is implemented? This is the only thing preventing us from using Netlify CMS in our project... and the documentation is not clear on what the option media_folder_relative: true allows to do. Thanks for the help :)

erquhart commented 4 years ago

It’s in progress, watch here for updates

_{Sent with GitHawk}

erezrokah commented 4 years ago

Latest code on master branch (not published yet to beta) lets you do:

slug: 'index'
path: '{{title}}/{{slug}}'

to save a file under title/index.md for example.

Next step will be to store images with the content.

heejongahn commented 4 years ago

Thanks a ton for implementing this feature!

KesleyDavid commented 4 years ago

ficaria muito melhor mesmo para gerenciarmos as postagens

Ir1d commented 4 years ago

Hi, I think the doc could be updated right? https://www.netlifycms.org/docs/gatsby/#get-to-know-gatsby

erezrokah commented 4 years ago

Good catch @Ir1d! removing that part will also require updating the config.yml yaml to support the starter directory structure. Do you mind opening an issue for that?

nidhi-wgl commented 4 years ago

hi what would happen if title has space will the folder be created with spaces or hyphen?

erezrokah commented 4 years ago

@nidhi-wgl, sanitisation is still applied but only to template variables.

theredwillow commented 3 years ago

@erezrokah This is still listed under beta features in the docs. Is that the case or should this be moved to the permanent docs?

decaporg / decap-cms