search

backdrop / backdrop-issues

Issue tracker for Backdrop core.
145 stars 40 forks source link

Add 'recipe' project type to Backdrop #3763

Open ghost opened 5 years ago

ghost commented 5 years ago

This is a spin-off of https://github.com/backdrop-ops/backdropcms.org/issues/46 specifically for proposing a new project type for Backdrop: recipes.

With Backdrop supporting modules for functionality, themes for styling, layouts for, well, layout and install profiles for the initial configuration and setup of a site, I see the addition of a 'recipe' as combining all these separate things into a bundled package of sorts.

Use cases include:

The reason I like the term 'recipe' (which @klonos originally suggested in the issue linked above) is that 'package' (my original suggestion) implies something that you open to find other things inside, whereas 'recipe' is a bunch of ingredients which, when combined, create something new. I think recipes could be as simple as a single info file, such as:

name = My demo
description = Sets up a Backdrop site for demoing.
backdrop = 1.x # Not sure if this line is necessary...
type = recipe

modules[] = devel
modules[] = webform
modules[] = flexible_layout

themes[] = summer_fun

layouts[] = flexible_template

profiles[] = demo_profile

Or maybe even a JSON file and use it in config somehow... Either way, a simple text file should suffice. And as per info files you should be able to optionally specify versions too.

So modules provide the features/functionality, themes the styling, layouts the layout and profiles the setup (install modules, set theme as default, add blocks to a layout, add custom content types, etc.). Of course all of these would be optional (you could have a recipe of just modules, or a theme and matching layout, etc.).

Wiki Page Summary Related META Issue: https://github.com/backdrop/backdrop-issues/issues/5240

laryn commented 5 years ago

Love it!

herbdool commented 5 years ago

Would be good to have a proof of concept. I think it could be mocked up with a custom module, which includes the dependencies and runs some hooks to set up a baseline config. Would need to figure out how to include the theme and layout as dependencies.

herbdool commented 5 years ago

Would need to test out what happens if the recipe is disabled or uninstalled, or the are updates to the recipe from upstream, how to deal with divergent changes...

Graham-72 commented 5 years ago

👍

klonos commented 5 years ago

Another thing to decide is where these recipes will live in the file system ...if they are actual files that is.

The reason why I say this, is because the idea behind these "recipes" was that one could include these as plain-text on say a blog post. Then the "consumer" would copy this text, and paste it into our Project Installer (via the "manual installation" dialog - or some similar UI):

Screen Shot 2019-05-13 at 2 05 08 pm

^^ that's a mockup 😄

So basically, when I thought of this initially, I have not factored in any new project type, .info files, or versioning. Not implying that that would not be the way to go; just saying.

ghost commented 5 years ago

Hmm, yes, I think that's a good idea. We want to make it as easy as possible for people to create and use recipes, so not having to write info/json files and 'install' them would be handy. That'll also solve @herbdool's problem of disabling/uninstalling recipes and I think would make for a simpler setup all-round.

klonos commented 5 years ago

I am adding the "needs feedback" label. If people agree that this should not be another project type after all, then we can rename this issue accordingly.

laryn commented 5 years ago

It sounds like a recipe could simply be an install script for a bunch of things, without those things being tracked via the recipe going forward. (ie. you use the recipe to make a cake, but the cake doesn't change next week if you later adjust the recipe).

That makes sense to me.

Would it also be possible to load example configuration and content via a recipe, or would that have to be done through a module that the recipe installed? (e.g. getting a shell of a site built out quickly with some sample content, blocks in place on certain layouts, etc.)

ghost commented 5 years ago

@laryn I think install profiles are still the best place for that.

ghost commented 5 years ago

I think I'm starting to get confused... I'm now wondering how recipes will be any different than the 'INSTALL PROJECTS BY NAME' field in @klonos's screenshot above...? If you can enter the names of modules, layouts and themes there and have them all installed for you (with dependencies, etc.) then isn't that what a recipe would have been essentially anyway? And if that field's only for modules, then perhaps a simple feature request to expand that to layouts and themes as well would suffice.

Unless I'm missing something and there's more to the idea of recipes than that...?

jenlampton commented 5 years ago

If you can enter the names of modules, layouts and themes there and have them all installed for you (with dependencies, etc.) then isn't that what a recipe would have been essentially anyway?

Maybe it becomes an info or json file, and the "instructions" just become to copy/paste it into that box?

docwilmot commented 5 years ago

I wasnt thinking that recipes would be entirely different from profiles, but the distinguisher would be that you could install them at any time, using the Installer UI. For example, you want a new church site, with menu and pages and so on; you select a church Recipe in Installer, and it walks you through the "recipe" for building one: you choose a theme, you choose some options, you decide on dummy content, etc, then click install and voila! a new church site. It eventually could probably have an advanced option to backup your dbase first so you can revert.

But something like that.

findlabnet commented 5 years ago

@docwilmot great scenario. Not church only, but restaurant, gallery or real estate agency - here we go to real users.

jlfranklin commented 5 years ago

How is this different from the Features module? What you're describing is close to its original purpose.

In the case of Features, someone would put together a Features module that included all the required modules as dependencies, the content types, and all the config to deploy a particular kind of site. Features went beyond that to allow a site to enable certain features -- enable the calendar or not, enable the blog or not, enable reservations (like for a restaurant), enable catering?

Features + the Installer module would allow someone to upload just the feature module and then download any additional modules they may need, enable all of it, and have a "pre-built" site.

docwilmot commented 5 years ago

How is this different from the Features module? What you're describing is close to its original purpose.

this is a reasonable point. Port Features instead?

herbdool commented 5 years ago

Sure, port Features... if you want to enter a world of pain. 😋 I imagine D7 version can't easily be converted to use config instead of the db as a source. Though maybe I'm wrong.

jlfranklin commented 5 years ago

Sure, port Features... if you want to enter a world of pain. 😋

Same question still applies: How is this different from Features? Specifically, how does it avoid the pain points that Features suffers?

Part of the problem with Features is people tried to use it for configuration management, not feature definitions. Re-applying a feature was never really part of its design, nor was removing something, like a field in a content type. Reapplying sometimes worked, and sometimes it didn't (often with hilarious results.) You could remove something from the feature module, but all Features would do is orphan it on the site.

Features itself had no concept of a migration path. A fantastic dev team could write migrations via hook_update_N(), but the amount of testing it required for an automation that would run exactly once... it was nearly always faster to just write down the migration steps in the ticket.

jenlampton commented 5 years ago

How is this different from Features?

The only real difference is that we'd be saving config, which Backdrop knows how to handle, not some crazy one-of thing that Drupal didn't really understand.

how does it avoid the pain points that Features suffers?

Features was painful because Drupal didn't really understand how to store things that are configuration anywhere other than in the database. This would avoid that because It's Backdrop, and not storing anything that's config in the database.

Part of the problem with Features is people tried to use it for configuration management,

This won't be a use-case for Backdrop. We'd only be using features for it's intended purpose: Packaging config together.

Sure, port Features... if you want to enter a world of pain

I was envisioning we'd build something completely new, and name it "Features". But a port miiiiight be possible?

stpaultim commented 5 years ago

I'm interested in this issue. I've started playing with a simple repo of config recipes: https://github.com/TeamTriplo/backdropcms-features https://github.com/backdrop-contrib/config_recipes

So far, I have two recipes/features. This is not an ideal system, because it requires that I load each config file separately and in the right order. It's not bad, but there must be another way.

Should I make my config recipes a contrib project? I am not sure if that his the right place for them?

EDIT: After a discussion during a dev meeting, I went ahead and created a github project for this experiment.

klonos commented 5 years ago

That's an interesting subject for the dev meeting @stpaultim. On the one hand, I don't like the idea of "polluting" contrib with recipes, but on the other, d.org did allow for distributions, which is the equivalent of (or similar to) what we are talking about here.

I am interested in this as well.

stpaultim commented 5 years ago

We talked about this on todays DEV call. As things currently stand, a user can ONLY input all config at once or one file at a time. It would be helpful if a user could enter a batch of config files that make up a recipe.

It is possible to export the current config directory, add a batch/recipe to the entire directory, and then import the entire batch.

Would it be possible (relatively easily possible) to add an option that automates this. An UI that allows the user to add a directory of config files and then have Backdrop combine the current files with the new batch?

(OK, now that I write it out, this is probably no easier than simply coming up with a way to add a batch of config on their own. I guess, the combining with active config files is only necessary because of the lack of the alternative UI).

jenlampton commented 5 years ago

Would it be possible (relatively easily possible) to add an option that automates this. A UI that allows the user to add a directory of config files and then have Backdrop combine the current files with the new batch?

Yes. This is exactly what I want in core :)

stpaultim commented 5 years ago

I documented my current process in the forum, which is cumbersome. https://forum.backdropcms.org/forum/how-add-bundle-or-group-config-files-site

I hope we can make progress on this issue.

Here is a related issue: https://github.com/backdrop/backdrop-issues/issues/3933 (CLOSED) in favor of https://github.com/backdrop/backdrop-issues/issues/661

AND - I created a github contrib project for a very simple library of config recipes: https://github.com/backdrop-contrib/config-recipes

stpaultim commented 5 years ago

I think this is related: Allow config files to have dependencies #4188

stpaultim commented 5 years ago

I created a video: https://youtu.be/VgFibVrEZEw

stpaultim commented 4 years ago

I started a very crude contrib project - ideas or PR's welcome. I'm stretching myself with this one. https://github.com/backdrop-contrib/config_batch_upload

cellear commented 4 years ago

As usual I'm late to the party, but "recipes" sounds like a great idea. Go, team, go! (Go, Tim, go! ;) )

(I'll wait until it gets more traction before I try to feature-creep the idea of including content in recipes :p )

klonos commented 4 years ago

Go, team, go! (Go, Tim, go! ;) )

🤣

klonos commented 4 years ago

During the VUG earlier today, @stpaultim mentioned that things like the need to have dependencies declared, as well as being able to specify the order in which config files get imported, often becomes a blocker (some .json files fail to be imported). I had the following thoughts on those:

Food for thought.

ghost commented 3 years ago

I've just re-read through (most of) this issue, and it seems we got sidetracked (a few times)... There are lots of different ideas and suggestions, so I'll try to summarise them and clarify what this issue should be focused on...

The are two main ideas around 'recipes':

Any ideas that deal with install profiles, demo sites, adding default content, etc. come under the first category. I don't believe recipes should cover this, as we already have install profiles that are very powerful (they're basically fancy modules). Install profiles can setup a new site from scratch, they can install modules, enable themes, add layouts, they can even add default configuration (e.g. Views). They can't yet add default content, but I feel like that should be a totally separate issue.

So I think recipes, as this issue defines them, come under the second category - adding functionality to an existing website. Some examples include:

I think we can ditch the idea of 'recipes' being a new type of project that has a version and can be updated, uninstalled, etc. That's complicated (as discussed re. Drupal's Features module). I see recipes as being something you 'install' on an existing site - content types are created, Views are added, etc. Then the site acts just as if you had manually built those things yourself. You can disable or uninstall the individual parts. The recipe doesn't exist after it's been 'installed'. There's no 'overridden' status, 'recipe update available', etc. Recipes are just a quick way of adding things to your site without you needing to create them all manually from scratch. This also means that you can 'install' a recipe, then change it. For example, install the FAQs recipe, but then add a new field to the content type that lets you upload an icon to be displayed next to each question.

If we want to keep recipes simple and have the ability to just copy & paste them into a UI in Backdrop to 'install' them, then in order to allow for configuration to be included (e.g. views, content types, etc.), I think we need to allow for linking to externally-hosted config files. We could support GitHub, Gists, or people's own websites. For example, a recipe could look like this:

# Modules
- devel
- simplify
- webform

# Themes
- lumi

# Layouts
- bamboo

# Configuration
- https://gist.github.com/BWPanda/123456789
- https://panda.id.au/sites/panda/files/recipes/config-123.json

Modules, themes and layouts would be downloaded from B.org, and the other URLs would be accessed directly (and then checked to make sure they're actually valid Backdrop config files).

Anyway, let's try to focus this issue on that type of idea for recipes and open other issues for the rest of the desired functionality surrounding this topic.

philsward commented 3 years ago

Idea:

Block recipes. A collection of things that could create a specific block usage.

Example: a block who's only function is to hold and display images. The recipe would create the content type (field type?) to hold the images and create the view with block to display the images.

stpaultim commented 3 years ago

@BWPanda - Thanks for that great summary. This issue is getting some traction in our priority survey, yet it occurs to me that we don't even know what a recipe will look like yet.

I agree with the following points: A recipe should be added to a module post install. It's a one-time thing that is not updated, maintained, or disabled. (The downside is that this makes it difficult to test a recipe to see if it does what you want). It's about adding a "feature" to a website (content types, fields, blocks, and views). A recipe may be dependent upon certain contrib modules or layouts.

It's not clear to me that a recipe should be dependent upon a theme. I think it would be best if recipes were independent of themes (that they should work on all or most themes). However, I can see where theme related dependencies might be necessary.

A few random thoughts:

1) Instead of being dependent upon a theme, maybe a recipe should be allowed to contain some custom CSS. In which case, maybe it should be possible to disable the recipe (to disable the custom css).

2) I could envision a scenario in which config files are included in the project repo, but that the project is uploaded using the config manager rather than the project browser (especially if they are upload only without need to disable).

3) This project https://github.com/backdrop-contrib/config_recipes has some simple sample recipes. The downside is that there is no way current way to require dependencies (modules) in these recipes.

philsward commented 3 years ago

Would a recipe automatically download the dependency modules? My vote is yes.

klonos commented 3 years ago

Would a recipe automatically download the dependency modules

Yep. That's the goal: run a recipe -> all necessary components and their dependencies get downloaded and configured for you.

Instead of being dependent upon a theme, maybe a recipe should be allowed to contain some custom CSS

We could allow recipes to include optional components, which would show checkboxes to allow selecting or skipping them.

philsward commented 3 years ago

Thanks. It was kind of implied; just wanted clarification.

On a side note, it was mentioned to have recipes use a .info file to hold the recipe information. What about a .recipe file? Maybe .reci?

laryn commented 3 years ago

I've had stewing in the back of my mind a module for Paragraphs (I've been calling it "Prefab Paragraphs") that would allow people to click a button to configure a paragraph type -- but perhaps a core "recipe" module would allow some way to do this in a module? e.g. in this case, a recipe for a content type with certain paragraph types pre-configured, or just pre-configuring paragraph types in general (some of which may have dependencies on other modules).

philsward commented 3 years ago

Any discussion on what to do if a machine name used by a recipe is already there?

Instead of a hard stop, I would imagine a page that would show the existing machine names and allow the admin to create a different one before continuing.

Example: This recipe wants to create a content type called "Page" with a machine name "page" but this already exists. Enter a new machine name to continue:

stpaultim commented 3 years ago

Any discussion on what to do if a machine name used by a recipe is already there? Instead of a hard stop, I would imagine a page that would show the existing machine names and allow the admin to create a different one before continuing.

This might be getting a little ahead of ourselves. The low tech solution is to give everything a machine name with a unique prefix that would help prevent duplication. But, there might be better solutions to this problem. I'm sure that others have thought about this more than me. It's a common problem in modular development.

NOTE: I created a Wikipage to help keep a summary of where this discussion is going. https://github.com/backdrop/backdrop-issues/wiki/New-%22Recipe%22-Project-Type

klonos commented 3 years ago

I created a Wikipage to help keep a summary of where this discussion is going.

Thanks for doing that @stpaultim 🙏

laryn commented 3 years ago

This is taking a step further back, but what if this isn't an entirely new project type, but more of a side-car to the existing configuration management? For example:

I'm not sure if that would help or not, but the thought just occurred to me that it could possibly simplify things for the end user and make it easy for modules to include recipes without making a completely separate project.

klonos commented 3 years ago
  • Core could provide some recipes by default

👍 ...I've always wanted core to provide an installer/wizard-like, step-by-step guide to make a site multilingual ...so basically:

I think that that could be a good use case for that, and it would provide value to Backdrop as a product.

philsward commented 3 years ago

This is taking a step further back, but what if this isn't an entirely new project type, but more of a side-car to the existing configuration management?

I think modules should be able to include recipes but I also feel like they need to stand on their own and be a community driven project.

"If" they are structured well enough that they are easy to create, non-dev people could help contribute their own recipes.

Now THIS idea is pretty far out there, but a "recipe builder" would be a fantastic way to deal with this. A person could create a playground site, set it up almost exactly the way they want it, then create a recipe from it. Kind of like the config management but without any specific data.

stpaultim commented 3 years ago

We had a discussion about "Config Recipes" at last weekends Backdrop LIVE. Here are notes that were taken. https://docs.google.com/document/d/1ia-ynczOfNsAUPiD1byTDyub7MWbF7CfLyv-Y26djJM/edit

Also - one notable takeaway from the discussion, was that one can actually create a module with nothing but a .info file and config files, which may in fact be the first step towards a MVP recipe. Atten: @hosef

I have yet to test it, but it may provide the same functionality as: https://github.com/backdrop-contrib/config_batch_upload and https://github.com/backdrop-contrib/config_recipes (but the module solutions is easier).

The downside to the module solution is that Backdrop has not way to tell this recipe type module from a regular module.

laryn commented 3 years ago

Also - one notable takeaway from the discussion, was that one can actually create a module with nothing but a .info file and config files, which may in fact be the first step towards a MVP recipe. Atten: @hosef

This old comment was what came vaguely to mind during the discussion. I dug it up again -- maybe @quicksketch can comment again on this in light of the recipe discussion? https://github.com/backdrop/backdrop-issues/issues/571#issuecomment-72962972

hosef commented 3 years ago

@laryn hmm, that is interesting. It seems like we should be calling hooks when we import files from modules. Otherwise, you can't really be sure that fields had their database tables set up properly.

hosef commented 3 years ago

Apparently this has been reported as a bug before. https://github.com/backdrop/backdrop-issues/issues/3224

stpaultim commented 3 years ago

We are nearing completion of a bug fix that will make "config recipes" possible in Backdrop. https://github.com/backdrop/backdrop-issues/issues/3224

The original idea for recipes introduced by @BWPanda is a bit more ambitious than what we have been talking about recently. The current test recipe (https://github.com/backdrop-contrib/testimonial_recipe) that I'm working with to test this other bug fix is a module with nothing but config files. I see no reason that this module could not also include module dependencies. Is it currently possible to make a module dependent upon a theme?

Anyway, we could have the basic building blocks in places for recipes 1.0 for the next release. However, we have to work out the following questions:

1) Do we want recipes that do more than simply install config files? 2) How do we want to manage them on BackdropCMS.org?

stpaultim commented 3 years ago

I've create a second test recipe. https://github.com/stpaultim/faq_recipe

(EDIT: Now in Backdrop Contrib at https://github.com/backdrop-contrib/faq_recipe)

stpaultim commented 3 years ago

See https://github.com/backdrop/backdrop-issues/issues/5173 for a discussion about a possible problem including config for modules in a recipe.

stpaultim commented 3 years ago

Discussion today during meeting about what a recipe should like: https://youtu.be/CoXHmSUGsXg?t=1632

Should they just be modules or should they be something else. Consensus seems to be leaning strongly towards them being modules, but I am not clear that a decision has been made yet.

The sense is that with the recent bug fix, we're better positioned to experiment with recipes. But, that it might be premature to write the press release announcing it as a new feature.