"Project Config" feature like CraftCMS

[jlahijani]

CraftCMS 3.1 Dev Preview, which was just released, comes with a new feature called Project Config. From the blog post:

Most Craft projects live on at least two environments: a local dev machine and a production server. While that’s certainly a good thing, keeping those environments in sync can be a pain. Developers have tried various tactics using plugins, database syncing scripts, content migrations, and more, but none have been great.

That all changes in Craft 3.1, thanks to a new file named project.yaml.

project.yaml lives in your project’s config/ folder, and it stores info about your sites, custom fields, sections, category groups, tag groups, user groups, global sets, and more—all the things that define your project from a development perspective. It gets updated automatically as you work, and whenever Craft detects that external changes have been made to it, it will update its internal setup to reflect those changes.

So the next time you need to add a new section to an existing site, you won’t need to worry about how to make the exact same custom field and section changes on Dev and Production, or make sure that everyone on the team is running a recent database backup. Just commit your new project.yaml file alongside any other template or CSS changes you’ve made, and push them out. When they are deployed to a new environment, Craft will detect the changes, and apply them locally with the click of a button.

There have been similar modules for PW, which attempt similar functionality: http://modules.processwire.com/modules/auto-export-templates-and-fields/ https://lostkobrakai.github.io/Migrations/ https://processwire.com/talk/topic/13758-autoexporttemplatesandfields-enables-continuous-integration-of-template-and-field-configuration-changes/

I wonder if this is something worthy of core.

[Toutouwai]

I'd love to see a core feature for this.

And maybe it wouldn't be too difficult to implement considering that the core already supports exporting fields and templates. The main difference is that the data would be saved to a single centralised file, and the data would update as field and template changes are made. And perhaps module config data could be included too - so what @adrianbj's ModuleSettingsImportExport module does but saving data to the centralised file as module settings are saved.

[Toutouwai]

One critical part is the stage where, as the Craft blog post says, the changes are applied locally "with the click of a button". I'm sure there is quite a bit more to it than that, as you would need to see notifications of what will change and have some way to roll back changes that would cause the loss of field data.

@jlahijani, if you are a Craft user maybe you could let us know how Craft 3.1 handles this and any feedback on what you think works well and what doesn't.

[ethanbeyer]

This is something I've thought a lot about. For a while, my main CMS of choice was [Statamic] (http://statamic.com), and they implemented all Fields and Templates as YAML files.

I had toyed with the idea of writing a module that would essentially attempt to translate the database schema to a YAML file, in order to do exactly what it seems CraftCMS 3.1 is now doing.

I think this is exceedingly helpful, as sites/deployments/environments become more complicated.

Finding a way to centralize that information, and version control it outside of migrations (which I love, but add their own complexity), gets a huge +1 from me.

---

Edit: Additionally, I think some of the back-end needs for this are already there. With a YAML Parser, it should be possible to translate the Template and Field Export JSON to the file. ProcessWire could, in theory, read the YAML file into memory, and any time a Field, Template, or Fieldgroup was edited, it could modify the data in memory, and rewrite the whole file.

On deployments, or maybe even through a mechanism similar to refreshing the Modules, one could trigger schema changes on the items in the YAML file.

Where it inevitably gets tricky is when things change on production. How does the dev or staging environment stay in sync at that point?

[thetuningspoon]

This would be amazing!

[jmartsch]

as @ethanbeyer mentioned, Statamic handles schemas this way, which is very nice and makes it easy to reuse field and templates in multiple projects. I think this is something, that the core really needs, as I often struggle to keep live and dev server in sync after making a change (or even when rolling back). I have to remember and copy all fields and templates.

[tnc1]

Sounds like a very useful feature to have!

[BernhardBaumrock]

This is exactly how I build all my sites and modules nowadays using RockMigrations. I know the docs are almost non-existant, but the basics are really simple and powerful at the same time:

$rm->migrate([
  'fields' => [
    'field_one' => [
      'type' => 'text',
      'label' => 'EINS',
    ],
    'field_two' => [
      'type' => 'text',
      'label' => 'ZWEI',
    ],
  ],
  'templates' => [
    'my_template' => [
      'fields' => ['title', 'field_one', 'field_two'],
      'icon' => 'database',
      'tags' => 'MyDemo',
    ],
  ],
  'pages' => [
    'my-page' => [
      'title' => "My example page",
      'template' => 'my_template',
      'parent' => 1,
      'status' => ['hidden', 'locked'],
    ],
  ],
]);

Not saying it is perfect, but it has worked extremely well for me over the last 2 years.

Also not saying that this wouldn't be great to have in the core :)

[MetaTunes]

My ProcessDbMigrate module exports field, template and (where necessary) page definitions to json files. It does this as a batch action for a defined scope, not all the time for everything and every change. The production environment maintains an export of the old definitions before applying the new ones, so that roll-back is possible. The core would need some fixes before using the existing export tools because I tried to use those in my module and found that they did not handle all types and contained bugs. My module is really only POC at this stage and does not handle all situations (nested repeaters for example). However, my view is that it should be possible to implement something similar to that proposed (or even better!), if there is sufficient demand. Whether the definitions are stored as json, yaml or php arrays is not the most relevant issue. Where they are stored (centrally or as separate units), how they are created (automatically or on demand), what the scope of the configuration is, how they are installed and rolled-back are matters that require more thought, based on the experiences of my module, RockMigrations and earlier modules.

[ivangretsky]

This is the topic I've though about and the feature I dreamed about for PW for about a decade now) That is why I dare to mention @ryancramerdesign , as there is still no response from him. This is the most liked feature request, so it must be needed by a lot of PW users too.

As @ethanbeyer said some of the functionality for this could be already in place. So might be a doable thing. At least for fields/templates.

P.S. Would be an awesome idea for a new Pro module too. I would be the 1st one to buy)

[BernhardBaumrock]

@ivangretsky what if I built that as an extension module for RockMigrations?

[ivangretsky]

@ivangretsky what if I built that as an extension module for RockMigrations?

That would be awesome! The one thing most of us need to be taken care of is Pro Modules, as many of those, especially Repeater Matrix and Combo are essential part of PW site-building process for many of us. That's why I was hoping for @ryancramerdesign 's answer first.

[BernhardBaumrock]

May I ask how you use the combo field and why it is so essential for you? It would maybe help me to understand the whole topic.

[ivangretsky]

@BernhardBaumrock , I would happily do so, but I guess we better do it in some other communication channel, as it would be a total offtop for this important topic)