Split globalset configuration from data (variables)

[royduin]

For some functions; for example global sets the configuration is stored with the data. Are there any plans to split this? Or will a PR with this (breaking) change be accepted? We'd like to have all configuration like blueprints, global sets, etc in files and all data in the database. Currently we've set statamic.eloquent-driver.blueprints.driver to file and statamic.eloquent-driver.global_sets.driver to eloquent. This way we can version control the configuration of the blueprints but can't do this with the global set configuration. Currently this is stored like:

{"default": {"data": {"field": "DATA"}, "locale": "default", "origin": null}, "site2": {"data": [], "locale": "site2", "origin": "default"}}

[ryanmitchell]

I'm not totally sure what you are wanting to achieve in splitting? Can you try and explain it a different way?

[royduin]

Statamic uses flat file by default; with a lot of entries this can become slow so that's why this Eloquent driver exists. As we're working with multiple environments (DTAP) we like to version control the configuration. Otherwise when we create a field, collection, global, etc and everything is stored within the database we've to redo this on every environment.

If we take global sets as an example; when I create a new set with some fields the blueprints are stored on the disk (as I configured it that way) but everything else is stored within the database. So when I commit and push this (so that's just the blueprint); the test environment doesn't have my new global set as it is stored within the database.

It would be nice to have all configuration in files and just the data in the database so all configuration can be version controlled.

Thoughts?

[ryanmitchell]

So on Globals you want the 'global set' (handle, title etc) to be stored in file, but the 'variables' within the global to be in the database?

[royduin]

Correct, but this also applies to the navigation and probably more parts.

[ryanmitchell]

I'd have no objection to that as long as it's configurable and you're happy to PR it.

It wouldnt be a breaking change if you make it an option in for each repository in the config file (which defaults to the current behaviour).

[royduin]

Aweomse! Am I the first one reporting something like this? Curious how others work with this. Maybe we've to change our workflow? I mean; if everything is stored within the database you've to redo your changes on all environments where you end up merging data as the end user / customer continues to edit data on production.

[ryanmitchell]

First time I've seen it be asked. We tend to seed data so this doesn't affect our workflows.

[royduin]

Automated the creation of a migration with the global set for the time being so at least the global set is created on all environments but splitting the data from the configuration is a better idea. As soon as we find some time to investigate that for all components we'll create a PR. Thanks for your quick replies!

AppServiceProvider.php

Event::listen(function (GlobalSetCreated $event) {
    $globalSet = $event->globals;

    $filename = database_path(
        'migrations/'.date('Y_m_d_His').'_create_statamic_globalset_'.$globalSet->handle().'.php'
    );

    $content = str_replace(
        '{{ data }}',
        json_encode(Arr::except($globalSet->model()->getRawOriginal(), 'id')),
        file_get_contents(resource_path('stubs/migration.globalset.stub'))
    );

    file_put_contents($filename, $content);

    $globalSet->model()->delete();

    Artisan::call('migrate');
});

migration.globalset.stub

<?php

use Illuminate\Database\Migrations\Migration;
use Illuminate\Support\Facades\DB;

return new class extends Migration
{
    public function up()
    {
        DB::table(config('statamic.eloquent-driver.table_prefix', '').'global_sets')->insert(
            json_decode('{{ data }}', true)
        );
    }
};

Edit: created a package with this: https://github.com/justbetter/statamic-eloquent-driver-globalset-migration-generator

[ryanmitchell]

I've opened a PR here: https://github.com/statamic/eloquent-driver/pull/168 Maybe you could pull it down and see how it works for you?

[royduin]

🚀 Thanks!