Improve the UX of moving "stuff" to the database

rtrudel commented 3 weeks ago

Context

When installing the eloquent package, users are presented with repository choices without clear explanations of their purposes. This is particularly challenging for users migrating from other CMS platforms (like Craft CMS, WordPress) where terminology and concepts may differ.

Current Issues

Repository selection during eloquent package installation lacks context
No clear distinction between content and structural elements
Insufficient guidance on environment-specific configurations
Missing cross-CMS terminology reference
Unclear content synchronization behaviors between flat-files and database

Proposed Improvements

1. Repository Documentation

Clear explanation of each available repository type
Use cases for each repository
Implications of choosing specific repositories
Database vs flat-file storage considerations

2. Content vs Structure Classification

Clear guidelines on what constitutes content vs structure
Best practices for managing each type
Migration considerations from other CMS platforms

3. Content Synchronization Behaviors

Detailed explanation of content priority rules:
- How the system handles content existing in both flat-files and database
- Which source takes precedence and why
- Configuration options for priority settings if available
Import/Export Behaviors:
- What happens to flat-files after database import
- What happens to database tables after flat-file export
- How to handle conflicts during synchronization
- Best practices for backup before import/export operations
Clear documentation of cleanup operations:
- Whether files are deleted after import
- Whether tables are truncated after export
- How to maintain data integrity during these operations

4. Environment Management

Guidelines for Git workflow
Examples of .gitignore configurations for:
- Flat-file setups
- Database-driven setups
- Hybrid approaches
Best practices for managing content across different environments

5. Migration Guide

Terminology mapping between common CMS platforms
Common migration patterns
Pitfalls to avoid

6. Configuration Examples

Include practical examples for common setups:

Pure flat-file configuration
Database-driven setup
Hybrid approach
Environment-specific considerations
Content synchronization scenarios with expected outcomes

Expected Benefits

Reduced learning curve for new users
Better understanding of system architecture
Improved content/structure separation
More reliable deployment processes
Clear understanding of content synchronization behaviors
Safer content migration operations

Implementation Suggestions

Add a new section in the main documentation
Include visual diagrams for complex concepts
Provide practical examples with common use cases
Add a glossary of terms with CMS-specific mappings
Include decision trees for content priority scenarios
Add warning boxes for potentially destructive operations (if applicable)

rtrudel commented 3 weeks ago

If I may suggest: Adding a CLI command that would provide certain presets to simplify storage strategy management and make it easy to switch between different configurations.

The data:schema command could help manage storage strategies for both content and structure:

# Predefined storage strategy presets
php please data:schema --preset=flat-file          # Everything in files
php please data:schema --preset=database-only      # Everything in database
php please data:schema --preset=hybrid-standard    # Content in DB, structure in files

# Custom storage strategy configuration
php please data:schema --content=db --structure=file

# Granular repository configuration
php please data:schema --collections=file --entries=db --forms=db --taxonomies=file

This would:

Simplify the migration process
Make it safer to experiment with different storage configurations
Provide clear visibility into current storage strategy
Allow for granular control over repository storage
Include safeguards against data loss

The command could also provide helpful feedback about the implications of each change and ensure data integrity during transitions.

duncanmcclean commented 3 weeks ago

Good idea!

I think we can probably adjust the install:eloquent-driver command (which is the CLI wizard you see) to ask what you want to move:

Everything
Only move content (keep configs flat file)
Choose specific repositories

^ those options would also be available when creating new sites

We can also improve the existing documentation around the Eloquent Driver to mention content vs config repos to clarify what actually gets moved.

rtrudel commented 3 weeks ago

I believe indeed that what I call structure is named config here, but we're talking about the same thing :)

rtrudel commented 3 weeks ago

Since Statamic's default mode is flat-file, it could also be interesting to have a simple way to manage .gitignore files if we wanted, for example, to keep everything without a database, while still ensuring that content remains independent between environments. This could also be kind of preset "flat-config" or something.

duncanmcclean commented 3 weeks ago

Since Statamic's default mode is flat-file, it could also be interesting to have a simple way to manage .gitignore files if we wanted, for example, to keep everything without a database, while still ensuring that content remains independent between environments. This could also be kind of preset "flat-config" or something.

That's probably a little opinionated for us to do in Core. As far as I know, most folks using flat file share the same content between sites.

However, you're more than welcome to do that in your own projects or create a starter kit that does that (although, that would be a very small kit 😄).

rtrudel commented 3 weeks ago

That's probably a little opinionated for us to do in Core. As far as I know, most folks using flat file share the same content between sites.

I understand the concern, and probably I just do not "get it", but I'm curious how a site would work when being modified by several developers working locally (so push/pull their changes without conflicts), while the site is in production and users or writers are modifying its content (ex: new articles, e-commerce orders, user comments and reviews, etc.). Why would we sync locally all the website content when only config matters on local environment? As I said, I'm just unsure to understand this bit, maybe the answer is very simple and I'm overcomplicating things...

ryanmitchell commented 3 weeks ago

That's probably a little opinionated for us to do in Core. As far as I know, most folks using flat file share the same content between sites.

I understand the concern, and probably I just do not "get it", but I'm curious how a site would work when being modified by several developers working locally (so push/pull their changes without conflicts), while the site is in production and users or writers are modifying its content (ex: new articles, e-commerce orders, user comments and reviews, etc.). Why would we sync locally all the website content when only config matters on local environment? As I said, I'm just unsure to understand this bit, maybe the answer is very simple and I'm overcomplicating things...

We often pull content down so we are working from the latest client content, especially when developing new functionality. You get around conflicts by using a branch other than main/master.

statamic / ideas