Remove key-value pairs with no blueprint field

[aerni]

I've been running into this many times and finally got around to filing an issue about what I believe to be a missed angle regarding the fundamentals of Statamic's data flow.

How should data be handled if there is no corresponding field in the blueprint?

Currently, you can add any key-value pair to an entry's file, and it will be available whenever you access an entry's data. This is an easy way of making additional values available without the need for a blueprint field. However, this concept can also lead to unwanted behavior as outlined below.

Example

Let's tackle this with a simple example of a collection of songs. The song blueprint contains a toggle field called downloadable. If the toggle is set to true, we will make the song available for download in the view.

Blueprint

title: Song
sections:
  main:
    display: Main
    fields:
      -
        handle: downloadable
        field:
          default: false
          display: Downloadable
          type: toggle
          icon: toggle
          localizable: false
          listable: hidden
          instructions_position: above
          visibility: visible
          always_save: false

Content

---
id: 1d366a68-855e-4145-81ea-87174826a4eb
blueprint: song
title: One
updated_by: 3984c1bc-0af3-4ee7-a953-52b0a832704c
updated_at: 1671636776
downloadable: true
---

View

{{ if downloadable }}
    Download
{{ /if }}

The issue

After some time, we no longer want the songs available for download. By now, the collection of downloadable songs has grown to the thousands. We remove the downloadable field from the blueprint and call it a day. But we soon realized that the songs were still available for download because we forgot to also remove the downloadable key-value pair from each song's file.

If Statamic filtered any key-value pair that doesn't have a corresponding blueprint field, this issue would never have happened.

(In this simplified example, we could have simply removed the download button from the view. But in more complex setups with many moving parts, things might not be as straightforward.)

Solution

I expect this to work similarly to how the data flow of conditional fields is handled. If a conditional field is invisible when an entry is saved, the field's data will be removed from the entry. What if we removed all key-value pairs from the entry that don't have a corresponding blueprint field? (There are some obvious exceptions, such as id or updated_by, etc.)

If someone still wants additional data, you can use the new computed values.

Pros Up-to-date data with no legacy fields. Simply saving an entry will ensure its data is up to speed.

Cons A breaking change that might cause issues for people relying on additional entry data. But we could simply make this new data flow optional similar to a field's always_save option.

[jasonvarga]

This would completely break the workflow of people who don't use blueprints at all. e.g. create entries without using the cp and just put arbitrary data in them.

But in general I think it is a good option to have, somehow.

[aerni]

Good point. I haven't thought of that.

[aerni]

This would completely break the workflow of people who don't use blueprints at all. e.g. create entries without using the cp and just put arbitrary data in them.

I just realized that this is handled differently for views and GraphQL. Views will have all data available that's saved in an entry. GraphQL only makes actual blueprint fields available. So there's a conceptual gap as well.