Bad Request: "You need to omit a field before deleting it"

[wiledal]

Hey,

I cannot import a space that has deleted fields in a content model.

info import Contentful Import Tool
info import Importing data ...
info Checking if destination space already has any content and retrieving it
info Pushing content to destination space
ERR! import BadRequest: {
ERR! import   "status": 400,
ERR! import   "statusText": "Bad Request",
ERR! import   "message": "You need to omit a field before deleting it",

Basically, we need to "Disable in response" before we can delete a field.

I get that this might be a design choice when it happens in the visual CMS, but it makes properly syncing spaces impossible.

Could we add a --force flag to allow "insecure updates" to happen?

[axe312ger]

There is no option to enable insecure updates in our API.

To properly migrate your content types, we just released https://github.com/contentful/migration-cli which takes care of omitting fields before deleting them.

I recommend you to use the --skip-content-model to import content and use the migration-cli to do your field transformations.

[axe312ger]

Okay, we discussed this and decided to integrate omitting of the fields to this tool as well. I still recommend the migration-cli for more complex usage, but expect a feature release for the import tools soon :)

[wiledal]

Hey, that migration tool looks powerful, but building the models becomes a less intuitive process. We enjoy using the visual CMS to build the content model for rapid results.

We use a Staging and a Production environment. The Staging is where we quickly do CMS updates to see what works best. Some models will change tens of times before we settle. Having to write migrations for those scenarios would slow down the experimentation process, and clutter our repository with hundreds of migration states. Being able to simply clone the most recent state when we are ready to push the corresponding code update is the faster option for us.

Thanks!

[wiledal]

Any update on this? Would be very useful to us currently.

[floelhoeffel]

Hey @wiledal - we know with the current offering working with multiple environments is a bit tricky. However we are about to release some new features on the API side which will make this a lot easier soon. Another thing we are considering for the migrations tooling is to support a workflow where you can generate a migration script from an existing content model.

In the meantime, we recommend to manually remove the field in the target space from the CT and try the import again. Import and export are not designed to write into non-empty spaces, it works in many cases but there is no logic in place to resolve conflicts.

Hope this helps

[floelhoeffel]

The reason why we don't want "merging/syncing" functionality in the import export tooling is that there is so many side effects. The only way to handle that fail save is through a migration script. As mentioned in my previous message, generating a migration script from a web app curated content model is a very interesting use case we are considering to add to the tooling.

[dnlmzw]

Any update on this? Also curious as the dev, staging, production cycle gets a little cumbersome at the moment.

[lukearmstrong]

Discovered this issue searching for the error message, and it's disappointing to see that whilst this has been raised several years ago, and bumped several times by people with the same issue, it was never fixed, and instead just closed.

That's a bit crap as the web interface doesn't allow you to do any sort of export or import. Nor does it allow you to do a mass-delete on content models. Even when you need to clear out content, this is limited to 40 entries during the mass-unpublish/mass-delete dance (providing all entries have the same status 🙄)

We use multiple environments across multiple spaces, for our Development, QA and Production websites.

During the current phase of development I have removed fields from several of our Content Models, and when running an export from our development environment everything is fine.

npx contentful space export --config ./config/export-content-and-models.json --space-id {{ SPACE_ID }} --environment-id {{ ENVIRONMENT_ID }}

# ./config/export-content-and-models.json
{
  "include-drafts" : true,
  "skip-content-model" : false,
  "skip-content" : false,
  "skip-roles" : false,
  "skip-webhooks" : true,
  "content-only" : false,
  "download-assets" : false,
  "save-file" : true
}

When I come to import I then get the errors above.

npx contentful space import --config ./config/import-content-and-models.json --content-file contentful-export-SPACE-ENVIRONMENT-2019-11-18T16-56-66.json --space-id {{ SPACE_ID }} --environment-id {{ ENVIRONMENT ID }}

# ./config/import-content-and-models.json
{
  "content-model-only" : false,
  "no-update" : false
}

The error message doesn't even use the same language as the web interface.

You must omit a field before deleting it

• Doesn't tell you which field.
• Doesn't tell you which content model. (you can determine this from the request URL)
• Doesn't tell you that "omit" means edit the field and mark it as "Disable in Response".

I see this has been raised as an issue over a year ago and still not addressed. https://github.com/contentful/contentful-cli/issues/74

But more importantly, why doesn't the export and import process just assume you want to wipe the target and replace all data from the source? It's confusing because this error shouldn't even be thrown as the field it is complaining about doesn't even exist in the data being imported, it only exists in the existing data you are expecting to overwrite.

To resolve this I'm going to have to write a script that goes through all the content, deletes it, and then goes through all the content models, loop through all fields, marks them as disabled from response, save them, and then delete the content models.

If that method works, I will share this script here so that it may help others who find this issue.

[orta-sanz]

Thanks @lukearmstrong I also don't understand why this has been closed

[makinwab]

@ruderngespra, can you have a look at this? The most up to date comment has a lot of info on what the issue is.

I'll be closing out a similar issue in the CLI because the import tool is what needs to be looked at.

[lukearmstrong]

@makinwab @ruderngespra In the end to resolve this I wrote some scripts which did this:

• Backup Source using CLI tool
• Backup Destination using CLI tool
• Write a script that connects to your API, and:
  • Loop through all Content Entries (Content), unpublish, then delete.
  • Loop through all Content Types (Content Models), unpublish, then delete.
• Import source.json into Destination using CLI Tool.

A really quick way of doing the same thing in the web control panel is just to delete your destination environment, and create a new one based on the source. This only works if your Environments are in the same Space. Ours were not.

[piyu-sh]

@makinwab @ruderngespra In the end to resolve this I wrote some scripts which did this:

• Backup Source using CLI tool
• Backup Destination using CLI tool

• Write a script that connects to your API, and:

  • Loop through all Content Entries (Content), unpublish, then delete.
  • Loop through all Content Types (Content Models), unpublish, then delete.
• Import source.json into Destination using CLI Tool.

A really quick way of doing the same thing in the web control panel is just to delete your destination environment, and create a new one based on the source. This only works if your Environments are in the same Space. Ours were not.

Hey @lukearmstrong, we are also stuck in a similar problem. We want to be able to sync model changes across different environments. And contentful doesn't provide any automated way of doing this consistently. Would you be able to share this script that you've developed, it would be a great help ?

[AntonioRedondo]

Any update on this issue? Due to it importing content models between different environments effectively doesn't work. It's really disappointing that a programmatic headless CMS software hasn't addressed such a serious bug reported almost 4 years ago.

[perevernihata]

6 years now @AntonioRedondo. Still, errors for the fields emitted must be resolved manually. Imaging DB schema migration tool where you have to delete columns manually. --force would be easiest solution ever for this

[Tim-Pet]

any updates on this?

[erockdotdev]

7 years :/