Feature request: Full data export for database migrations and backups

[sanderkooger]

Support to export ( and import) all data in fusionauth

Problem

Fusionauth has no method to export all data (settings, config, and users) or bulk import such data. In our case, we would like to be able to do this because of a migration from MySQL to Postgres, but there are more examples imaginable.

Solution

an export function that dumps all fusionauth data in a manner that can be imported into fusionauth again

Alternatives/workarounds

I have tried migrating the data manually, tried to migrate the data in the databases, this results in the weirdest errors due to inconsistent data.

Related

• https://github.com/FusionAuth/fusionauth-issues/issues/1953

How to vote

Please give us a thumbs up or thumbs down as a reaction to help us prioritize this feature. Feel free to comment if you have a particular need or comment on how this feature should work.

[sanderkooger]

@onair-lena, @checco,

[mooreds]

We have some options to configure FusionAuth (outlined here: https://fusionauth.io/docs/v1/tech/admin-guide/configuration-management ), and you can move user password hashes with direct database access, but you are correct, there's no standard way to move FusionAuth configuration wholesale.

May have some overlap with https://github.com/FusionAuth/fusionauth-issues/issues/526

[brianjlacy]

This is a pretty critical feature, is this on the roadmap yet?

[mooreds]

@brianjlacy I'm not sure I totally understand the use case for this. It seems to me like there are two possible reasons to want to do this:

• Case 1: you've set things up manually and want to be able to copy config to a different server
• Case 2: you want to migrate from FusionAuth to another provider

In case 1, what we've seen people do is either use our Terraform support or one of our client libraries paired with a migration framework. They introduce configuration management at the very beginning. This way all changes are version controlled and managed. This doesn't cover dynamic data like new users, but I'm not sure I understand why you'd want to manage user data in this way.

In case 2, you can get your database any time you want if you self-host. If you are in the cloud, your database is just a support ticket away, if you want to migrate off of FusionAuth--it's your data! Either way you will have to do all kinds of data munging to move from FusionAuth to the new system, so I'm not sure how bulk exporting configuration would work.

Finally, you can absolutely get almost everything via the API, but there would be some work removing readonly attributes (such as insert instant) before you can update a different FusionAuth system with the config.

With all that said, @brianjlacy , can you please explain what your use case is for this feature in a bit more detail?

[brianjlacy]

I'm obviously not the only one who wants this, but it's still surprising that FA has come this far without inport/export capability. It's an essential feature for any configuration-heavy platform or tool to be able to move that configuration around.

Server migrations and environment shifts, etc. are part of the natural lifecycle for any application, but especially in the early period when you're building out and preparing to deploy and then scale a new platform a lot of infrastructure restructuring tends to occur, and in our case there isn't much bandwidth to devote to scripting out these migrations.

We'll end up doing the extra work because we haven't got a choice, since the feature doesn't exist; but it would save us many hours if we could simply export a full backup from our localhost or sandbox environment once we've got everything setup correctly, and import it to other instances or environments -- then repeat as our application, infrastructure, and identity requirements rapidly evolve.

In my particular case I'm not so worried about individual users, necessarily (though to me it makes sense to allow us to specify which record types we're exporting/importing), but for everything else -- tenants, applications, roles, groups, etc. -- it would make my life as an engineer so much simpler if I could export it all from our dev environments and import in other environments with a simple UI export/import feature.

I'm honestly quite frustrated that I'm having to write scripts to do this myself via the API, it seems so fundamental.

[brianjlacy]

I'll add that in a former role, we were using an open source competing product which similarly had no reliable export/import, and we ended up spending dozens and dozens of hours troubleshooting issues that ultimately came down to inconsistencies in auth config between our dev, staging, and prod environments. Because we had different people managing each environment, subtle variances inevitably cropped up which led to some immensely frustrating bugs that only occurred in production and couldn't be easily/safely troubleshooted.

That said, Terraform seems like a reasonable solution to this problem long term; but we're simply not at the stage where we can invest resources in setting up a smooth automated deployment process at that level.

[mooreds]

@brianjlacy really appreciate you taking the time to explain your use case. I understand your frustration.

If I were to repeat back what I read and summarize, it would be:

You want the ability to configure an instance manually, then generate an output file which can be applied against a different instance. The main reason to do this instead of scripts or terraform is lowering the barrier to creating multiple environments with the same configuration.

Is that a fair summary? Again appreciate the feedback.

Finally, this is not on our near term roadmap, but appreciate you giving this the thumbs up, as we do consult this issue list, sorted by the number of upvotes when considering future work.