Help: trying to migrate schema

[pingu2k4]

Hey,

looks like a great package, thanks. I'm having a little trouble using it however, sorry.

My intended usage is to run CLI commands to store details of my DB schema, and then run CLI commands to push this schema to a database on a new project at a new location.

So I would have my localhost DB, I design the schema as whatever, and when I am happy I extract the schema with CLI. Then I commit this to my repo, and when I PR to staging branch, it runs my CI/CD where it will push said schema up to my staging DB hosted elsewhere. Then the same when I PR into prod, but with my prod instance.

Is this correct / intended usage?

I've run appwrite-setup and it was complaining that there wasn't a src/ folder so I created that and ran it again and now I have some config which I have played about with a little.

Trying to run appwrite-setup or appwrite-migrate with --help as a flag doesn't provide any help, and it just tries to execute something instead so I am struggling with what I should be doing...

I've modified the config yaml file with my endpoint, project ID and appwrite key. It looks like I need to define my databases and collections here also. The permissions and attributes for collections seem to be required, but I expected that this would not be the case and instead they would be extracted when running a migration to store the schemas?

I don't really understand what I should do from this point sorry.

If it helps, I would be able to write some js script that I can run rather than doing it all with CLI, but have no idea where to begin with that if needed.

Thanks in advance! :)

[ZachHandley]

Hey! What a great issue, I love things like this because it helps me further point the CLI tool in the direction that is needed.

So if you delete the collections in the created YAML file, and then run the appwrite-utils-cli -- sync, will that work for what you're trying to achieve?

Ah I reread it right now -- so the intended usage is to import data and keep your prod, staging, and dev databases in line, schema wise, throughout the cycle. My general Usage is to use it to keep my attributes in sync with my DB, and to import data, backup, and generate schema. How would you use it? Like you described? Trying to grow it's instructions so it's easier to use

Just woke up, so give me 30 minutes to an hour and I'll come back with a better answer if that isn't the issue ♥️

[pingu2k4]

Heya, thanks for the response! :)

I have a busy saturday today, so not able to test too much right now sorry...

My intended usage has nothing to do with the actual data (documents I believe is the terminology for appwrite - Appwrite itself is very new to me right no, so apologies!)

I am just looking to update the schema, so the DB, collections and attributes.

Intended use, is that I dev using locally hosted AppWrite, and make changes to the DB schema in the console. When I am happy with everything and I am ready to make a PR into staging branch, I will run something to create a config definition of the schema. Then when I PR, my CI/CD stuff as part of deployment will take a look at what is in that saved schema, compare it to my running staging instance, and make any changes necessary. So if I added a database, it will add it. If I added a collection, it adds it. If I modify a collection, or its attributes, it will copy the same thing.

I've taken a look at using the AppWrite CLI stuff, and it mainly works but in the process it drops all the actual data. (Specifically, running appwrite deploy collection - This will drop the data from all documents.

[pingu2k4]

Hey @ZachHandley I've been ill over the past few days (and still am), so haven't had the chance to look at this that I was expecting sorry.

I've had a test, and I managed to get something working for what I would need, but its a little finicky right now...

So after I run appwrite-migrate --init to generate the file, and then alter the endpoint etc as required, I point it towards my dev instance.

I remove all the collections, but point it at my DB in config.

Then I run appwrite-migrate --sync and it will grab the schema from my dev DB.

Now, the finicky part... My staging instance will have a different endpoint, may or may not have a different project ID, and will have a different key... So, I need to maintain this config file, but replace these 3 values.

Its doable, but it would mean that my intention would be to commit the config file to my repo, pointing at my dev localhost instance, and sync it before PRing. Then, my workflow would need to replace the endpoint, project ID and key before running the next stage.

Then, to push the schema changes up to the staging DB, I would run appwrite-migrate --prod, and this would push the changes in config up to my staging instance.

The same would then be done for prod as well.

Does this sound roughly as intended usage? It would be great if we could override the endpoint, app id and key in the CLI itself - If we specify them in the CLI then ignore them in the yaml file. Would just make things a little easier.

Am I doing something siginificantly different here to what was expected?

Side note, I also came across a bug, which may or may not be related to your code (It could be coming from AppWrite) but will create a seperate issue for this.

[ZachHandley]

Understood, I like that, I was thinking of using the staging as just the Appwrite Database in general, but that works too! I'll add a local development option to it for the CLI and for the config

[pingu2k4]

Thanks. :)

I have to look at the source code a bit to figure out what flags I could set in the CLI, as the --help mentioned in docs wasn't giving me anything :P

I'm still early in working with Appwrite in any sense right now, so I might change what I think about things etc but from what I am thinking right now, the ability to override the config settings for endpoint, app ID and key would be great - from what I can tell it would mean that I could write some GH actions to deploy my schema as part of my deployment process. :)

[ZachHandley]

@pingu2k4 I added the --endpoint, --project, and --key to the tool, fixed the enum bug, and I am almost done adding automatic OpenAPI generation via the zod-to-openapi package, I will most likely end up finishing that this weekend, but maybe tomorrow/Friday. Let me know if it works how you need it to :)

Don't forget to use --package=appwrite-utils-cli@latest so it installs the latest one on run