search

bcc-code / directus-schema-sync

The better way to sync your Directus schema and data between environments
Apache License 2.0
90 stars 4 forks source link

Ability to see if there were changes during an import #40

Closed martin-braun closed 3 weeks ago

martin-braun commented 1 month ago

Describe the Improvement

I'm using Ansible for my deployment. A crucial feature of Ansible is the ability to have a changed_when expression for tasks, so I'm looking for a way to reliably detect if an import actually changed anything. My first guess was the mv_hash in the directus settings would change, so that I could have a shell task that executes this:

current="$(psql -XAtc 'SELECT mv_hash FROM directus_settings LIMIT 1;')"
next="$(cat 'schema-sync/hash.txt' | cut -d '@' -f -1)"
printf "Current: $current\nNext: $next"
test "$current" = "$next"

A non-zero return code would indicate the next import will change my schema and data, but apparently this is not the case, the mv_hash doesn't match the contents of the hash.txt after import, so I guess I was wrong on that.

I also inspected the output of npx directus schema-sync import, but nothing indicates that changes where made, so I couldn't even parse stdout.

Could we please do something about it? I need a reliable way to see if changes where made.

u12206050 commented 1 month ago

I'm assuming you are talking about changes to the schema right? Since there is some logging indicating updates/changes done to data.

martin-braun commented 1 month ago

@u12206050 Schema and data on schema-sync import and just the schema on schema-sync import-schema respectively. I started testing this only after changing the schema, so the logging of data changes went over my head. Is this logged to stdout?

In the end parsing the stdout works, but it's always a bit of a hack. I would kindly enjoy if there was a system in place to hash the state before and after. Even on a database level would work for me, if it's not a hog. I'm using PostgreSQL by the way.

martin-braun commented 1 month ago

I just re-test schema-sync import --merge and all it prints is INFO: Importing everything from: ..../schema-sync/data and this info also prints when I re-run the command despite there being no new data to import.

u12206050 commented 3 weeks ago

Closing this as not planned

martin-braun commented 3 weeks ago

@u12206050 I respect your decision, after all this is a very nuanced problem and Ansible is not a direct dependency of the project, although I have two questions:

  1. Would you still be interested in a PR to address this?
  2. Could you still give some thoughts on how I could detect schema and data changes?

Thanks.

u12206050 commented 3 weeks ago

Sure a PR is welcome.

Neither schema nor data changes should be too hard to compare and print of the diff. The main problem I see is displaying the diff in a reasonable understanding way.

martin-braun commented 3 weeks ago

@u12206050 maybe a verbose flag could help, it could print much more without disrupting the default behavior?

u12206050 commented 3 weeks ago

Yes that should be very doable. I believe in the code where we are comparing db rows to file rows would be the ideal place for this. Displaying sensible output including schema changes might still be hard though.