search

k1LoW / tbls

tbls is a CI-Friendly tool for document a database, written in Go.
MIT License
3.37k stars 165 forks source link

doc schema.json differs when no schema changes exist #455

Open awhitford-cip opened 1 year ago

awhitford-cip commented 1 year ago

The schema.json file generated by doc may differ between runs when no schema changes exist.

I have a workflow where doc output is generated and we look for differences. I often see differences in the schema.json file even when there were no database schema changes. Note that I am witnessing this using Snowflake.

I would expect that the schema.json file to be consistent across runs when there are no database schema changes. (Maybe the output needs to be sorted to be consistent across runs.)

Additionally, it would be nice to see this file as not one giant line so that it is easier to see the actual differences between runs. (I'm unclear on what is changing because of this property.)

k1LoW commented 1 year ago

@awhitford-cip Thank you for your report

Unfortunately I do not have a Snowflake environment ( so Snowflake is experimental support). If you have found a schema that can be reproduced in another open source database, please let me know.

awhitford-cip commented 1 year ago

❄️ Snowflake offers a 30-day free trial.

k1LoW commented 1 year ago

I have already used the trial for Snowflake support 💦

awhitford-cip commented 1 year ago

Additionally, it would be nice to see this file as not one giant line so that it is easier to see the actual differences between runs. (I'm unclear on what is changing because of this property.)

https://github.com/k1LoW/tbls/blob/a21522e27f4209a38e440fd7a79b126c5656fd67/cmd/doc.go#L171

Why is inline set to true here? (I think if it was set to false then it would have newlines and indentation. Then it would be easier to see what changed.)

awhitford-cip commented 1 year ago

Overall, I am going to try specifying disableOutputSchema: true to skip the schema.json file as I don't think it is necessary.

k1LoW commented 1 year ago

Why is inline set to true here?

Here is why it is inline.

MovaUA commented 7 months ago

Hi, Guys :) The same issue is discovered by our CI pipelines. We use PostgreSQL 15, tbls 1.72.0 from the docker image, the pipeline calls tbls in Ubuntu 22.04.

k1LoW commented 7 months ago

@MovaUA Could you tell which field has the difference?

MovaUA commented 7 months ago

@MovaUA Could you tell which field has the difference?

First of all, I would like to say a great thanks to @k1LoW for this amazing project! It helps us a lot to review the database changes in my project!

Sure. We found that the sorting in an array of column names in tables.constraints.columns is fluky - it kind of unpredictably produces items in different order for the tables that have not been changed. Also, the order of column names in that array does not correspond to the actual order of the columns in the constraint.

👉 Note that we use format.sort: false in .tbls.yml

k1LoW commented 7 months ago

Sure. We found that the sorting in an array of column names in tables.constraints.columns is fluky

Thank you for your report!