customize.py should opt out if `migration_hash` is the same (no changes detected)

[MyuddinKhatri]

@agritheory During migration, Frappe checks the migration hash for standard DocTypes. If there are any changes in the JSON file, it returns a different hash and then syncs the changes accordingly. https://github.com/frappe/frappe/blob/d1f5e14e8e18b1e2b9c80fdf7bf567fc663b7202/frappe/modules/import_file.py#L141

However, when I used frappe.db.get_value("DocType", "BOM Scrap Item", "migration_hash"), it returned the same value as calculate_hash("/apps/erpnext/erpnext/manufacturing/doctype/bom_scrap_item/bom_scrap_item.json") (without any changes in the file). After I changed the standard bom_scrap_item.json, calculate_hash returned a different value. Upon running bench migrate, the changes got synced and both hashes matched.

The hash for our customized file and the database value will always differ. I didn't find anything that stores the hash for the customized DocTypes. calculate_hash("apps/beam/beam/beam/custom/bom_scrap_item.json") imported caluculate_hash from https://github.com/frappe/frappe/blob/d1f5e14e8e18b1e2b9c80fdf7bf567fc663b7202/frappe/modules/import_file.py#L14 I am not sure how should we proceed to check for migration_hash for customized doctypes. I would like to know your thoughts on this.

[agritheory]

@MyuddinKhatri Summarizing our conversation from the call: We want to reimplement the hashing Frappe does but extend it to all custom/*.json files for that doctype for all applications. This will detect if there's been a deterministic change in schema based on this static analysis and not migrate if there's no change but there have been user-applied customizations. I think its appropriate to add our own column to the database the same way that Frappe does but with a different name (customization_hash).

[MyuddinKhatri]

@MyuddinKhatri Summarizing our conversation from the call:

We want to reimplement the hashing Frappe does but extend it to all custom/*.json files for that doctype for all applications. This will detect if there's been a deterministic change in schema based on this static analysis and not migrate if there's no change but there have been user-applied customizations. I think its appropriate to add our own column to the database the same way that Frappe does but with a different name (customization_hash).

How should we handle if same doctype is customized by multiple apps? Should we include custom app name in the column?

[agritheory]

How should we handle if same doctype is customized by multiple apps? Should we include custom app name in the column?

Compute the hash on multiple files as concatenated text. This has an outside chance of triggering a migration due to whitespace, but I think that's acceptable.

[MyuddinKhatri]

How should we handle if same doctype is customized by multiple apps? Should we include custom app name in the column?

Compute the hash on multiple files as concatenated text. This has an outside chance of triggering a migration due to whitespace, but I think that's acceptable.

Okay, we can implement this.

[agritheory]

Some examples:

1

• Frappe makes changes to the Address doctype (frappe/contacts/doctype/address/address.json). For example let's say that they add a new option of "Foreign" to the address_type for reasons that are unimportant. When the schema is synced it should detect this change and apply the customization.
• US Regional also has customizations to Address (us_regional/us_regional/custom/address.json) but they have not changed. For example the address_type could be the normal list plus "Home"
• ERPNext has customizations for Address but they have not changed

We should reapply our customizations. If we don't, Frappe's version of the schema will be the only one that gets applied and we loose our changes to address_type. This is why we need to include the doctype/doctype.json file in the hash that computes a change in schema.

2

• ERPNext makes changes to the Employee doctype (erpnext/setup/doctype/employee/employee.json)
• HRMS makes changes to the employee doctype (applied as code and therefore ignored)
• US Regional also has customizations to Employee (us_regional/us_regional/custom/employee.json) and they have changed
• A client app also has customizations to Employee (client_app/client_app/custom/employee.json) and they have changed

We need to reapply all customizations from all apps when we detect there's a customization that has changed. Frappe doesn't do this, it only applies the customizations from the last app in installed apps/ "hooks resolution order". We should respect the installed apps order to give the customizations the correct priority.