schema checks at server start

[dhiaayachi]

The server should quit early if the Persistence Store database has a mismatched schema. Ideally, there'd be a check when the server initially connects to the database. Logging an error for a schema mismatch for the Visibility Store would also be valuable telemetry.

If this has already been implemented please go ahead and close. Otherwise if there's interest I'd be happy to take this on.

[dhiaayachi]

Server should quit early if the Persistence Store database has a mismatched schema.

This issue proposes that the Temporal server should exit early if it detects a schema mismatch with the Persistence Store database. Ideally, this check would occur when the server initially connects to the database. Additionally, logging an error for a schema mismatch for the Visibility Store would provide valuable telemetry.

This feature would improve server stability and prevent potential issues related to database schema mismatches.

Here's a breakdown of the proposal:

1. Early Exit on Schema Mismatch:

• When the Temporal server starts, it should perform a schema validation check against the Persistence Store database.
• If a mismatch is detected, the server should immediately exit, preventing any further processing and potential data corruption.

2. Logging Schema Mismatch Errors:

• The server should log an error message whenever a schema mismatch is detected for both the Persistence Store and the Visibility Store.
• These logs would provide valuable telemetry for identifying and resolving schema mismatch issues.

3. Implementation:

• The implementation would likely involve adding a schema validation step during the database connection process. This could be achieved by querying the database for specific schema information and comparing it against the expected schema defined within the server.
• Error logging would be achieved through the server's existing logging infrastructure.

4. Benefits:

• Improved Server Stability: By exiting early on schema mismatch, the server prevents potential data corruption and ensures consistent behavior.
• Early Problem Detection: Logging schema mismatch errors provides valuable telemetry for identifying and resolving schema issues promptly.
• Simplified Debugging: Clear error messages and logs make it easier to troubleshoot schema related problems.

5. Existing Implementation:

• It is unclear if this feature is currently implemented. The issue author suggests that if it has already been implemented, the issue should be closed. Otherwise, they are willing to contribute to the implementation.

6. References:

• Temporal Server Configuration
• Temporal Service
• Persistence Store
• Visibility Store

7. Next Steps:

• If this feature is not yet implemented, the issue author should collaborate with the Temporal team to determine if it is feasible and desirable for the project.
• The implementation should be tested thoroughly and documented appropriately.

This proposal aims to enhance the reliability and stability of the Temporal server by detecting and addressing schema mismatches with the Persistence Store and Visibility Store.

[dhiaayachi]

Thank you for the feature request! We appreciate you taking the time to share your idea with us.

While we don't currently have this feature, we think it would be valuable to ensure schema consistency and provide better telemetry.

For now, you can manually check the schema by using the temporal-sql-tool to query the database, and use the server logs to monitor for schema mismatches.

We will consider this for a future release.

[dhiaayachi]

Thank you for the feature request! It would be helpful to have the server quit early if there is a database schema mismatch. Currently, Temporal has no mechanism to automatically detect schema mismatches and gracefully shut down the server.

One way to work around this limitation is to add a step to your deployment process that performs a schema validation check before starting the Temporal Server. You can use a tool like temporal-sql-tool to compare your database schema with the expected schema provided in the Temporal repository.

We appreciate you bringing this to our attention and will consider adding schema validation functionality in future releases.

[dhiaayachi]

Thanks for reporting this issue! It's a great idea to have the server quit early if a schema mismatch is detected in the persistence store. Currently, Temporal doesn't have this feature, but you can manually check for schema mismatches by using temporal-sql-tool and comparing the current schema with the schemas found in the schema directory.