When translating Hive view definitions to a format that Trino can interpret and process, the Coral library is converting DECIMAL data types to DECIMAL(12,0) in its convertQuery() method. This is causing a type mismatch when the actual data in the view is of type DECIMAL(19,0), leading to a VIEW_IS_STALE exception in Trino.
Scenario:
A Hive view is created with a column of type DECIMAL(19,0) that holds values such as 12345623423423.
This view is read by Trino, which uses the Coral library for translation.
Coral incorrectly converts the DECIMAL(19,0) type to DECIMAL(12,0).
When the view is queried, Trino checks if the actual data type (DECIMAL(19,0)) can be coerced to the defined data type (DECIMAL(12,0)).
This check fails, leading to a VIEW_IS_STALE exception.
Suspected Fix:
The suspected fix involves updating the convertQuery() method in Coral to correctly translate DECIMAL data types without reducing precision. The method should be able to handle and preserve larger DECIMAL types such as DECIMAL(19,0). This change will ensure that the translated data type in Trino matches the actual data type in Hive, avoiding the VIEW_IS_STALE exception.
Proposed Solution:
Update the convertQuery() method in the Coral library to correctly handle and preserve the original precision of DECIMAL data types during translation. This would involve investigating how the current conversion to DECIMAL(12,0) is happening and updating the logic to maintain the original precision of the data type. We also need to ensure that this change does not adversely affect other parts of the Coral library or its functionality.
Akanksha-kedia
commented
1 month ago
Description:
When translating Hive view definitions to a format that Trino can interpret and process, the Coral library is converting DECIMAL data types to DECIMAL(12,0) in its convertQuery() method. This is causing a type mismatch when the actual data in the view is of type DECIMAL(19,0), leading to a VIEW_IS_STALE exception in Trino.
Scenario:
A Hive view is created with a column of type DECIMAL(19,0) that holds values such as 12345623423423. This view is read by Trino, which uses the Coral library for translation. Coral incorrectly converts the DECIMAL(19,0) type to DECIMAL(12,0). When the view is queried, Trino checks if the actual data type (DECIMAL(19,0)) can be coerced to the defined data type (DECIMAL(12,0)). This check fails, leading to a VIEW_IS_STALE exception.
Suspected Fix:
The suspected fix involves updating the convertQuery() method in Coral to correctly translate DECIMAL data types without reducing precision. The method should be able to handle and preserve larger DECIMAL types such as DECIMAL(19,0). This change will ensure that the translated data type in Trino matches the actual data type in Hive, avoiding the VIEW_IS_STALE exception.
Proposed Solution:
Update the convertQuery() method in the Coral library to correctly handle and preserve the original precision of DECIMAL data types during translation. This would involve investigating how the current conversion to DECIMAL(12,0) is happening and updating the logic to maintain the original precision of the data type. We also need to ensure that this change does not adversely affect other parts of the Coral library or its functionality.