When running an ALTER TABLE Hive stores the types of columns from table schema in sequential order to enforce the same types after the ALTER TABLE. However when performing our abstraction of DROP COLUMN, the types may change, since the schema will definitely change.
Currently, if we try to drop a column that changes this 'types order', the following error returns:
thrift_files.libraries.thrift_hive_metastore_client.ttypes.InvalidOperationException: InvalidOperationException(message='The following columns have types incompatible with the existing columns in their respective positions :\ncolumn_name')
What? :wrench:
Adding a call to change metastore settings while in drop_columns_from_table:
Allowing, in the current execution, the changing of types when comparing the table schema prior and after ALTER TABLE.
Adjusting tests
Type of change :file_cabinet:
[X] Bug fix (non-breaking change which fixes an issue)
How everything was tested? :straight_ruler:
Ran the config command both alone and together with the drop_columns_from_table and it only succeed in the same 'transaction'.
Checklist :memo:
[X] I have added labels to distinguish the type of pull request.
[X] My code follows the style guidelines of this project (docstrings, type hinting and linter compliance);
[X] I have performed a self-review of my own code;
[X] I have made corresponding changes to the documentation;
[X] I have added tests that prove my fix is effective or that my feature works;
[X] I have made sure that new and existing unit tests pass locally with my changes;
Why? :open_book:
When running an ALTER TABLE Hive stores the types of columns from table schema in sequential order to enforce the same types after the ALTER TABLE. However when performing our abstraction of DROP COLUMN, the types may change, since the schema will definitely change.
Currently, if we try to drop a column that changes this 'types order', the following error returns:
What? :wrench:
drop_columns_from_table
: Allowing, in the current execution, the changing of types when comparing the table schema prior and after ALTER TABLE.Type of change :file_cabinet:
How everything was tested? :straight_ruler:
Ran the config command both alone and together with the
drop_columns_from_table
and it only succeed in the same 'transaction'.Checklist :memo: