Shopify / ghostferry

The swiss army knife of live data migrations
https://shopify.github.io/ghostferry
MIT License
694 stars 65 forks source link

mediumint not recognized as numeric type #293

Closed HemeraOne closed 3 years ago

HemeraOne commented 3 years ago

following error occurs when table has mediumint as primary ID, both signed and unsigned are not working.

Error:

ERRO[0000] invalid table                                 error="Pagination Key `id` for `database`.`table` is non-numeric" tag=table_schema_cache
ERRO[0000] fatal error detected                          errfrom=ferry.initialize error="Pagination Key `id` for `database`.`table` is non-numeric" tag=error_handler
error: failed to initialize ferry: Pagination Key `id` for `database`.`table` is non-numeric

primary column:

`id` mediumint(8) unsigned NOT NULL AUTO_INCREMENT
shuhaowu commented 3 years ago

Interesting. I don't see why this would be the case. The relevant code in ghostferry is here:

https://github.com/Shopify/ghostferry/blob/40ac9a891e881b10b892ad28988648588b215a70/table_schema_cache.go#L271-L273

We rely on an upstream library, which sets TYPE_NUMBER:

https://github.com/Shopify/ghostferry/blob/40ac9a891e881b10b892ad28988648588b215a70/vendor/github.com/siddontang/go-mysql/schema/schema.go#L123-L124

Some additional investigation is required, but I'm not sure if we have the bandwidth right now. We can definitely review and accept PRs, tho.

HemeraOne commented 3 years ago

Thanks for pointing me in the direction, I will see if I can debug this for a bit. I'm new to the go language so it's not really a smooth ride.