hyperlane-xyz / hyperlane-monorepo

The home for Hyperlane core contracts, sdk packages, and other infrastructure
https://hyperlane.xyz
Other
295 stars 312 forks source link

Postgres doesn't support unsigned integers, but scraper DB schema assumes it does #3121

Open tkporter opened 8 months ago

tkporter commented 8 months ago

Sadly https://github.com/SeaQL/sea-query will silently use signed integers for Postgres even when the schema written as code calls for them to be unsigned 🤮

This means that e.g. the domain ID, which supposed to be an unsigned 32 bit integer: https://github.com/hyperlane-xyz/hyperlane-monorepo/blob/8ccfdb7f0119ce28ceaef4fbcfc05bafec97225a/rust/agents/scraper/migration/src/m20230309_000001_create_table_domain.rs#L248 will actually be an integer, which is signed 😭

Similar deal for bigints, etc. Interestingly we originally chose chain_id to be a big int, but the domain ID to be an int..

explorer4=> \d domain
                                                     Table "public.domain"
    Column     |            Type             | Collation | Nullable |                         Default
---------------+-----------------------------+-----------+----------+----------------------------------------------------------
 id            | integer                     |           | not null |
 time_created  | timestamp without time zone |           | not null | '2023-10-06 23:47:05.28212'::timestamp without time zone
 time_updated  | timestamp without time zone |           | not null |
 name          | text                        |           | not null |
 native_token  | text                        |           | not null |
 chain_id      | bigint                      |           |          |
 is_test_net   | boolean                     |           | not null |
 is_deprecated | boolean                     |           | not null |

The impact of this is we can't add domains like ancient8testnet whose domain ID is greater than (2^31) - 1 to the scraper.

We should probably:

  1. spot everywhere we rely on unsigned integer types and find out if this poses any other issues for us
  2. possibly move to larger data types to account for this deficiency

Maybe good to do this at the same time as considering how we'll get alt execution envs to work, as that'll probably require a schema change anyways

tkporter commented 8 months ago

related: https://github.com/SeaQL/sea-query/issues/731

tkporter commented 6 months ago

These as i32s seem unsafe as well https://github.com/hyperlane-xyz/hyperlane-monorepo/blob/dan/block-by-block-indexing/rust/agents/scraper/src/db/message.rs#L208