Discuss: Behavior of preserved table id (0~1023) and system tables

[waynexia]

What type of enhancement is this?

Refactor, Tech debt reduction

What does the enhancement do?

The first 1024 numbers are preserved for special system table (defined in the constant MAX_SYS_TABLE_ID). In practice those ID are allocated statically (in code) to system tables. E.g., 0 for SystemCatalog table and 1 for ScriptsTable.

One key difference between those 1024 special IDs and other normal table IDs is, those 1024 IDs will be duplicated cross different catalogs. Since we put table ID in the data path, it's not allowed to have the same table ID for different table otherwise their data will be mixed up. Except they don't have persisted state.

Luckily, this constrain holds for most system table, like information_schema.tables, information_schema.metrics, public.numbers etc. But other tables like scripts should not have a static ID because they have to write data.

Based on this, my perspectives to preserved table IDs are:

• They are only for system tables that doesn't have persisted data
• They are (special) placeholders
• We should change ScriptsTable and SystemCatalog to allocate their table IDs on created

Implementation challenges

No response

[tisonkun]

@waynexia it seems that:

/// system_catalog table id
pub const SYSTEM_CATALOG_TABLE_ID: u32 = 0;
/// scripts table id
pub const SCRIPTS_TABLE_ID: u32 = 1;

these two consts are no longer used. What's the status of this issue now?