Table-level WAL switch

[v0y4g3r]

What problem does the new feature solve?

Not all tables require durability at the cost of insert throughput. With periodical table flush, losing a short period of data in memtable is acceptable in some scenarios.

What does the feature do?

Allow user to specify whether to skip writing WAL on data ingestion.

Implementation challenges

Add a table option on creation, maybe skip_wal. In write_to_wal, check if this table should skip WAL. https://github.com/GreptimeTeam/greptimedb/blob/b31a6cb506c5fd4cee417ddb3c0780d25272b505/src/storage/src/wal.rs#L103-L113

If skip_wal is true, simply return the sequence number as entry id without actually writing to underlying log store.

[WangTingZheng]

I want to try this issues.

[killme2008]

@WangTingZheng hello, are u still working on this feature?

[WangTingZheng]

@WangTingZheng hello, are u still working on this feature?

I'm sorry, I may not have enough time to complete this task recently.

[killme2008]

@WangTingZheng That's fine. So may I assign this task to other developers? Thank u.

[evenyag]

We are refactoring the mito table engine and the implementation hint in the description will soon become outdated. So I self-assign this task first.

[niebayes]

See the implementation of the Kafka Remote Wal #2722 .