Currently, WAL segments use a column-based format similar to the one met in the tables. For scenarios with many columns/tables, this may lead to random write I/O patterns due to WalWriters having to write into many column files.
To prevent that, we should introduce an alternative WAL format to be used in row-first workloads, like ILP ingestion. When this format is in use, the WalWriter will write the data row-first into a single file leading to a sequential write I/O pattern and overall better performance. The old column-first format should be kept for specific scenarios like INSERT INTO tab_y SELECT * FROM tab_x; where we can copy the data column-wise.
Currently, WAL segments use a column-based format similar to the one met in the tables. For scenarios with many columns/tables, this may lead to random write I/O patterns due to WalWriters having to write into many column files.
To prevent that, we should introduce an alternative WAL format to be used in row-first workloads, like ILP ingestion. When this format is in use, the WalWriter will write the data row-first into a single file leading to a sequential write I/O pattern and overall better performance. The old column-first format should be kept for specific scenarios like
INSERT INTO tab_y SELECT * FROM tab_x;where we can copy the data column-wise.