Shannon-Data / ShannonBase

A MySQL HTAP Database, Open Source version of MySQL Heatwave, Powered by AI.
https://www.shannonbase.org
Other
16 stars 6 forks source link

Feature: add new mtr_log_t for pop thread #175

Closed ShannonBase closed 5 months ago

ShannonBase commented 5 months ago

Summary Now that, we parse all the mlog records, but we should just care about three type operations: (1) insert; (2) delete; (3) update. Besides, the other operations is not our concern. Taking create database or drop database as examples, all the mlog records generate in these operations should not be added into sys_pop_buff.

A new type MTR_LOG_WITH_POP is added, if the mlog is not this type, we do not add these mlogs into pop buff, otherwise, these mlogs are added.

ShannonBase commented 5 months ago
/** Logging modes for a mini-transaction */
enum mtr_log_t {
  /** Default mode: log all operations modifying disk-based data */
  MTR_LOG_ALL = 0,

  /** Log no operations and dirty pages are not added to the flush list */
  MTR_LOG_NONE = 1,

  /** Don't generate REDO log but add dirty pages to flush list */
  MTR_LOG_NO_REDO = 2,

  /** Inserts are logged in a shorter form */
  MTR_LOG_SHORT_INSERTS = 3,

  /** Last element */
  MTR_LOG_MODE_MAX = 4
};

a new type of log mode is added to indicate that should we route the mlog records to pop buffer too or not. MTR_LOG_POP means that it's a default mode with adding mlog to sys_pop_buff when mtr commits.

ShannonBase commented 5 months ago

or another approach is adding a member field in mtr to indicates whether the mlog should be added or not.

/** Mini-transaction handle and buffer */
struct mtr_t {
  /** State variables of the mtr */
  struct Impl {
    /** memo stack for locks etc. */
    mtr_buf_t m_memo;

    /** mini-transaction log */
    mtr_buf_t m_log;

    /** true if inside ibuf changes */
    bool m_inside_ibuf;

    /** true if the mini-transaction might have modified buffer pool pages */
    bool m_modifications;

  /**true to add to sys_pop_buff*/
    bool m_to_pop;

at commit to check the flag.

void mtr_t::Command::execute() {
  ut_ad(m_impl->m_log_mode != MTR_LOG_NONE);

#ifndef UNIV_HOTBACKUP
  ulint len = prepare_write();

  if (len > 0) {
    mtr_write_log_t write_log;

    write_log.m_left_to_write = len;

    auto handle = log_buffer_reserve(*log_sys, len);

    write_log.m_handle = handle;
    write_log.m_lsn = handle.start_lsn;

    m_impl->m_log.for_each_block(write_log);

    ut_ad(write_log.m_left_to_write == 0);
    ut_ad(write_log.m_lsn == handle.end_lsn);
RingsC commented 5 months ago

approach 3: use SQLCOM_INSERT, SQLCOM_DELETE, SQLCOM_UPDATE to determine that should we route the redo log to pop or not..

2.2.7 Change Propagation After tables are loaded into HeatWave, data changes are automatically propagated from InnoDB tables on the MySQL DB System to their counterpart tables in the HeatWave Cluster. DML operations (INSERT, UPDATE, DELETE) on the MySQL DB System do not wait for changes to be propagated to the HeatWave Cluster; that is, DML operations on the MySQL DB System are not delayed by HeatWave change propagation. [https://dev.mysql.com/doc/heatwave/en/mys-hw-change-propagation.html]