unexpected 'Duplicate entry' when insert empty value

[wjhuang2016]

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

Create a function to force full load:

func (do *Domain) FullReload(startTS uint64) error {
    beginTime := time.Now()
    defer func() {
        infoschema_metrics.LoadSchemaDurationTotal.Observe(time.Since(beginTime).Seconds())
    }()
    snapshot := do.store.GetSnapshot(kv.NewVersion(startTS))
    // Using the KV timeout read feature to address the issue of potential DDL lease expiration when
    // the meta region leader is slow.
    snapshot.SetOption(kv.TiKVClientReadTimeout, uint64(3000)) // 3000ms.

    currentSchemaVersion := int64(0)
    if oldInfoSchema := do.infoCache.GetLatest(); oldInfoSchema != nil {
        currentSchemaVersion = oldInfoSchema.SchemaMetaVersion()
    }

    m := meta.NewSnapshotMeta(snapshot)
    neededSchemaVersion, err := m.GetSchemaVersionWithNonEmptyDiff()
    if err != nil {
        return err
    }
    // fetch the commit timestamp of the schema diff
    schemaTs, err := do.getTimestampForSchemaVersionWithNonEmptyDiff(m, neededSchemaVersion, startTS)
    if err != nil {
        logutil.BgLogger().Warn("failed to get schema version", zap.Error(err), zap.Int64("version", neededSchemaVersion))
        schemaTs = 0
    }

    schemas, err := do.fetchAllSchemasWithTables(m)
    if err != nil {
        return err
    }

    policies, err := do.fetchPolicies(m)
    if err != nil {
        return err
    }

    resourceGroups, err := do.fetchResourceGroups(m)
    if err != nil {
        return err
    }
    // clear data
    do.infoCache.Data = infoschema.NewData()
    newISBuilder, err := infoschema.NewBuilder(do, do.sysFacHack, do.infoCache.Data).InitWithDBInfos(schemas, policies, resourceGroups, neededSchemaVersion)
    if err != nil {
        return err
    }
    infoschema_metrics.LoadSchemaDurationLoadAll.Observe(time.Since(beginTime).Seconds())
    logutil.BgLogger().Info("full load InfoSchema success",
        zap.Int64("currentSchemaVersion", currentSchemaVersion),
        zap.Int64("neededSchemaVersion", neededSchemaVersion),
        zap.Duration("start time", time.Since(beginTime)))

    is := newISBuilder.Build(startTS)
    do.infoCache.Insert(is, schemaTs)
    return nil
}

Then add these code to TestRenameTable

    ver, err := store.CurrentVersion(kv.GlobalTxnScope)
    require.NoError(t, err)
    version := ver.Ver
    err = dom.FullReload(version)
    require.NoError(t, err)

before

tk.MustExec("insert rename2.t values ()")

Then enable info schema cache by set DefTiDBSchemaCacheSize to 100

2. What did you expect to see? (Required)

Test pass

3. What did you see instead (Required)

Duplicate entry '1' for key 't.PRIMARY' reported by insert rename2.t values ()

4. What is your TiDB version? (Required)

master

[tiancaiamao]

In the full load, the value get from tikv is 0, while it should be 5000 Something must be wrong, maybe the value is not sync to tikv, or changed/removed later, I'll still not sure

[tiancaiamao]

Related to https://github.com/pingcap/tidb/issues/46904

[tiancaiamao]

This is caused by https://github.com/pingcap/tidb/pull/47892, after this commit, we keep using the old allocator after the rename operation.

    tk.MustExec("create database rename1")    // DB ID = 104
    tk.MustExec("create database rename2")    // DB ID = 106
    tk.MustExec("create database rename3")
    tk.MustExec("create table rename1.t (a int primary key auto_increment)")   // table ID = 110

        // Before this operation, rename1.t is  dbID=104, tableID = 110
    tk.MustExec("rename table rename1.t to rename2.t")   
        // After this operation, table info become dbID=106, tableID = 110

        // **But the allocator is still dbID=104, tableID=110**

    tk.MustExec("drop database rename1")     // clear all data for dbID=104

        tk.MustExec("insert rename2.t values ()")    // id allocator is dbID=104, tableID=110, and the meta data for dbID=104 is clear

[tiancaiamao]

This is a 'rename table & autoid' compatibility issue.