Extremely slow Batch write performance on Windows 11

[axww]

Recently I'm doing some test on BBolt, aim to fill 100,000 IDs into Bucket. Here's my test code "main.go", using latest BBolt version, very simple:

package main

import (
    bolt "go.etcd.io/bbolt"
)

type Number interface {
    int | int8 | int16 | int32 | int64 | uint | uint8 | uint16 | uint32 | uint64
}

func Number_bytes[T Number](num T, size uint8) []byte {
    bytes := make([]byte, size)
    for i := uint8(0); i < size; i++ {
        bytes[size-i-1] = byte(num >> (8 * i))
    }
    return bytes
}

func main() {

    db, err := bolt.Open("test.db", 0600, nil)
    if err != nil {
        panic(err)
    }
    defer db.Close()

    db.Update(func(tx *bolt.Tx) error {
        tx.CreateBucketIfNotExists([]byte("test"))
        return nil
    })

    for a := range 100000 {
        db.Batch(func(tx *bolt.Tx) error {
            b := tx.Bucket([]byte("test"))
            b.Put(Number_bytes(a, 8), []byte("test"))
            return nil
        })
    }

}

When I run this on Windows 11, it totally stucked. Then I reduced range 100000 to range 10000, it works. However it takes abount 1 minute, longer than other databases. Also these data occupied 1MB, obviously larger than other databases.

I wonder is there anything wrong with my test method, any way to solve if I want to handle such type of massive puts? Thank you very much!

[fyfyrchik]

Hi, 0w0c! Batch can be called concurrently from different goroutines. In your scenario each batch contains exactly one transaction and every iteration takes at least 20ms (default batch waiting time). To make better use of batches you might want to spawn a goroutine inside your loop.

[axww]

Hi, 0w0c! Batch can be called concurrently from different goroutines. In your scenario each batch contains exactly one transaction and every iteration takes at least 20ms (default batch waiting time). To make better use of batches you might want to spawn a goroutine inside your loop.

Thank you so much! I've tried merge all loop into one Batch and the problem solved.