crash when returned without limit in large data sets

[sprappcom]

the below crash if without LIMIT

SELECT name FROM users WHERE name LIKE '%%s%%'  

but doesnt crash if put with LIMIT

SELECT name FROM users WHERE name LIKE '%s%' LIMIT 1000

can u check what's wrong with the crossdb c program? one with limit and one without, using the example below. u can convert the below to c program using claude sonnet or equivalent.

i think it's memory overused or something. not sure if it's the problem

package main

import (
    "fmt"
    "log"
    "math/rand"
    "time"

    "crossdb/go" // <- using the golang package previously sent
)

const (
    numUsers        = 1000000
    numBenchmarkRuns = 1000
)

var firstNames = []string{"John", "Jane", "Michael", "Emily", "David", "Sarah", "Christopher", "Jessica", "Matthew", "Jennifer"}
var lastNames = []string{"Smith", "Johnson", "Williams", "Brown", "Jones", "Garcia", "Miller", "Davis", "Rodriguez", "Martinez"}

func generateRandomName() string {
    return firstNames[rand.Intn(len(firstNames))] + " " + lastNames[rand.Intn(len(lastNames))]
}

func main() {
    rand.Seed(time.Now().UnixNano())

    // Open a connection to CrossDB
    conn, err := crossdb.Open("./benchmark_db")
    if err != nil {
        log.Fatal(err)
    }
    defer conn.Close()

    // Create the users table
    _, err = conn.Exec("DROP TABLE IF EXISTS users")
    if err != nil {
        log.Fatal(err)
    }
    _, err = conn.Exec("CREATE TABLE users (id INT PRIMARY KEY, name CHAR(50), age INT)")
    if err != nil {
        log.Fatal(err)
    }

    // Insert 1,000,000 users
    fmt.Println("Inserting 1,000,000 users...")
    startTime := time.Now()
    for i := 1; i <= numUsers; i++ {
        name := generateRandomName()
        age := rand.Intn(50) + 20 // Random age between 20 and 69
        _, err := conn.Exec(fmt.Sprintf("INSERT INTO users (id, name, age) VALUES (%d, '%s', %d)", i, name, age))
        if err != nil {
            log.Printf("Error inserting user %d: %v", i, err)
        }
        if i%100000 == 0 {
            fmt.Printf("%d users inserted...\n", i)
        }
    }
    fmt.Printf("Insertion completed in %v\n", time.Since(startTime))

    // Benchmark random SELECT with LIKE queries
    fmt.Println("\nRunning benchmark...")
    totalDuration := time.Duration(0)
    for i := 0; i < numBenchmarkRuns; i++ {
        searchName := firstNames[rand.Intn(len(firstNames))]
        query := fmt.Sprintf("SELECT * FROM users WHERE name LIKE '%%%s%%'", searchName)

        startTime := time.Now()
        res, err := conn.Exec(query)
        if err != nil {
            log.Printf("Error executing query: %v", err)
            continue
        }
        duration := time.Since(startTime)
        totalDuration += duration

        // Count the results
        count := 0
        for res.FetchRow() != nil {
            count++
        }

        fmt.Printf("Run %d: Query '%s' returned %d results in %v\n", i+1, query, count, duration)
    }

    averageDuration := totalDuration / numBenchmarkRuns
    fmt.Printf("\nBenchmark completed.\n")
    fmt.Printf("Average query time: %v\n", averageDuration)
}

[jcwangxp]

I've added the go connector and invite you as the maintainer for this repo.

https://github.com/crossdb-org/crossdb-connector-go

[sprappcom]

can you pls add username : kolinfluence instead? i just recovered my github account, previously has issue with 2fa.

p.s. : i will contribute rust binding soon

On Tue, Oct 8, 2024 at 10:48 PM JC Wang @.***> wrote:

I've added the go connector and invite you as the maintainer for this repo.

https://github.com/crossdb-org/crossdb-connector-go

— Reply to this email directly, view it on GitHub https://github.com/crossdb-org/crossdb/issues/24#issuecomment-2401163201, or unsubscribe https://github.com/notifications/unsubscribe-auth/BFM3SPNZJN5FUF73JUCHTWDZ2SKRBAVCNFSM6AAAAABPOEISFWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMBRGE3DGMRQGE . You are receiving this because you authored the thread.Message ID: @.***>

[sprappcom]

i can code in C / C++ as well. if you need my assistance, do mention. i will mention if it's within my reach

i didnt look at the code at all coz focusing on rustlang and golang binding

On Wed, Oct 9, 2024 at 5:13 AM Kolin Fluence @.***> wrote:

can you pls add username : kolinfluence instead? i just recovered my github account, previously has issue with 2fa.

p.s. : i will contribute rust binding soon

On Tue, Oct 8, 2024 at 10:48 PM JC Wang @.***> wrote:

I've added the go connector and invite you as the maintainer for this repo.

https://github.com/crossdb-org/crossdb-connector-go

— Reply to this email directly, view it on GitHub https://github.com/crossdb-org/crossdb/issues/24#issuecomment-2401163201, or unsubscribe https://github.com/notifications/unsubscribe-auth/BFM3SPNZJN5FUF73JUCHTWDZ2SKRBAVCNFSM6AAAAABPOEISFWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMBRGE3DGMRQGE . You are receiving this because you authored the thread.Message ID: @.***>

[jcwangxp]

I've invited you. I just delivered the cpp driver. If you want to contribute, I can invite you too. https://github.com/crossdb-org/crossdb-cpp For go/rust driver, I hope it's a standard driver, then in future, application can test or migrate with this new DB easily.

[sprappcom]

no it's fine. i'm not into cpp driver. i'm more comfortable with c / go and rust. if c++ core (if ever need) maybe i can help look at it.

other than that, i'm ok with c core, go and rust

On Fri, Oct 11, 2024 at 5:54 AM JC Wang @.***> wrote:

I've invited you. I just delivered the cpp driver. If you want to contribute, I can invite you too. https://github.com/crossdb-org/crossdb-cpp For go/rust driver, I hope it's a standard driver, then in future, application can test or migrate with this new DB easily.

— Reply to this email directly, view it on GitHub https://github.com/crossdb-org/crossdb/issues/24#issuecomment-2407058219, or unsubscribe https://github.com/notifications/unsubscribe-auth/BFM3SPKXVIJUKAQW4MNMRTTZ26N6VAVCNFSM6AAAAABPOEISFWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMBXGA2TQMRRHE . You are receiving this because you authored the thread.Message ID: @.***>