fix bigint unsigned null column scan to err type int64

elonnzhang commented 1 month ago

Description

when i use gorm scan sql rows to map[string]any, i got an error (out of range). gorm scan sql.Rows to map[string]any by ScanRows there has three step:

prepare values according to column type. https://github.com/go-gorm/gorm/blob/master/scan.go#L25,
```
columnTypes, _ := rows.ColumnTypes()
prepareValues(values, db, columnTypes, columns)
```
rows.Scan(...any)
```
rows.Scan(values...)
```
assign column name and value to map
```
scanIntoMap(mapValue, values, columns)
```

in step 2, when occur number(tiny int short long longlong )null column, this type will be all set to scanTypeNullInt,

scanTypeNullInt    = reflect.TypeOf(sql.NullInt64{})

# "database/sql"#L210
// NullInt64 represents an int64 that may be null.
// NullInt64 implements the [Scanner] interface so
// it can be used as a scan destination, similar to [NullString].
type NullInt64 struct {
    Int64 int64
    Valid bool // Valid is true if Int64 is not NULL
}

tiny int short long type will be ok, its Maximum value will not out of range int64 but unsigned longlong may be out of range. int64:-9223372036854775808 ~9223372036854775807, uint64:0 ~ 18446744073709551615

a uint64 assign to NullInt64 will be out of range, because the value store into int64

so to fix it

Must add a new scan type NullUint64 to represent unsigned bigint null column. and though flag to judge unsigned or not.

Dur to database/sql support generics Null[T] type, it will be easy to support NullUint64.

scanTypeNullUInt   = reflect.TypeOf(sql.Null[uint64]{})

Reproduce the problem

table example

CREATE TABLE `tbl_name` (
  `id` int NOT NULL COMMENT '',
  `ubigint` bigint unsigned DEFAULT NULL COMMENT '9223372036854775808',
  PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
INSERT INTO `tbl_name` (`id`, `ubigint`) VALUES (1, 9223372036854775808);

code

func main() {
    // connect to db
    db, err := sql.Open("mysql", "dsn")
    if err != nil {
        log.Fatal(err)
    }
    defer func() { _ = db.Close() }()

    gormDB, err := gorm.Open(gormMysql.New(gormMysql.Config{
        Conn: db,
    }), &gorm.Config{})
    var result map[string]any
    if err := gormDB.Raw("select * from tbl_name where id = 1").Scan(&result).Error; err != nil {
        log.Fatal(err)
    }
    // print type and value
    fmt.Printf("Type: %v\n", reflect.TypeOf(result))
    fmt.Printf("Value: %v\n", result)
}

Env

go version
go version go1.22.3 darwin/arm64

Error log

sql: Scan error on column index 1, name "ubigint": converting driver.Value type uint64 ("9223372036854775808") to a int64: value out of range
[9.744ms] [rows:1] select * from tbl_name where id = 1
2024/07/26 19:36:29 sql: Scan error on column index 1, name "ubigint": converting driver.Value type uint64 ("9223372036854775808") to a int64: value out of range

Checklist

[x] Code compiles correctly
[ ] Created tests which fail without the change (if possible)
[ ] All tests passing
[ ] Extended the README / documentation, if necessary
[x] Added myself / the copyright holder to the AUTHORS file

Summary by CodeRabbit

Documentation
- Updated the AUTHORS file to include Jiabin Zhang as a contributor.
Enhancements
- Improved handling of unsigned integer types in the MySQL field representation, enhancing type safety for data scanning processes.

coderabbitai[bot] commented 1 month ago

Walkthrough

This update enhances the codebase by adding Jiabin Zhang as a contributor in the AUTHORS file, recognizing their contributions to the project. Additionally, a new reflection type, scanTypeNullUInt, is introduced in fields.go to improve the handling of unsigned integer types in MySQL fields. The scanType method in the mysqlField struct is modified to ensure better type safety during the processing of these unsigned integers.

Changes

Files	Change Summary
`AUTHORS`	Added Jiabin Zhang's name and email address to the authors list.
`fields.go`	Introduced `scanTypeNullUInt` for `sql.Null[uint64]{}`; modified `scanType` method to handle unsigned integers.

Poem

🐇 In fields of code where rabbits play,
New authors join, hip-hip-hooray!
With types anew, we hop with glee,
Reflecting all our unity!
A project blooms, together we thrive,
In every line, our spirit's alive! 🌼✨

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share

- [X](https://twitter.com/intent/tweet?text=I%20just%20used%20%40coderabbitai%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20the%20proprietary%20code.%20Check%20it%20out%3A&url=https%3A//coderabbit.ai) - [Mastodon](https://mastodon.social/share?text=I%20just%20used%20%40coderabbitai%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20the%20proprietary%20code.%20Check%20it%20out%3A%20https%3A%2F%2Fcoderabbit.ai) - [Reddit](https://www.reddit.com/submit?title=Great%20tool%20for%20code%20review%20-%20CodeRabbit&text=I%20just%20used%20CodeRabbit%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20proprietary%20code.%20Check%20it%20out%3A%20https%3A//coderabbit.ai) - [LinkedIn](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fcoderabbit.ai&mini=true&title=Great%20tool%20for%20code%20review%20-%20CodeRabbit&summary=I%20just%20used%20CodeRabbit%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20proprietary%20code)

Tips

### Chat There are 3 ways to chat with [CodeRabbit](https://coderabbit.ai): - Review comments: Directly reply to a review comment made by CodeRabbit. Example: - `I pushed a fix in commit .` - `Generate unit testing code for this file.` - `Open a follow-up GitHub issue for this discussion.` - Files and specific lines of code (under the "Files changed" tab): Tag `@coderabbitai` in a new review comment at the desired location with your query. Examples: - `@coderabbitai generate unit testing code for this file.` - `@coderabbitai modularize this function.` - PR comments: Tag `@coderabbitai` in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples: - `@coderabbitai generate interesting stats about this repository and render them as a table.` - `@coderabbitai show all the console.log statements in this repository.` - `@coderabbitai read src/utils.ts and generate unit testing code.` - `@coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.` - `@coderabbitai help me debug CodeRabbit configuration file.` Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. ### CodeRabbit Commands (invoked as PR comments) - `@coderabbitai pause` to pause the reviews on a PR. - `@coderabbitai resume` to resume the paused reviews. - `@coderabbitai review` to trigger an incremental review. This is useful when automatic reviews are disabled for the repository. - `@coderabbitai full review` to do a full review from scratch and review all the files again. - `@coderabbitai summary` to regenerate the summary of the PR. - `@coderabbitai resolve` resolve all the CodeRabbit review comments. - `@coderabbitai configuration` to show the current CodeRabbit configuration for the repository. - `@coderabbitai help` to get help. Additionally, you can add `@coderabbitai ignore` anywhere in the PR description to prevent this PR from being reviewed. ### CodeRabbit Configuration File (`.coderabbit.yaml`) - You can programmatically configure CodeRabbit by adding a `.coderabbit.yaml` file to the root of your repository. - Please see the [configuration documentation](https://docs.coderabbit.ai/guides/configure-coderabbit) for more information. - If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: `# yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json` ### Documentation and Community - Visit our [Documentation](https://coderabbit.ai/docs) for detailed information on how to use CodeRabbit. - Join our [Discord Community](https://discord.com/invite/GsXnASn26c) to get help, request features, and share feedback. - Follow us on [X/Twitter](https://twitter.com/coderabbitai) for updates and announcements.

coveralls commented 1 month ago

coverage: 82.364% (-0.04%) from 82.403% when pulling 11f9287323f7249519405560407d4f5cd2b10bc0 on elonnzhang:fix_bigint_unsigned_scan_type into 44553d64bcde78a5b58cb133a5cc708281c333e0 on go-sql-driver:master.

methane commented 1 month ago

We don't drop Go 1.21 support yet. So we can not use sql.Null without dirty build tag hacks.

elonnzhang commented 1 month ago

We don't drop Go 1.21 support yet. So we can not use sql.Null without dirty build tag hacks.

ok，i just notice sql.Null introduced in 1.22

elonnzhang commented 1 month ago

We don't drop Go 1.21 support yet. So we can not use sql.Null without dirty build tag hacks.

Can we use sql.NullString to temporarily support all supported version avoid this problem (out of range) until we support minimum version is go 1.22?

methane commented 1 month ago

I prefer 1 breaking change over 2 breaking changes.

go-sql-driver / mysql