Issues with NULL value in type `map` and `struct`

[gilsegment]

When there are NULL values in map/struct type. The Scan will translate it to Golang zero value rather than returning NULL. This make it impossible for the client to know if Databricks saves zero value or null for that field. In case ALL the values of the map/struct are NULL the scan will simply fail.

See below code:

func selectFromMap(db *sql.DB) {
    var row interface{}

    /********************************* Basic map ************************************/
    rows, err := db.Query("select map('red', 1, 'green', 2) as sample_map")
    if err != nil {
        panic(err)
    }
    defer rows.Close()

    if rows.Next() {
        if err = rows.Scan(&row); err != nil {
            panic(err)
        }
        fmt.Printf("basic map: %v\n", row) // Good! {"red":1,"green":2}
    }

    /********************************* Map with null value ****************************/
    rows, err = db.Query("select map('red', 1, 'green', null) as sample_map")
    if err != nil {
        panic(err)
    }
    if rows.Next() {
        if err = rows.Scan(&row); err != nil {
            panic(err)
        }
        fmt.Printf("map with null value: %v\n", row) // Bad! {"red":1,"green":0} -> should print NULL not 0 
    }

    /***************************** Map with ONLY null values *************************/
    rows, err = db.Query("select map('red', NULL, 'green', NULL) as sample_map")
    if err != nil {
        panic(err)
    }
    if rows.Next() {
        if err = rows.Scan(&row); err != nil {
            panic(err)
        }
        fmt.Printf("map with ONLY Null values: %v\n", row) // ERR databricks: arrow row scanner unhandled type null
    }
}

Expected output:

basic map: {"red":1,"green":2}
map with null value: {"red":1,"green":null}
map with ONLY Null values: {"red":NULL,"green":NULL}

Actual output:

basic map: {"red":1,"green":2}
map with null value: {"red":1,"green":0}
11:50AM ERR databricks: arrow row scanner unhandled type null connId=*** corrId= queryId=****

[rcypher-databricks]

Thank you for reporting this. We will look into it and update you.

[gilsegment]

@rcypher-databricks Thanks for quickly handling that, can you please start a new release if that change was tested and ready to be used.

[rcypher-databricks]

release 1.3.0 should be available now

[rcypher-databricks]

Were you able to check if 1.3.0 resolves your issue?

[gilsegment]

@rcypher-databricks Yes looks better now. Thanks!