database/sql: expose database original column type

[kirubasankars]

Please expose original database column type at least as string. this is currently so restricting on this package. this will help us to do some level dynamic column value parsing

[kardianos]

First and foremost, why separate the interfaces at all? In practice, driver authors are not going to implement just a subset of this new API; what would the reasoning for that be?

The reason is for backwards compatibility and extensibility. If we need to add another property, it can be done. Also the driver package cannot reference the sql package, so we would need to copy the values from each driver.ColumnType to sql.ColumnType.

As a consumer of the API, a partial implementation by a driver would be somewhat silly. The sort of use cases I'm thinking of that require this type of column metadata generally aren't satisfied by a partial implementation.

Some drivers may not have all the information available. But I generally agree with this point.

This increases the burden on the driver author who now has five methods they need to write test cases for, as opposed to one. Also it potentially complicates their implementation (will they need to keep data structures hanging around expecting subsequent calls for the next col etc?)

I don't really buy this one. At the end of the day you are testing the same amount of properties.

This breaks the design symmetry between the sql and driver packages (as exemplified by the Rows.Columns() methods).

I don't understand this.

A single call to sql.Rows.ColumnTypes() results in 5 x [num cols] calls to the driver impl. That's potentially a great deal of calls (at Teradata I saw customer queries with hundreds of cols). Even if in practice this is not a huge perf burden, something about it seems wrong (an "API smell" if you will).

This is true. I'm not expecting this to be in a super hot loop. I also want to be able to work with tables with hundreds of columns ( http://www.jdetables.com/ sigh). In this case backwards compatibility and extensibility are the key design issues here.

---

Let's take a step back and impose the following restrictions on design:

• ColumnType needs to be defined in the database/sql package.
• If we want to add another property in the future, we need to be able to do so without breaking backwards compatibility.
• We need to not break backwards compatibility on this change.

Thank you for the above links and the concrete counter proposal. Could you modify the proposal to meet the above, or describe how it meets the above? I agree that it meets the last point.

[neilotoole]

The reason is for backwards compatibility and extensibility. If we need to add another property, it can be done.

Yeah, I take your point on the extensibility. I guess my assumption was that this proposal is the fix for the sql package, and that there won't be any changes coming ever again (in our lifetimes at least).

Also the driver package cannot reference the sql package, so we would need to copy the values from each driver.ColumnType to sql.ColumnType.

The example I gave above is from my own hack of the API for a private project where I didn't care about exposing the driver package (I'll add a comment to clarify this, probably should have omitted that example). But yes, I was imagining the two ColumnTypes, but rather than copying the values, the sql.ColumnType would wrap the driver.ColumnType, similar to what is done with Rows:

package sql

type Rows struct {
    dc          *driverConn // owned; must call releaseConn when closed to release
    releaseConn func(error)
    rowsi       driver.Rows

[neilotoole]

I guess ^^^ would look something like this:

package sql

type ColumnType struct {
    cti driver.ColumnType
}

func (c *ColumnType) Name() string {
    return c.cti.Name()
}

func (c *ColumnType) Length() (length int64, ok bool) {
    return c.cti
}
// etc...

func (rs *Rows) ColumnTypes() ([]ColumnType, error) {
    if rs.closed {
        return nil, errors.New("sql: Rows are closed")
    }
    if rs.rowsi == nil {
        return nil, errors.New("sql: no Rows available")
    }

    if colTyper, ok := rs.rowsi.(driver.ColumnTyper); ok {

        drvCols, err := colTyper.ColumnTypes()

        if err != nil {
            sqlCols := make([]ColumnType, len(drvCols))
            for i := range drvCols {
                sqlCols[i] = ColumnType{drvCols[i]}
            }
            return sqlCols, nil
        }

        return nil, err
    }

    return nil, errors.New("sql: driver does not provide column type data")
}

Which does result in a bunch of allocations, hmmmn.

[neilotoole]

@kardianos I'm coming around to your approach due to your very valid point about future extensibility (which wasn't a design consideration for me in my own hack).

Also, it's probably the case that my point about the amount of calls to the driver to construct the sql.ColumnType object is a bit of a red herring. In practice for most applications I imagine that sql.Rows.ColumnTypes() will likely only be invoked once per unique query, and the details of how that function is implemented will only have negligible overall impact on performance. I'll give it some more thought this weekend, but given that the key thing here is the API design for the sql package (as opposed to the driver package changes), I don't want to hold up the overall proposal. Thanks for taking the time to get back to me.