SQL_ATTR_TYPE_EXCEPTION_BEHAVIOR and SQL_ATTR_LENGTH_EXCEPTION_BEHAVIOR when doing multipart retrieval

[matthew-wozniczka]

The intention of SQL_ATTR_TYPE_EXCEPTION_BEHAVIOR & SQL_ATTR_LENGTH_EXCEPTION_BEHAVIOR are to allow the application to recover and try to retrieve the value again if there was an issue with the binding, but if the column in question is variable-length, bound as a variable-length C type, and the conversion error doesn't occur until one of the SQLGetDatas on that column after SQLFetch returned SQL_MORE_DATA, then my understanding is that the application will not have the opportunity to try to fetch that value as another type.

This scenario could happen, for example, when retrieving a SQL_(W)?((LONG)?VAR)?CHAR column as SQL_C_CHAR/SQL_C_WCHAR, and somewhere in the middle of the value, there was a codepoint which couldn't be represented in the target encoding (On that note, should that be a conversion error, or should it simply insert a replacement character and possibly generate a warning?). Other scenarios would be retrieving custom variable length types (which seems to be allowed by the spec currently, if I'm reading it correctly).

Should we add a mechanism to deal with this scenario? Unfortunately, in general I think it would require the driver to cache the entire value in memory until it was successfully retrieved (Although I can imagine similar problems when doing variable length type -> fixed length type conversions; for example, if you allowed a string containing an integer, followed by an arbitrary amount of whitespace to be converted successfully to SQL_INTEGER, you can construct arbitrarily large inputs which will only fail after processing the last byte of the input.)

[mikepizzo]

I would not expect that SQLGetData would restart at the beginning due to a conversion failure. I think I'd be okay saying that mid-stream conversion errors are not recoverable.

[mikepizzo]

Added note:

Note that SQLGetData starts at the beginning of the value and reads sequentially until all bytes have been consumed. If a conversion failure occurs after the first call to SQLGetData, the rest of the data for that column will not be retrievable.

[mikepizzo]

Clarify that SQLGetData should pick up where binding left off...

[matthew-wozniczka]

On that note, should that be a conversion error, or should it simply insert a replacement character and possibly generate a warning?

Still curious about this, is it just driver-defined?

[mikepizzo]

It looks as though Hex 1A is a commonly used substitution character. We could consider defining that as default behavior, and adding a flag for returning an error, or just leave driver-defined.

Unclear that there would be much advantage in returning a warning; would the app behave differently? If it has logic to handle a substitution character then it would probably just always handle it; if it wanted to know whether or not there was as substitution character it could scan for it.

For that matter, if it wanted to error in the case of a missing character in the target encoding, as long as we define what the substitution character should be the app can always scan for that character and handle it's presence as an exception.

Propose: Define Hex 1A as the substitution character and not define a warning. If we find applications really need the ability to return an error, we can add a flag in the future.

[mikepizzo]

Added:

2.1.2 Use of Substitution Character

Drivers should use a substitution character (HEX 1A), rather than raise an exception, if a character is not available in the target codepage. Clients should be aware of this behavior, for example, when attempting to round-trip values.