dotnet / runtime

.NET is a cross-platform runtime for cloud, mobile, desktop, and IoT apps.
https://docs.microsoft.com/dotnet/core/
MIT License
15.08k stars 4.69k forks source link

ODBC Arithmetic overflow error Accessing AS400 DB2 (Update, Delete, Insert) #31533

Open BockTim91 opened 4 years ago

BockTim91 commented 4 years ago

We are trying to hit our AS400 running a DB2 instance. When we run a OdbcCommand.ExecuteNonQuery() for select, everything works no issue. However, when executing a simple update, or delete command, we encounter the below error. Our current workaround is to just squish all arithmetic errors out of our execute command, which is unacceptable.

Arithmetic operation resulted in an overflow.
   at System.Data.Odbc.OdbcDataReader.FirstResult()
   at System.Data.Odbc.OdbcCommand.ExecuteReaderObject(CommandBehavior behavior, String method, Boolean needReader, Object[] methodArguments, SQL_API odbcApiMethod)
   at System.Data.Odbc.OdbcCommand.ExecuteReaderObject(CommandBehavior behavior, String method, Boolean needReader)
   at System.Data.Odbc.OdbcCommand.ExecuteNonQuery()

We are using VS 2017 with netcore 2.2

danmoseley commented 4 years ago

Is it possible to attach a debugger to figure out exactly where (what line/values) this is coming from in the code?

VS should be able to resolve symbols and sources when you debug.

BockTim91 commented 4 years ago

In enabling full debugging, it appears the issue occurs at line 1912 and 1913 of OdbcDataReader.

            cRowsAffected = GetRowCount();              // get rowcount of the current resultset (if any)
            CalculateRecordsAffected(cRowsAffected);    // update recordsaffected

When running a select statement cRowsAffected gets -1 as expected, however when running a delete it appears to be overloading the int causing it to crash into SR GetResourceString.

        internal static string GetResourceString(string resourceKey, string defaultString)
        {
            string resourceString = null;
            try { resourceString = InternalGetResourceString(resourceKey); }
            catch (MissingManifestResourceException) { }

            if (defaultString != null && resourceKey.Equals(resourceString, StringComparison.Ordinal))
            {
                return defaultString;
            }

            return resourceString;
        }

Upon trying to add cRowsAffected into my watch and casting as an int or long it gives a, "Cannot obtain value of the local variable or argument because it is not available." when doing a delete. However when running a Select it returns a -1 as expected.

Let me know if there is anything else you need.

Clockwork-Muse commented 4 years ago

CalculateRecordsAffected is takes an int:

internal int CalculateRecordsAffected(int cRowsAffected)
{
    if (0 <= cRowsAffected)
    {
        if (-1 == _recordAffected)
        {
            _recordAffected = cRowsAffected;
        }
        else
        {
            _recordAffected += cRowsAffected;
        }
    }
    return _recordAffected;
}

this is causing cRowsAffected (which is a wrapper around an IntPtr) to call this operator:

public static unsafe implicit operator int(SQLLEN value)
{
#if WIN32
    return (int)value._value.ToInt32();
#else
    long l = (long)value._value.ToInt64();
    return checked((int)l);
#endif
}

I'm assuming it's the return checked((int)l); line that is actually throwing, which implies that your statement affected 2 billion+ rows.

Which is certainly in a believable range (I'm not sure if they're all stored in the same or partitioned tables, but we definitely have cause for that many rows on an iSeries database here). Unfortunately DbDataReader.RowsAffected is a strict 32-bit int...


GetResourceString is almost certainly just from reporting the overflow, an not at issue.

BockTim91 commented 4 years ago

I can safely say the table I was updating has less than 12 million records in it. The statement I am running is on the key affecting only one record.

tb-mtg commented 4 years ago

I too am having this problem.

I created a simple test project and with basic insert, update & delete OdbcCommands. They only work when the following is set:

<PlatformTarget>x86</PlatformTarget>.

The problem is our real application platform target is x64.

danmoseley commented 4 years ago

@saurabh500

saurabh500 commented 4 years ago

I think this one suffers from the lack of an WIN32 build for System.Data.Odbc, which has surfaced in System.Data.OleDb also. I will look into it.

tb-mtg commented 4 years ago

Turns out I had an out of date IBM Client Access ODBC driver 13.0.1.0.

I downloaded and installed the latest AS400 Client Access service pack si68573 from IBM and I think it may have fixed the problem.

The IBM i Access ODBC drivers versions are now:

ajcvickers commented 4 years ago

@saurabh500 Do you believe that there is still work to be done here?

misha-p commented 3 years ago

Experiencing the same issue with Vertica DB. When ODBC client is in 64 bit (.NET 5) and rows affected > Int32.MaxValue, overflow exception is thrown. I believe @Clockwork-Muse is exactly right in his analysis of the root cause https://github.com/dotnet/runtime/issues/31533#issuecomment-557211264

Unfortunately recent optimization done in .NET 6 preview branch - https://github.com/dotnet/runtime/commit/3d200308e1c9eb8daabd0498d7a2ee02635452b1 - will not solve the issue: checked((int)l) is removed, but ToInt32() called on IntPtr will throw OverflowException still.

We are building x86 target for now, just to overcome that issue. I guess getting some negative int for rows affected instead of the real count (> int.MaxValue) would be better then an overflow exception.

PoojaSingh15 commented 3 years ago

@BockTim91, could you please share how are you connecting to as400? I am using .net5 and IBM.data.DB2.core package to connect to my iSeries database. had no luck so far.

BockTim91 commented 3 years ago

@PoojaSingh15 We are using System.Data.ODBC (.net core 3.1 currently) and using basic ODBC connections straight to our AS400 connecting to it through windows ODBC Data Sources. The ODBC data source is using iSeries Access ODBC Driver to connect.

franciskhoury commented 3 years ago

Hello, I am encountering this problem in .NET5. The ArithmeticException is being raised on INSERT and DELETE statements (even if affecting 0 rows). However, the MERGE statement is not raising an exception, and is returning the correct row count.

danmoseley commented 3 years ago

@saurabh500 per @ajcvickers is there more work here?

franciskhoury commented 3 years ago

EDIT: I take back what I had said here about not being able to reproduce the issue. It does happen consistently after all. Sorry, I forgot that the latest code I was testing is handling the AritmeticException, so in my tired state, I fooled myself into thinking it wasn't happening any more.

danmoseley commented 3 years ago

@franciskhoury if you repro under a debugger (VS should download sources and symbols) can you confirm the diagnosis above? And that the driver is returning an erroneous number of rows?

Although (I’m not familiar with this area) if our code simply does not support a number of rows that it ought to work with, perhaps that needs a fix as well.