Base Repository for the Script Language Container for user defined functions (UDF's) that can be used in the EXASOL database. You can find the release repository under https://github.com/exasol/script-languages-release
also needs to be deallocated (call to Py_XDECRED()). In current implementation, we decreased reference counter only for the transposed array. Debugging showed the reference counter:
Ref count of colArray = 1
Ref count of pyArray = 2
This mean the array retrieved from PyArray_Transpose() is a new object
=> We need to decrease reference counter for both.
2. Items returned from PyList_GetItem() must not be released
...
Return value: Borrowed reference. Part of the [Stable ABI](https://docs.python.org/3/c-api/stable.html#stable)
...
Currently we assign the object returned from PyList_GetItem() to a std::unique_ptr which calls Py_XDECREF() in the destructor.
This can lead do undefined behavior as we might decrease the reference counter to many times.
3. emit with datetime only object fails
Running emit on a dataframe which contains only datetime64[ns] columns fails with error message:
pyodbc.DataError: ('22002', '[22002] [EXASOL][EXASolution driver]VM error: F-UDF-CL-LIB-1127: F-UDF-CL-SL-PYTHON-1002: F-UDF-CL-SL-PYTHON-1026: ExaUDFError: F-UDF-CL-SL-PYTHON-1114: Exception during run \nTEST_DTYPE_EMIT:7 run\nRuntimeError: F-UDF-CL-SL-PYTHON-1136: F-UDF-CL-SL-PYTHON-1130: PyObject is unexpectedly a null pointer\n (Session: 1800240827916484608) (-3452546) (SQLExecDirectW)')
Reason is that the default conversion to numpy expects only objects as cell items. For the case where only one column of type NPY_DATETIME is in the source dataframe, a workaround was already implemented (see here).
Solution: Convert all items in the dataframe to type object if all columns are of type NPY_DATETIME.
fixes #902
3 problems were identified, first 2 are memory related:
1. Numpy object leaked
Py-Object returned from
also needs to be deallocated (call to
Py_XDECRED()
). In current implementation, we decreased reference counter only for the transposed array. Debugging showed the reference counter:This mean the array retrieved from
PyArray_Transpose()
is a new object=> We need to decrease reference counter for both.
2. Items returned from
PyList_GetItem()
must not be releasedSee documentation
PyList_GetItem()
to a std::unique_ptr which callsPy_XDECREF()
in the destructor.3. emit with datetime only object fails
Running emit on a dataframe which contains only datetime64[ns] columns fails with error message:
Reason is that the default conversion to
numpy
expects only objects as cell items. For the case where only one column of typeNPY_DATETIME
is in the source dataframe, a workaround was already implemented (see here). Solution: Convert all items in the dataframe to typeobject
if all columns are of typeNPY_DATETIME
.Minor changes
checkPyPtrIsNull()
->checkPyPtrIsNotNull()
checkPyObjectIsNotNull()