BritishGeologicalSurvey / etlhelper

ETL Helper is a Python ETL library to simplify data transfer into and out of databases.
https://britishgeologicalsurvey.github.io/etlhelper/
GNU Lesser General Public License v3.0
100 stars 25 forks source link

Skip test that passes locally but fails in CI #210

Closed ximenesuk closed 2 months ago

ximenesuk commented 2 months ago

The SQL Server implementation of executemany takes advantage of a cursor flag fast_executemany to increase the speed of bulk queries ten-fold. However, some circumstances cause this method to fail, in particular tables with deprecated column types such as TEXT and NTEXT. The executemany method catches the MemoryError raised in such cases, issues a warning and tries the query again with fast_executemany set to False.

The test for this special case checks for the warning. Since moving to msodbcsql18, locally this test still passes, but on the BGS internal gitlab CI pipeline, using the same docker image, it fails as no warning is issued. So far the reason for this hasn't been found and using a mock was ruled out as it would effectively only test the mock. So, in order to move ahead, it has been decided that this test should be skipped until the issue can be investigated further..

See https://github.com/mkleehammer/pyodbc/wiki/Features-beyond-the-DB-API#fast_executemany for the background to this.