Bobspadger / pyodbc

Automatically exported from code.google.com/p/pyodbc
MIT No Attribution
0 stars 0 forks source link

Crashes with large data sets if you close before fetching all results #92

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago
What steps will reproduce the problem?
1. cursor.execute a query on a vertica table that returns 50k+ rows
2. cursor.fetchmany(1000)
3. cursor.close()

What is the expected output? What do you see instead?
I expected to get the first 1000 rows of the result.  Instead, it crashed 
silently with exit code 245.  Trying this in the repl prints Segmentation 
Fault.

What version of the product are you using? On what operating system?
'2.1.7-beta0'
CentOS
Python 2.6.4
/usr/local/lib/vertica_3.5.9_odbc_3.5_unixodbc_x86_64_linux.so

Please provide any additional information below.
I'm not sure whether this is the fault of pyodbc, unixodbc, or the vertica 
odbc driver, but it's an aggravating problem.  I must support arbitrary 
queries since this is an internal tool and users will be writing queries all 
the 
time, so I can't rely on people always putting in limits.  I don't want to 
fetch 50k rows.

Original issue reported on code.google.com by nickreta...@gmail.com on 9 Mar 2010 at 8:00

GoogleCodeExporter commented 9 years ago
Huh, it's worse than I thought.  You can skip step 2 and it'll still crash.  
Any time you 
execute with a large result set and then neglect to fetch any or all of the 
rows, it 
crashes.

Original comment by nickreta...@gmail.com on 9 Mar 2010 at 8:07

GoogleCodeExporter commented 9 years ago
Curious.  Calling rowcount on my query returns 51,214, even though the result 
set 
should include 1,299,764,619 rows.  Calling my query with a limit of 51,219 
works 
properly and returns that many rows, but raising the limit to 51,220+ makes it 
claim 
only 51,214 rows were returned and then segfault on closing.

Original comment by nickreta...@gmail.com on 9 Mar 2010 at 8:20

GoogleCodeExporter commented 9 years ago
Hey Nick,  did you try bringing this up with vertica tech support?  Perhaps the 
error is in the vertica driver 
implementation?

Original comment by bs1...@gmail.com on 29 Mar 2010 at 9:31

GoogleCodeExporter commented 9 years ago
Any more information on this?  I'm not reproducing it here, but I don't have 
access to that particular database/driver.

Original comment by mkleehammer on 24 Aug 2010 at 4:06

GoogleCodeExporter commented 9 years ago
I don't know.  I don't work with that project anymore.  If I still had access 
to those emails, I could be more specific.

I did bring it up with the Vertica techs.  I think this is related to the other 
bug I posted, where memory issues happen due to the double size characters in 
Vertica.  The Vertica techs told me to always configure it to expect double 
size characters when working with pyodbc, or something.

Original comment by nickreta...@gmail.com on 24 Aug 2010 at 10:39

GoogleCodeExporter commented 9 years ago
Since we are unable to reproduce and you are no longer working with this, I'm 
going to close this.  Feel free to reopen if you encounter this again.

Original comment by mkleehammer on 21 Nov 2010 at 6:28