Closed victormartins closed 8 years ago
@victormartins I can say that the only line which could cause NPE on that is if a null value is passed in for resultset. Is this arjdbc or active_record? I don't know. I am working on Rails 5 support right and I am so rusty on the code base I don't know what to suggest.
I guess one thing to try would be to go back to same versions of active-record and ar-jdbc and make sure this is not a JRuby 9k issue. I don't expect it to be but it is something you can do which will eliminate an possible problem. After that perhaps change those two gems and see which one breaks stuff?
Thank you for the input @enebo . I've been following the source code and this is how far I managed to reach at the moment:
On that line, the resultSet
is created in a way that when do resultSet.getMetaData()
, we get a null.
But at this moment I'm not sure how the resultSet
is created.
I haven't find a way to understand the statement object because it comes from here:
https://github.com/jruby/activerecord-jdbc-adapter/blob/v1.3.21/src/java/arjdbc/jdbc/RubyJdbcConnection.java#L610
Somehow the method executeQuery
that gets called on statement
is not producing the right resultSet
.
Maybe it would be a good idea to check and raise if the metada is null here, because we depend on it on like 3444:
https://github.com/jruby/activerecord-jdbc-adapter/blob/v1.3.21/src/java/arjdbc/jdbc/RubyJdbcConnection.java#L3444
Can you help me understand the requirements needed to to have the metadata on the resultSet
object?
Thank you.
@victormartins unless there is more to your backtrace I would say resultSet has to be null itself. Unless we are truncating the output or something? Otherwise it would say where in getMetaData() or within setupColumns the NPE was really occurring.
I guess I would also look at what you are performing in execute_query (which can be printed out by modifying ruby adapter or if you are comfortable by recompiling RubyJdbcConnection.executeQuery.
Once you have a particular query which fails we can maybe think about why that would be but it is strange this used to work in our older setup. So either arjdbc changed in an unexpected way or perhaps Rails changed something internally and it is performing a query that sqlanywhere is incapable of running?
@enebo you are right, the resultSet is null, that's why it breaks on line 3444 on resultSet.getMetaData(), I got confused since I've been trying to understand the rusultSet object what what it should be populated with.
So the pain point is statement.executeQuery(query)
returning null.
I haven't found a way to recompile RubyJdbcConnection but I can easly see all the arguments passed by inspecting this line:
https://github.com/jruby/activerecord-jdbc-adapter/blob/v1.3.21/lib/arjdbc/jdbc/adapter.rb#L455
I know for sure I'm hitting that line, because I have a Pry on it. The following is a Pry session were I try to execute a very simple INSERT query.
I can see that the row is actually inserted on the table, so the problem is definitely only when trying to map the result.
Thank you very much for looking into this.
@victormartins can you pry this with 1.7 and your other deps and see if it called executeQuery? I don't know this for a fact but I am surprised it is not calling executeInsert since it is an insert (or update depending on how you think about it). These I think will only return resultsets if they are asked to.
@kares do you think it is possible we changed something between arjdbc releases which would have flopped us into having some inserts calling query instead of insert?
AR-JDBC 1.3 did not run any tests against SqlAnywhere (or Sybase) ... as far as I know. I actually believe for some future major-ish release it should be drop-ed completely. first time I hear of activerecord-sqlanywhere-jdbc-in4systems-adapter
- no clue whether it depends on any AR-JDBC specific details and thus might have ended up broken.
at some point SAP guys where working on an adapter (on top of AR-JDBC) for their data-source but I am not sure at what state their adapter is. there's likely a query type detection logic that got somehow screwed for Sybase/SqlAnywhere due the lack of testing. if you got a Ruby back-trace (and the SQL) it should be easy to identify (and potentially fix on the 1.3 stable line).
seems these are un-maintained (abandoned) but might work better with newer AR-JDBC :
... I think they might still have better support for SqlAnywhere than the one in AR-JDBC
Hello @enebo I think you just highlighted the problem.
I've just been checking what happens in with our older version that uses jRuby 1.7x and activerecord-jdbc-adapter 1.2.9.1
.
The activerecord-sqlanywhere_jdbc_in4systems_adapter 0.13
sends all our sql like this: conn.exec_query(sql, name, binds)
This will reach the activerecord-jdbc-adapter
here:
https://github.com/jruby/activerecord-jdbc-adapter/blob/v1.2.9.1/lib/arjdbc/jdbc/adapter.rb#L241
https://github.com/jruby/activerecord-jdbc-adapter/blob/v1.2.9.1/lib/arjdbc/jdbc/adapter.rb#L291
So it does not really care what type of SQL it is.
Before I believed that we could use a single entry (exec_query) and that the appropriate method would be fired given the type of SQL it received, but given what you said, it seems that we have to change the activerecord-sqlanywhere_jdbc_in4systems_adapter
code to redirect the SQL appropriately to the new API:
SELECT: https://github.com/jruby/activerecord-jdbc-adapter/blob/v1.3.21/lib/arjdbc/jdbc/adapter.rb#L447
INSERT: https://github.com/jruby/activerecord-jdbc-adapter/blob/v1.3.21/lib/arjdbc/jdbc/adapter.rb#L464
DELETE: https://github.com/jruby/activerecord-jdbc-adapter/blob/v1.3.21/lib/arjdbc/jdbc/adapter.rb#L481
UPDATE: https://github.com/jruby/activerecord-jdbc-adapter/blob/v1.3.21/lib/arjdbc/jdbc/adapter.rb#L498
I will be trying this now, hopefully it will be the end of this problem. :pray:
Hello @enebo , @kares ,
After sending the same simple insert query, through the exec_insert method I now get another error :bomb: . Can you guys help me out understanding what can raise this type of error?
sql = "INSERT INTO \"DBA\".\"test_table\" (\"id\",\"my_col_1\",\"my_col_2\") VALUES(2,3,4)"
conn.exec_insert(sql, name, binds)
ActiveRecord::StatementInvalid: ActiveRecord::JDBCError: Generated keys not supported: INSERT INTO "DBA"."test_table" ("id","my_col_1","my_col_2") VALUES(2,3,4)
from arjdbc/jdbc/RubyJdbcConnection.java:647:in `execute_insert'
@victormartins I am guessing getGeneratedKeys() is not working. One article I read claims this is a JDBC 3.0+ feature but perhaps it depends on the particular JDBC driver you are using? Do you know which jdbc driver you are using?
@enebo do you think we can have the final argument here as something we can pass in to avoid the return of generated keys? https://github.com/jruby/activerecord-jdbc-adapter/blob/v1.3.21/src/java/arjdbc/jdbc/RubyJdbcConnection.java#L647 https://github.com/jruby/activerecord-jdbc-adapter/blob/v1.3.21/src/java/arjdbc/jdbc/RubyJdbcConnection.java#L714
The driver we are using is the one installed by the Sybase DB.
In my case it's installed here C:\Program Files\SQL Anywhere 16\Java\sajdbc4.jar
This is how we configure it:
$CLASSPATH << 'sajdbc4.jar'
$CLASSPATH << Pathname.new(ENV['SQLANY16']).join('java').join('sajdbc4.jar').to_s
driver = 'sybase.jdbc4.sqlanywhere.IDriver'
url = 'jdbc:sqlanywhere:' + connection_string
conn = ActiveRecord::Base.jdbc_connection({adapter: 'jdbc', driver: driver, url: url})
@victormartins maybe. I guess it depends on whether that generated key data is required higher up the call stack or not. As I have discovered working on active-record 5 support we have lots of code paths which are their to support various versions of active-record. What a particular version of active-record wants may dictate whether a change like that will work or not. Of course you can try it.
jruby -S rake build:adapters
This builds all adapters but with JRuby and a JDK you can try out some stuff. There are so many moving parts here I feel like my advice might not be so great.
we certainly should not be changing the base JdbcConnection code (at least not in 1.3 stable) - that is what I meant by Sybase being not really supported. that change is likely to affect other adapters/drivers that are considered more stable (testing all is quite a time-consuming option). the generated keys JDBC feature is certainly required by some adapters and AR-JDBC is relying on it.
to work-around such issues we usually include custom code (and custom JdbcConnection sub-class) as needed but I feel like in your case using one of the adapters (AR-JDBC extension gems) I recommended might be the solution. please read my comments and give them a try before assuming shared pieces might be adapted due a non-compilant JDBC driver.
Hello guys, after a bit of tweaking we got a working solution. We now redirect our SQL to the right API, with the only odd case being the inserts. To execute inserts properly we are sending it through the exec_update because it has the Generated Keys turned off.
This is a snippet that represents what we do:
case statement_type(sql)
when 'select'
:exec_query
when 'insert'
# needs to go through the exec_update to avoid the error of Generated Keys
:exec_update
when 'delete'
:exec_delete
when 'update'
:exec_update
when 'call'
# stored proceadures use this one.
:execute
else
raise ArgumentError, "Cannot identify a SQL type from the statement:\"#{sql}\" using regex: /#{SQL_TYPE_REGEX}/"
end
I'm closing the issue since we have a workaround that's working. The main objective is to drop the Sybase DB, so hopefully this shouldn't be a problem for long.
Thank you very much for the help guys.
Hello everyone, I've been fighting an integration problem with a SqlAnywhere database and currently I'm stuck with a
Java::JavaLang::NullPointerException
in activerecord-jdbc-adapter.I've started from an application running the following configuration, that has no problems using the DB.
jRuby 1.7.15
,Rails 3.2.15
activerecord-jdbc-adapter 1.2.91
activerecord-sqlanywhere-jdbc-in4systems-adapter 1.0.13
However, updating to this configuration I now have the exception:
jRuby 9.1.5
rails 3.2.22.5
activerecord-jdbc-adapter 1.3.21
activerecord-sqlanywhere-jdbc-in4systems-adapter 1.0.13
When trying to execute a INSERT query, I can see that the query is executed correctly, that data gets set in the DB, but I immediately get the exception bellow. I've been trying to find a way to compile a new jar file with some log to debug this, but I've been failing so far. I'm trying to understand what is causing this error, but I can't pinpoint the change that broke it. Can someone please try to highlight what can cause such an exception?
Thank you.