search

brianmario / mysql2

A modern, simple and very fast Mysql library for Ruby - binding to libmysql
http://github.com/brianmario/mysql2
MIT License
2.24k stars 550 forks source link

MySQL 8.1 and mysql2 gem version 0.5.5 - ArgumentError: negative string size (or size too big) #1333

Open masak2009 opened 1 year ago

masak2009 commented 1 year ago

1) Installed fresh Mysql 8.1 on Debian 11 2) Project - Ruby on Rails 6.1 with Ruby 2.6.5 on MacOS 10.15.7

when run: 

rake db:migrate

or

 rake db:seed

will generate error in abstract_mysql_adapter.rb:760:in configure_connection:

ArgumentError: negative string size (or size too big)

Here is database.yml:

development:
  adapter: mysql2
  encoding: utf8
  pool: 5
  timeout: 5000
  port: 3307
  reconnect: true
  username: someuser
  password: somepassword
  database: some_database
  host: 127.0.0.1

Important information to understand the configuration in yml file: Connect to Mysql is established via ssh tunnel to Debian 11 Server.

Here is full error:

rake aborted!
ArgumentError: negative string size (or size too big)
/Users/someuser/.rvm/gems/ruby-2.6.5@awesomeproject/gems/activerecord-6.1.7.4/lib/active_record/connection_adapters/mysql2_adapter.rb:103:in `escape'
/Users/someuser/.rvm/gems/ruby-2.6.5@awesomeproject/gems/activerecord-6.1.7.4/lib/active_record/connection_adapters/mysql2_adapter.rb:103:in `quote_string'
/Users/someuser/.rvm/gems/ruby-2.6.5@awesomeproject/gems/activerecord-6.1.7.4/lib/active_record/connection_adapters/abstract/quoting.rb:229:in `_quote'
/Users/someuser/.rvm/gems/ruby-2.6.5@awesomeproject/gems/activerecord-6.1.7.4/lib/active_record/connection_adapters/abstract/quoting.rb:20:in `quote'
/Users/someuser/.rvm/gems/ruby-2.6.5@awesomeproject/gems/activerecord-6.1.7.4/lib/active_record/connection_adapters/abstract_mysql_adapter.rb:760:in `configure_connection'
/Users/someuser/.rvm/gems/ruby-2.6.5@awesomeproject/gems/activerecord-6.1.7.4/lib/active_record/connection_adapters/mysql2_adapter.rb:144:in `configure_connection'
/Users/someuser/.rvm/gems/ruby-2.6.5@awesomeproject/gems/activerecord-6.1.7.4/lib/active_record/connection_adapters/mysql2_adapter.rb:53:in `initialize'
/Users/someuser/.rvm/gems/ruby-2.6.5@awesomeproject/gems/activerecord-6.1.7.4/lib/active_record/connection_adapters/mysql2_adapter.rb:22:in `new'
/Users/someuser/.rvm/gems/ruby-2.6.5@awesomeproject/gems/activerecord-6.1.7.4/lib/active_record/connection_adapters/mysql2_adapter.rb:22:in `mysql2_connection'
/Users/someuser/.rvm/gems/ruby-2.6.5@awesomeproject/gems/activerecord-6.1.7.4/lib/active_record/connection_adapters/abstract/connection_pool.rb:882:in `public_send'
/Users/someuser/.rvm/gems/ruby-2.6.5@awesomeproject/gems/activerecord-6.1.7.4/lib/active_record/connection_adapters/abstract/connection_pool.rb:882:in `new_connection'
/Users/someuser/.rvm/gems/ruby-2.6.5@awesomeproject/gems/activerecord-6.1.7.4/lib/active_record/connection_adapters/abstract/connection_pool.rb:926:in `checkout_new_connection'
/Users/someuser/.rvm/gems/ruby-2.6.5@awesomeproject/gems/activerecord-6.1.7.4/lib/active_record/connection_adapters/abstract/connection_pool.rb:905:in `try_to_checkout_new_connection'
/Users/someuser/.rvm/gems/ruby-2.6.5@awesomeproject/gems/activerecord-6.1.7.4/lib/active_record/connection_adapters/abstract/connection_pool.rb:866:in `acquire_connection'
/Users/someuser/.rvm/gems/ruby-2.6.5@awesomeproject/gems/activerecord-6.1.7.4/lib/active_record/connection_adapters/abstract/connection_pool.rb:588:in `checkout'
/Users/someuser/.rvm/gems/ruby-2.6.5@awesomeproject/gems/activerecord-6.1.7.4/lib/active_record/connection_adapters/abstract/connection_pool.rb:428:in `connection'
/Users/someuser/.rvm/gems/ruby-2.6.5@awesomeproject/gems/activerecord-6.1.7.4/lib/active_record/connection_adapters/abstract/connection_pool.rb:1128:in `retrieve_connection'
/Users/someuser/.rvm/gems/ruby-2.6.5@awesomeproject/gems/activerecord-6.1.7.4/lib/active_record/connection_handling.rb:327:in `retrieve_connection'
/Users/someuser/.rvm/gems/ruby-2.6.5@awesomeproject/gems/activerecord-6.1.7.4/lib/active_record/connection_handling.rb:283:in `connection'
/Users/someuser/.rvm/gems/ruby-2.6.5@awesomeproject/gems/activerecord-6.1.7.4/lib/active_record/tasks/database_tasks.rb:237:in `migrate'
/Users/someuser/.rvm/gems/ruby-2.6.5@awesomeproject/gems/activerecord-6.1.7.4/lib/active_record/railties/databases.rake:92:in `block (3 levels) in <main>'
/Users/someuser/.rvm/gems/ruby-2.6.5@awesomeproject/gems/activerecord-6.1.7.4/lib/active_record/railties/databases.rake:90:in `each'
/Users/someuser/.rvm/gems/ruby-2.6.5@awesomeproject/gems/activerecord-6.1.7.4/lib/active_record/railties/databases.rake:90:in `block (2 levels) in <main>'
/Users/someuser/.rvm/gems/ruby-2.6.5@awesomeproject/gems/rake-13.0.6/exe/rake:27:in `<top (required)>'
/Users/someuser/.rvm/gems/ruby-2.6.5@awesomeproject/bin/ruby_executable_hooks:22:in `eval'
/Users/someuser/.rvm/gems/ruby-2.6.5@awesomeproject/bin/ruby_executable_hooks:22:in `<main>'
sodabrew commented 1 year ago

Does this succeed if the server is MySQL 8.0? That would help narrow down if there is a recent change in MySQL 8.1 or if there is some other problem to solve in the Rails configs, specifically in your db migrations or db structure files.

sodabrew commented 1 year ago

Hmm, ok it's here in Rails: https://github.com/rails/rails/blob/90b0266e9bfaaadbd2d249ad2eeec042077a0063/activerecord/lib/active_record/connection_adapters/abstract_mysql_adapter.rb#L756-L771

As a workaround, try defining sql_mode: and then sql_mode: STRICT_ALL_TABLES in your database.yml. That may or may not be the actual setting you want, but the point is just to get a string value in there (you could try sql_mode: helloworld to force an error that confirms the quoting behavior worked). It's possible that the mysqlclient escape function used to return something usable for an empty string or null string but no longer does. In this case, it may not be the server version but the client library version that's at issue. What MySQL client library are you using on your local (macOS) installation? (Or are you running under Docker locally?)

https://github.com/rails/rails/blob/6-1-stable/activerecord/lib/active_record/connection_adapters/mysql2_adapter.rb#L102-L106

masak2009 commented 1 year ago

Hi, I tried sql_mode:whatever but still same error "ArgumentError: negative string size (or size too big)". I don't uderstand what you mean about mysqlclient. Do you mean binary file "mysql" under "mysql installation dir"? I have two versions ( available is only one at time of running rails app by symlink in right available path): Ver 14.14 Distrib 5.7.29, for macos10.14 (From MySQL 5.7 Server installation) or Ver 8.0.22 for macos10.15 on x86_64 (From Mysql Workbench 8 application)

I tried both mysql files for connect to Mysql 8.1 on Debian 11 and it works.

I found on stackoverflow that "ArgumentError: negative string size (or size too big)" should be raised because encoding did not match DB vs client (but issue was for Ruby on Rails 3). I found in Workbench that Mysql after create schema show utf8mb3 and utf8mb3_czech_ci but Ruby on Rails database.yml definition is utf8 , utf8_czech_ci. When i tried to change it in database yml to utf8mb3 then it fails on "Mysql2::Error: Unsupported charset: '"utf8mb3"'"

sodabrew commented 1 year ago

The executable file mysql that runs the command-line client is itself a wrapper around the libmysqlclient.so shared library. The library manages the connection to the server. The mysql2 gem uses the same library as well, so roughly speaking anything you can do in mysql command line you can do in Ruby code.

Ok, that's really good debugging you found that this error message is a bit misleading! Try these:

  encoding: utf8

or

  encoding: utf8mb4
masak2009 commented 1 year ago

I tried both before posting this issue: 1) Create DB schema as utf8 on Mysql 8.1 and then connect with encoding: utf8. Workbench show encoding as utf8mb3 in schema inspector. 2) Create DB schema as utf8mb4 on Mysql 8.1 and then connect with encoding: utf8mb4. Workbench show encoding as utf8mb4 in schema inspector.

Both connections end on same error: "ArgumentError: negative string size (or size too big)"

I remember when install Mysql 8.1 i chose "compatible connection" which allow connect old mysql clients to Mysql 8.1 (i dont remember the exact name of the setting). I think it works, because mysql 5.7 binary client should connect and operate and mysql 8.0 client too.

sodabrew commented 1 year ago

I think that setting actually refers to very old MySQL protocols. MySQL 5.7 and 8.0 speak the same.