I encountered a cryptic error when trying to run schema migration via RAILS_ENV=test rails db:create db:schema:load:
bin/rails aborted!
ActiveRecord::ActiveRecordError: Response code: 500: (ActiveRecord::ActiveRecordError)
Code: 33. DB::Exception: Cannot read data after semicolon (and input_format_values_allow_data_after_semicolon=0): While executing ValuesBlockInputFormat. (CANNOT_READ_ALL_DATA) (version 24.1.5.6 (official build))
Full log
bin/rails aborted!
ActiveRecord::ActiveRecordError: Response code: 500: (ActiveRecord::ActiveRecordError)
Code: 33. DB::Exception: Cannot read data after semicolon (and input_format_values_allow_data_after_semicolon=0): While executing ValuesBlockInputFormat. (CANNOT_READ_ALL_DATA) (version 24.1.5.6 (official build))
[REDACTED]/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/clickhouse-activerecord-1.0.4/lib/active_record/connection_adapters/clickhouse/schema_statements.rb:135:in `process_response'
[REDACTED]/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/clickhouse-activerecord-1.0.4/lib/active_record/connection_adapters/clickhouse/schema_statements.rb:80:in `block in do_execute'
[REDACTED]/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/activesupport-7.1.2/lib/active_support/notifications/instrumenter.rb:58:in `instrument'
[REDACTED]/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.2/lib/active_record/connection_adapters/abstract_adapter.rb:1143:in `log'
[REDACTED]/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/clickhouse-activerecord-1.0.4/lib/active_record/connection_adapters/clickhouse/schema_statements.rb:74:in `do_execute'
[REDACTED]/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/clickhouse-activerecord-1.0.4/lib/active_record/connection_adapters/clickhouse/schema_statements.rb:100:in `assume_migrated_upto_version'
[REDACTED]/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.2/lib/active_record/schema.rb:59:in `define'
[REDACTED]/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.2/lib/active_record/schema.rb:50:in `define'
/Users/jensk/Projects/comgy/mh-platform/db/clickhouse_schema.rb:13:in `'
[REDACTED]/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.2/lib/active_record/tasks/database_tasks.rb:358:in `load'
[REDACTED]/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.2/lib/active_record/tasks/database_tasks.rb:358:in `load_schema'
[REDACTED]/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.2/lib/active_record/tasks/database_tasks.rb:453:in `block (2 levels) in load_schema_current'
[REDACTED]/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.2/lib/active_record/tasks/database_tasks.rb:501:in `block in with_temporary_connection'
[REDACTED]/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.2/lib/active_record/tasks/database_tasks.rb:518:in `with_temporary_pool'
[REDACTED]/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.2/lib/active_record/tasks/database_tasks.rb:500:in `with_temporary_connection'
[REDACTED]/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.2/lib/active_record/tasks/database_tasks.rb:452:in `block in load_schema_current'
[REDACTED]/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.2/lib/active_record/tasks/database_tasks.rb:562:in `block (2 levels) in each_current_configuration'
[REDACTED]/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.2/lib/active_record/tasks/database_tasks.rb:559:in `each'
[REDACTED]/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.2/lib/active_record/tasks/database_tasks.rb:559:in `block in each_current_configuration'
[REDACTED]/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.2/lib/active_record/tasks/database_tasks.rb:558:in `each'
[REDACTED]/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.2/lib/active_record/tasks/database_tasks.rb:558:in `each_current_configuration'
[REDACTED]/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.2/lib/active_record/tasks/database_tasks.rb:451:in `load_schema_current'
[REDACTED]/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/activerecord-7.1.2/lib/active_record/railties/databases.rake:476:in `block (3 levels) in '
[REDACTED]/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/task.rb:281:in `block in execute'
[REDACTED]/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/task.rb:281:in `each'
[REDACTED]/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/task.rb:281:in `execute'
[REDACTED]/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/task.rb:219:in `block in invoke_with_call_chain'
[REDACTED]/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/task.rb:199:in `synchronize'
[REDACTED]/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/task.rb:199:in `invoke_with_call_chain'
[REDACTED]/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/task.rb:188:in `invoke'
[REDACTED]/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/application.rb:182:in `invoke_task'
[REDACTED]/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/application.rb:138:in `block (2 levels) in top_level'
[REDACTED]/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/application.rb:138:in `each'
[REDACTED]/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/application.rb:138:in `block in top_level'
[REDACTED]/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/application.rb:147:in `run_with_threads'
[REDACTED]/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/application.rb:132:in `top_level'
[REDACTED]/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/railties-7.1.2/lib/rails/commands/rake/rake_command.rb:27:in `block (2 levels) in perform'
[REDACTED]/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/application.rb:208:in `standard_exception_handling'
[REDACTED]/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/railties-7.1.2/lib/rails/commands/rake/rake_command.rb:27:in `block in perform'
[REDACTED]/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/railties-7.1.2/lib/rails/commands/rake/rake_command.rb:44:in `block in with_rake'
[REDACTED]/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.1.0/lib/rake/rake_module.rb:59:in `with_application'
[REDACTED]/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/railties-7.1.2/lib/rails/commands/rake/rake_command.rb:41:in `with_rake'
[REDACTED]/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/railties-7.1.2/lib/rails/commands/rake/rake_command.rb:20:in `perform'
[REDACTED]/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/railties-7.1.2/lib/rails/command.rb:156:in `invoke_rake'
[REDACTED]/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/railties-7.1.2/lib/rails/command.rb:73:in `block in invoke'
[REDACTED]/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/railties-7.1.2/lib/rails/command.rb:149:in `with_argv'
[REDACTED]/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/railties-7.1.2/lib/rails/command.rb:69:in `invoke'
[REDACTED]/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/railties-7.1.2/lib/rails/commands.rb:18:in `'
[REDACTED]/ruby/3.3.0/lib/ruby/3.3.0/bundled_gems.rb:74:in `require'
[REDACTED]/ruby/3.3.0/lib/ruby/3.3.0/bundled_gems.rb:74:in `block (2 levels) in replace_require'
[REDACTED]/ruby/3.3.0/lib/ruby/gems/3.3.0/gems/bootsnap-1.18.3/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:30:in `require'
bin/rails:4:in `'
Tasks: TOP => db:schema:load
I was logging the statements made to the database and found this offending line:
INSERT INTO schema_migrations (version) VALUES
(20230704190200); FORMAT JSONCompact
The semicolon comes from insert_versions_sql.. I assume from the abstract adapter. I believe adding the format clause here is not necessary in insert calls. The exec_insert also sets format to nil, so in my eyes this is a proper fix and increases consistency.
Alternative solution is to configure the clickhouse server to ignore anything after semicolon. However the default configuration is set to false. So to make this gem easier to integrate, I believe that's not the best way to go.
Hey @PNixx,
thank you for developing this gem :).
I encountered a cryptic error when trying to run schema migration via
RAILS_ENV=test rails db:create db:schema:load:Full log
I was logging the statements made to the database and found this offending line:
The semicolon comes from
insert_versions_sql.. I assume from the abstract adapter. I believe adding the format clause here is not necessary in insert calls. The exec_insert also sets format to nil, so in my eyes this is a proper fix and increases consistency.Alternative solution is to configure the clickhouse server to ignore anything after semicolon. However the default configuration is set to false. So to make this gem easier to integrate, I believe that's not the best way to go.