search

stripe-archive / mosql

MongoDB → PostgreSQL streaming replication
MIT License
1.63k stars 225 forks source link

`literal_other_append': can't express BSON::ObjectId('546ae5f4e4b08b18f2f77c0b') as a SQL literal (Sequel::Error) #81

Closed sbailliez closed 9 years ago

sbailliez commented 9 years ago

Upgraded to 0.4.1 from 0.3.2 (and changed the _extra_props to JSON from true).

I get:

2014-11-18_06:52:05.81922 INFO MoSQL: Mongd DB 'admin' not found in config file. Skipping. 2014-11-18_06:52:05.81937 INFO MoSQL: Mongd DB 'testdb' not found in config file. Skipping. 2014-11-18_06:52:05.81947 INFO MoSQL: Mongd DB 'config' not found in config file. Skipping. 2014-11-18_06:52:05.81958 INFO MoSQL: Mongd DB 'mongo' not found in config file. Skipping. 2014-11-18_06:52:05.81968 INFO MoSQL: Mongd DB 'primary' not found in config file. Skipping. 2014-11-18_06:52:05.81983 INFO MoSQL: Mongd DB 'test' not found in config file. Skipping. 2014-11-18_06:52:05.83156 INFO Mongoriver: Saved state: {"time"=>nil, "position"=>seconds: 1416291027, increment: 7} 2014-11-18_06:52:05.84843 INFO Mongoriver: Saved state: {"time"=>2014-11-18 06:10:27 +0000, "position"=>seconds: 1416291027, increment: 1} 2014-11-18_06:52:05.87047 INFO Mongoriver: Saved state: {"time"=>2014-11-18 06:11:40 +0000, "position"=>seconds: 1416291100, increment: 1} 2014-11-18_06:52:05.89515 INFO Mongoriver: Saved state: {"time"=>2014-11-18 06:13:01 +0000, "position"=>seconds: 1416291181, increment: 1} 2014-11-18_06:52:05.94582 INFO Mongoriver: Saved state: {"time"=>2014-11-18 06:14:27 +0000, "position"=>seconds: 1416291267, increment: 1} 2014-11-18_06:52:05.95051 INFO Mongoriver: Saved state: {"time"=>2014-11-18 06:16:27 +0000, "position"=>seconds: 1416291387, increment: 1} 2014-11-18_06:52:06.03588 INFO Mongoriver: Saved state: {"time"=>2014-11-18 06:17:47 +0000, "position"=>seconds: 1416291467, increment: 1} 2014-11-18_06:52:06.11116 INFO Mongoriver: Saved state: {"time"=>2014-11-18 06:18:59 +0000, "position"=>seconds: 1416291539, increment: 1} 2014-11-18_06:52:06.16356 INFO Mongoriver: Saved state: {"time"=>2014-11-18 06:20:07 +0000, "position"=>seconds: 1416291607, increment: 1} 2014-11-18_06:52:06.20929 INFO Mongoriver: Saved state: {"time"=>2014-11-18 06:21:27 +0000, "position"=>seconds: 1416291687, increment: 1} 2014-11-18_06:52:06.28503 INFO Mongoriver: Saved state: {"time"=>2014-11-18 06:22:44 +0000, "position"=>seconds: 1416291764, increment: 1} 2014-11-18_06:52:06.31137 INFO Mongoriver: Saved state: {"time"=>2014-11-18 06:23:48 +0000, "position"=>seconds: 1416291828, increment: 1} 2014-11-18_06:52:06.31346 /usr/lib64/ruby/gems/1.9.1/gems/sequel-4.16.0/lib/sequel/dataset/sql.rb:1230:in literal_other_append': can't express BSON::ObjectId('546ae5f4e4b08b18f2f77c0b') as a SQL literal (Sequel::Error) 2014-11-18_06:52:06.31374 from /usr/lib64/ruby/gems/1.9.1/gems/sequel-4.16.0/lib/sequel/dataset/sql.rb:107:inliteral_append' 2014-11-18_06:52:06.31380 from /usr/lib64/ruby/gems/1.9.1/gems/sequel-4.16.0/lib/sequel/dataset/sql.rb:484:in complex_expression_sql_append' 2014-11-18_06:52:06.31387 from /usr/lib64/ruby/gems/1.9.1/gems/sequel-4.16.0/lib/sequel/adapters/shared/postgres.rb:1247:incomplex_expression_sql_append' 2014-11-18_06:52:06.31394 from /usr/lib64/ruby/gems/1.9.1/gems/sequel-4.16.0/lib/sequel/sql.rb:107:in to_s_append' 2014-11-18_06:52:06.31410 from /usr/lib64/ruby/gems/1.9.1/gems/sequel-4.16.0/lib/sequel/dataset/sql.rb:1192:inliteral_expression_append' 2014-11-18_06:52:06.31416 from /usr/lib64/ruby/gems/1.9.1/gems/sequel-4.16.0/lib/sequel/dataset/sql.rb:85:in literal_append' 2014-11-18_06:52:06.31422 from /usr/lib64/ruby/gems/1.9.1/gems/sequel-4.16.0/lib/sequel/dataset/sql.rb:1429:inselect_where_sql' 2014-11-18_06:52:06.31429 from /usr/lib64/ruby/gems/1.9.1/gems/sequel-4.16.0/lib/sequel/dataset/sql.rb:230:in delete_sql' 2014-11-18_06:52:06.31435 from /usr/lib64/ruby/gems/1.9.1/gems/sequel-4.16.0/lib/sequel/dataset/actions.rb:118:indelete' 2014-11-18_06:52:06.31441 from /usr/lib64/ruby/gems/1.9.1/gems/mosql-0.4.1/lib/mosql/streamer.rb:195:in sync_object' 2014-11-18_06:52:06.31447 from /usr/lib64/ruby/gems/1.9.1/gems/mosql-0.4.1/lib/mosql/streamer.rb:239:inhandle_op' 2014-11-18_06:52:06.31454 from /usr/lib64/ruby/gems/1.9.1/gems/mosql-0.4.1/lib/mosql/streamer.rb:176:in block in optail' 2014-11-18_06:52:06.31461 from /usr/lib64/ruby/gems/1.9.1/gems/mongoriver-0.4.0/lib/mongoriver/abstract_persistent_tailer.rb:33:inblock in stream' 2014-11-18_06:52:06.31467 from /usr/lib64/ruby/gems/1.9.1/gems/mongoriver-0.4.0/lib/mongoriver/tailer.rb:162:in call' 2014-11-18_06:52:06.31487 from /usr/lib64/ruby/gems/1.9.1/gems/mongoriver-0.4.0/lib/mongoriver/tailer.rb:162:instream' 2014-11-18_06:52:06.31513 from /usr/lib64/ruby/gems/1.9.1/gems/mongoriver-0.4.0/lib/mongoriver/abstract_persistent_tailer.rb:32:in stream' 2014-11-18_06:52:06.31520 from /usr/lib64/ruby/gems/1.9.1/gems/mosql-0.4.1/lib/mosql/streamer.rb:175:inoptail' 2014-11-18_06:52:06.31526 from /usr/lib64/ruby/gems/1.9.1/gems/mosql-0.4.1/lib/mosql/cli.rb:170:in run' 2014-11-18_06:52:06.31532 from /usr/lib64/ruby/gems/1.9.1/gems/mosql-0.4.1/lib/mosql/cli.rb:16:inrun' 2014-11-18_06:52:06.31538 from /usr/lib64/ruby/gems/1.9.1/gems/mosql-0.4.1/bin/mosql:5:in <top (required)>' 2014-11-18_06:52:06.31544 from /usr/bin/mosql:23:inload' 2014-11-18_06:52:06.31550 from /usr/bin/mosql:23:in `

'

nelhage commented 9 years ago

I believe this bug is fixed in 0.4.2, which I just pushed to rubygems. Can you test that and reopen if you still have this issue?

sbailliez commented 9 years ago

Thanks, will give it a shot this week end and let you know !

jtmarmon commented 9 years ago

hey @nelhage @sbailliez - I'm on mosql v 0.4.2 and still getting this error. any ideas?

AboulEinein commented 8 years ago

@jtmarmon I ran into the same problem and the cause of the problem was that in my Mongo schema I had an Array of Objects, and each object had _id and in my YAML Collection Map file I was mapping this field to JSONB ARRAY and I guess it was failing to cast that _id.

I'm not sure if there's a fix for this, what I did for now was removing the definition of that field so it ends up in the _extra_props column.