search

stripe-archive / mosql

MongoDB → PostgreSQL streaming replication
MIT License
1.62k stars 224 forks source link

literal_other_append error #79

Closed ryanatwork closed 10 years ago

ryanatwork commented 10 years ago

Just updated our MoSQL instance to 0.4.1 and Ruby 2.1

Getting the following error:

/app/vendor/bundle/ruby/2.1.0/gems/sequel-4.16.0/lib/sequel/dataset/sql.rb:1230:in `literal_other_append': can't express BSON::ObjectId('52f3c511e045f9b21d002dc9') as a SQL literal (Sequel::Error)

nelhage commented 10 years ago

Can you paste a full stack trace?

ryanatwork commented 10 years ago

Sure..

2014-11-07T20:51:13.098247+00:00 app[worker.1]: /app/vendor/bundle/ruby/1.9.1/gems/sequel-4.16.0/lib/sequel/dataset/sql.rb:1230:in `literal_other_append': can't express BSON::ObjectId('52f3c511e045f9b21d002dc9') as a SQL literal (Sequel::Error)
2014-11-07T20:51:13.098254+00:00 app[worker.1]:     from /app/vendor/bundle/ruby/1.9.1/gems/sequel-4.16.0/lib/sequel/adapters/shared/postgres.rb:1247:in `complex_expression_sql_append'
2014-11-07T20:51:13.098256+00:00 app[worker.1]:     from /app/vendor/bundle/ruby/1.9.1/gems/sequel-4.16.0/lib/sequel/sql.rb:107:in `to_s_append'
2014-11-07T20:51:13.098261+00:00 app[worker.1]:     from /app/vendor/bundle/ruby/1.9.1/gems/sequel-4.16.0/lib/sequel/dataset/sql.rb:85:in `literal_append'
2014-11-07T20:51:13.098262+00:00 app[worker.1]:     from /app/vendor/bundle/ruby/1.9.1/gems/sequel-4.16.0/lib/sequel/dataset/sql.rb:1429:in `select_where_sql'
2014-11-07T20:51:13.098253+00:00 app[worker.1]:     from /app/vendor/bundle/ruby/1.9.1/gems/sequel-4.16.0/lib/sequel/dataset/sql.rb:484:in `complex_expression_sql_append'
2014-11-07T20:51:13.098257+00:00 app[worker.1]:     from /app/vendor/bundle/ruby/1.9.1/gems/sequel-4.16.0/lib/sequel/dataset/sql.rb:1192:in `literal_expression_append'
2014-11-07T20:51:13.098263+00:00 app[worker.1]:     from /app/vendor/bundle/ruby/1.9.1/gems/sequel-4.16.0/lib/sequel/dataset/sql.rb:230:in `delete_sql'
2014-11-07T20:51:13.098267+00:00 app[worker.1]:     from /app/vendor/bundle/ruby/1.9.1/gems/mosql-0.4.1/lib/mosql/streamer.rb:239:in `handle_op'
2014-11-07T20:51:13.098268+00:00 app[worker.1]:     from /app/vendor/bundle/ruby/1.9.1/gems/mosql-0.4.1/lib/mosql/streamer.rb:176:in `block in optail'
2014-11-07T20:51:13.098269+00:00 app[worker.1]:     from /app/vendor/bundle/ruby/1.9.1/gems/mongoriver-0.4.0/lib/mongoriver/abstract_persistent_tailer.rb:33:in `block in stream'
2014-11-07T20:51:13.098270+00:00 app[worker.1]:     from /app/vendor/bundle/ruby/1.9.1/gems/mongoriver-0.4.0/lib/mongoriver/tailer.rb:162:in `call'
2014-11-07T20:51:13.098265+00:00 app[worker.1]:     from /app/vendor/bundle/ruby/1.9.1/gems/mosql-0.4.1/lib/mosql/streamer.rb:195:in `sync_object'
2014-11-07T20:51:13.098278+00:00 app[worker.1]:     from /app/vendor/bundle/ruby/1.9.1/gems/mosql-0.4.1/lib/mosql/cli.rb:16:in `run'
2014-11-07T20:51:13.098273+00:00 app[worker.1]:     from /app/vendor/bundle/ruby/1.9.1/gems/mongoriver-0.4.0/lib/mongoriver/tailer.rb:162:in `stream'
2014-11-07T20:51:13.098275+00:00 app[worker.1]:     from /app/vendor/bundle/ruby/1.9.1/gems/mongoriver-0.4.0/lib/mongoriver/abstract_persistent_tailer.rb:32:in `stream'
2014-11-07T20:51:13.098276+00:00 app[worker.1]:     from /app/vendor/bundle/ruby/1.9.1/gems/mosql-0.4.1/lib/mosql/streamer.rb:175:in `optail'
2014-11-07T20:51:13.098277+00:00 app[worker.1]:     from /app/vendor/bundle/ruby/1.9.1/gems/mosql-0.4.1/lib/mosql/cli.rb:170:in `run'
2014-11-07T20:51:13.098251+00:00 app[worker.1]:     from /app/vendor/bundle/ruby/1.9.1/gems/sequel-4.16.0/lib/sequel/dataset/sql.rb:107:in `literal_append'
2014-11-07T20:51:13.098264+00:00 app[worker.1]:     from /app/vendor/bundle/ruby/1.9.1/gems/sequel-4.16.0/lib/sequel/dataset/actions.rb:118:in `delete'
2014-11-07T20:51:13.098279+00:00 app[worker.1]:     from /app/vendor/bundle/ruby/1.9.1/gems/mosql-0.4.1/bin/mosql:5:in `<top (required)>'
2014-11-07T20:51:13.098297+00:00 app[worker.1]:     from /app/vendor/bundle/ruby/1.9.1/bin/mosql:23:in `load'
2014-11-07T20:51:13.098298+00:00 app[worker.1]:     from /app/vendor/bundle/ruby/1.9.1/bin/mosql:23:in `<main>'
2014-11-07T20:51:13.916233+00:00 heroku[worker.1]: Process exited with status 1
2014-11-07T20:51:13.927391+00:00 heroku[worker.1]: State changed from up to crashed
nelhage commented 10 years ago

@ryanatwork Just pushed a fix. If you're able to test it and confirm you're working again, that'd be awesome. I can cut a new release if that would be helpful.

ryanatwork commented 10 years ago

@nelhage Thanks! Just deployed it, looking good so far.

ryanatwork commented 10 years ago

@nelhage well.. spoke to soon maybe. We deployed this but now data isnt passing from our mongo to postgres instance. I tried reverting but now its stuck (not sending data from mongo to postgres). Any ideas?

nelhage commented 10 years ago

Can you include more information about what concretely you're seeing, as well as some log output from running with -vv?

ryanatwork commented 10 years ago

I'm not sure what happened. It looks like it may have been a timestamp thing? Anyway, I deleted the contents of mosql_tailers and it caused a reimport. So our data is back to flowing.

Not sure if anyone else has had an issue going from 0.3 to 0.4.