search

stripe-archive / mosql

MongoDB → PostgreSQL streaming replication
MIT License
1.63k stars 225 forks source link

Can't express BSON::ObjectId() as a SQL literal #60

Closed sambauers closed 10 years ago

sambauers commented 10 years ago

Getting this error on initial import. Seems to be related to previous issues posted here with extracting an ID out of MongoDB. I'm not sure what info you need about the MongoDB setup. It's on compose.io, and other collection IDs import fine, so it may be something to do with this particular collection. I can get more info on how this collection may differ from others if you can direct me as to what to look for. On the surface this collection seems the same as all the others.

INFO MoSQL: Importing for api-uat.commercials...
/usr/local/lib64/ruby/gems/2.1.0/gems/sequel-4.13.0/lib/sequel/dataset/sql.rb:1230:in `literal_other_append': can't express BSON::ObjectId('53a93e7efd487b0c9eb267e2') as a SQL literal (Sequel::Error)
    from /usr/local/lib64/ruby/gems/2.1.0/gems/sequel-4.13.0/lib/sequel/dataset/sql.rb:107:in `literal_append'
    from /usr/local/lib64/ruby/gems/2.1.0/gems/sequel-4.13.0/lib/sequel/extensions/pg_array.rb:517:in `block in _literal_append'
    from /usr/local/lib64/ruby/gems/2.1.0/gems/sequel-4.13.0/lib/sequel/extensions/pg_array.rb:512:in `each'
    from /usr/local/lib64/ruby/gems/2.1.0/gems/sequel-4.13.0/lib/sequel/extensions/pg_array.rb:512:in `_literal_append'
    from /usr/local/lib64/ruby/gems/2.1.0/gems/sequel-4.13.0/lib/sequel/extensions/pg_array.rb:496:in `sql_literal_append'
    from /usr/local/lib64/ruby/gems/2.1.0/gems/sequel-4.13.0/lib/sequel/dataset/sql.rb:1226:in `literal_other_append'
    from /usr/local/lib64/ruby/gems/2.1.0/gems/sequel-4.13.0/lib/sequel/dataset/sql.rb:107:in `literal_append'
    from /usr/local/lib64/ruby/gems/2.1.0/gems/sequel-4.13.0/lib/sequel/dataset/sql.rb:1543:in `block in update_set_sql'
    from /usr/local/lib64/ruby/gems/2.1.0/gems/sequel-4.13.0/lib/sequel/dataset/sql.rb:1535:in `each'
    from /usr/local/lib64/ruby/gems/2.1.0/gems/sequel-4.13.0/lib/sequel/dataset/sql.rb:1535:in `update_set_sql'
    from /usr/local/lib64/ruby/gems/2.1.0/gems/sequel-4.13.0/lib/sequel/dataset/sql.rb:228:in `_update_sql'
    from /usr/local/lib64/ruby/gems/2.1.0/gems/sequel-4.13.0/lib/sequel/dataset/sql.rb:174:in `update_sql'
    from /usr/local/lib64/ruby/gems/2.1.0/gems/sequel-4.13.0/lib/sequel/dataset/actions.rb:769:in `update'
    from /usr/local/lib64/ruby/gems/2.1.0/gems/mosql-0.3.2/lib/mosql/sql.rb:51:in `upsert!'
    from /usr/local/lib64/ruby/gems/2.1.0/gems/mosql-0.3.2/lib/mosql/streamer.rb:61:in `block (2 levels) in bulk_upsert'
    from /usr/local/lib64/ruby/gems/2.1.0/gems/mosql-0.3.2/lib/mosql/streamer.rb:39:in `unsafe_handle_exceptions'
    from /usr/local/lib64/ruby/gems/2.1.0/gems/mosql-0.3.2/lib/mosql/streamer.rb:60:in `block in bulk_upsert'
    from /usr/local/lib64/ruby/gems/2.1.0/gems/mosql-0.3.2/lib/mosql/streamer.rb:57:in `each'
    from /usr/local/lib64/ruby/gems/2.1.0/gems/mosql-0.3.2/lib/mosql/streamer.rb:57:in `rescue in bulk_upsert'
    from /usr/local/lib64/ruby/gems/2.1.0/gems/mosql-0.3.2/lib/mosql/streamer.rb:52:in `bulk_upsert'
    from /usr/local/lib64/ruby/gems/2.1.0/gems/mosql-0.3.2/lib/mosql/streamer.rb:150:in `block (4 levels) in import_collection'
    from /usr/local/lib64/ruby/gems/2.1.0/gems/mosql-0.3.2/lib/mosql/streamer.rb:85:in `track_time'
    from /usr/local/lib64/ruby/gems/2.1.0/gems/mosql-0.3.2/lib/mosql/streamer.rb:149:in `block (3 levels) in import_collection'
    from /usr/local/lib64/ruby/gems/2.1.0/gems/mongo-1.10.2/lib/mongo/cursor.rb:335:in `each'
    from /usr/local/lib64/ruby/gems/2.1.0/gems/mosql-0.3.2/lib/mosql/streamer.rb:144:in `block (2 levels) in import_collection'
    from /usr/local/lib64/ruby/gems/2.1.0/gems/mosql-0.3.2/lib/mosql/streamer.rb:70:in `block in with_retries'
    from /usr/local/lib64/ruby/gems/2.1.0/gems/mosql-0.3.2/lib/mosql/streamer.rb:68:in `times'
    from /usr/local/lib64/ruby/gems/2.1.0/gems/mosql-0.3.2/lib/mosql/streamer.rb:68:in `with_retries'
    from /usr/local/lib64/ruby/gems/2.1.0/gems/mosql-0.3.2/lib/mosql/streamer.rb:143:in `block in import_collection'
    from /usr/local/lib64/ruby/gems/2.1.0/gems/mongo-1.10.2/lib/mongo/collection.rb:291:in `find'
    from /usr/local/lib64/ruby/gems/2.1.0/gems/mosql-0.3.2/lib/mosql/streamer.rb:142:in `import_collection'
    from /usr/local/lib64/ruby/gems/2.1.0/gems/mosql-0.3.2/lib/mosql/streamer.rb:119:in `block (2 levels) in initial_import'
    from /usr/local/lib64/ruby/gems/2.1.0/gems/mosql-0.3.2/lib/mosql/streamer.rb:117:in `each'
    from /usr/local/lib64/ruby/gems/2.1.0/gems/mosql-0.3.2/lib/mosql/streamer.rb:117:in `block in initial_import'
    from /usr/local/lib64/ruby/gems/2.1.0/gems/mosql-0.3.2/lib/mosql/streamer.rb:105:in `each'
    from /usr/local/lib64/ruby/gems/2.1.0/gems/mosql-0.3.2/lib/mosql/streamer.rb:105:in `initial_import'
    from /usr/local/lib64/ruby/gems/2.1.0/gems/mosql-0.3.2/lib/mosql/streamer.rb:28:in `import'
    from /usr/local/lib64/ruby/gems/2.1.0/gems/mosql-0.3.2/lib/mosql/cli.rb:162:in `run'
    from /usr/local/lib64/ruby/gems/2.1.0/gems/mosql-0.3.2/lib/mosql/cli.rb:16:in `run'
    from /usr/local/lib64/ruby/gems/2.1.0/gems/mosql-0.3.2/bin/mosql:5:in `<top (required)>'
    from /usr/local/bin/mosql:23:in `load'
    from /usr/local/bin/mosql:23:in `<main>'
sambauers commented 10 years ago

Hey! Thanks for this fix, seems to have worked for my case.