search

stripe-archive / mosql

MongoDB → PostgreSQL streaming replication
MIT License
1.63k stars 225 forks source link

PG::NotNullViolation: ERROR: null value in column "id" violates not-null constraint (Sequel::NotNullConstraintViolation) #93

Closed dcu closed 9 years ago

dcu commented 9 years ago

Crash when the id is "null", I think in this case it should just log the error instead of crashing the whole process.

    /var/lib/gems/1.9.1/gems/sequel-4.14.0/lib/sequel/adapters/postgres.rb:161:in `async_exec': PG::NotNullViolation: ERROR:  null value in column "id" violates not-null constraint (Sequel::NotNullConstraintViolation)
    DETAIL:  Failing row contains (null, 2014-10-22 02:07:23.797, 2014-10-22 02:07:23.797, XXXXX@gmail.com, null, null).
    from /var/lib/gems/1.9.1/gems/sequel-4.14.0/lib/sequel/adapters/postgres.rb:161:in `block in execute_query'
    from /var/lib/gems/1.9.1/gems/sequel-4.14.0/lib/sequel/database/logging.rb:33:in `log_yield'
    from /var/lib/gems/1.9.1/gems/sequel-4.14.0/lib/sequel/adapters/postgres.rb:161:in `execute_query'
    from /var/lib/gems/1.9.1/gems/sequel-4.14.0/lib/sequel/adapters/postgres.rb:148:in `block in execute'
    from /var/lib/gems/1.9.1/gems/sequel-4.14.0/lib/sequel/adapters/postgres.rb:124:in `check_disconnect_errors'
    from /var/lib/gems/1.9.1/gems/sequel-4.14.0/lib/sequel/adapters/postgres.rb:148:in `execute'
    from /var/lib/gems/1.9.1/gems/sequel-4.14.0/lib/sequel/adapters/postgres.rb:492:in `_execute'
    from /var/lib/gems/1.9.1/gems/sequel-4.14.0/lib/sequel/adapters/postgres.rb:316:in `block (2 levels) in execute'
    from /var/lib/gems/1.9.1/gems/sequel-4.14.0/lib/sequel/adapters/postgres.rb:513:in `check_database_errors'
    from /var/lib/gems/1.9.1/gems/sequel-4.14.0/lib/sequel/adapters/postgres.rb:316:in `block in execute'
    from /var/lib/gems/1.9.1/gems/sequel-4.14.0/lib/sequel/database/connecting.rb:250:in `block in synchronize'
    from /var/lib/gems/1.9.1/gems/sequel-4.14.0/lib/sequel/connection_pool/threaded.rb:104:in `hold'
    from /var/lib/gems/1.9.1/gems/sequel-4.14.0/lib/sequel/database/connecting.rb:250:in `synchronize'
    from /var/lib/gems/1.9.1/gems/sequel-4.14.0/lib/sequel/adapters/postgres.rb:316:in `execute'
    from /var/lib/gems/1.9.1/gems/sequel-4.14.0/lib/sequel/dataset/actions.rb:906:in `execute'
    from /var/lib/gems/1.9.1/gems/sequel-4.14.0/lib/sequel/adapters/postgres.rb:636:in `fetch_rows'
    from /var/lib/gems/1.9.1/gems/sequel-4.14.0/lib/sequel/dataset/actions.rb:1010:in `returning_fetch_rows'
    from /var/lib/gems/1.9.1/gems/sequel-4.14.0/lib/sequel/dataset/actions.rb:333:in `insert'
    from /var/lib/gems/1.9.1/gems/sequel-4.14.0/lib/sequel/adapters/shared/postgres.rb:1297:in `insert'
    from /var/lib/gems/1.9.1/gems/sequel-4.14.0/lib/sequel/adapters/shared/postgres.rb:1306:in `insert'
    from /var/lib/gems/1.9.1/gems/mosql-0.4.3/lib/mosql/sql.rb:62:in `upsert!'
    from /var/lib/gems/1.9.1/gems/mosql-0.4.3/lib/mosql/streamer.rb:61:in `block (2 levels) in bulk_upsert'
    from /var/lib/gems/1.9.1/gems/mosql-0.4.3/lib/mosql/streamer.rb:39:in `unsafe_handle_exceptions'
    from /var/lib/gems/1.9.1/gems/mosql-0.4.3/lib/mosql/streamer.rb:60:in `block in bulk_upsert'
    from /var/lib/gems/1.9.1/gems/mosql-0.4.3/lib/mosql/streamer.rb:57:in `each'
    from /var/lib/gems/1.9.1/gems/mosql-0.4.3/lib/mosql/streamer.rb:57:in `rescue in bulk_upsert'
    from /var/lib/gems/1.9.1/gems/mosql-0.4.3/lib/mosql/streamer.rb:52:in `bulk_upsert'
    from /var/lib/gems/1.9.1/gems/mosql-0.4.3/lib/mosql/streamer.rb:152:in `block (4 levels) in import_collection'
    from /var/lib/gems/1.9.1/gems/mosql-0.4.3/lib/mosql/streamer.rb:85:in `track_time'
    from /var/lib/gems/1.9.1/gems/mosql-0.4.3/lib/mosql/streamer.rb:151:in `block (3 levels) in import_collection'
    from /var/lib/gems/1.9.1/gems/mongo-1.10.2/lib/mongo/cursor.rb:335:in `each'
    from /var/lib/gems/1.9.1/gems/mosql-0.4.3/lib/mosql/streamer.rb:146:in `block (2 levels) in import_collection'
    from /var/lib/gems/1.9.1/gems/mosql-0.4.3/lib/mosql/streamer.rb:70:in `block in with_retries'
    from /var/lib/gems/1.9.1/gems/mosql-0.4.3/lib/mosql/streamer.rb:68:in `times'
    from /var/lib/gems/1.9.1/gems/mosql-0.4.3/lib/mosql/streamer.rb:68:in `with_retries'
    from /var/lib/gems/1.9.1/gems/mosql-0.4.3/lib/mosql/streamer.rb:145:in `block in import_collection'
    from /var/lib/gems/1.9.1/gems/mongo-1.10.2/lib/mongo/collection.rb:291:in `find'
    from /var/lib/gems/1.9.1/gems/mosql-0.4.3/lib/mosql/streamer.rb:144:in `import_collection'
    from /var/lib/gems/1.9.1/gems/mosql-0.4.3/lib/mosql/streamer.rb:122:in `block (2 levels) in initial_import'
    from /var/lib/gems/1.9.1/gems/mosql-0.4.3/lib/mosql/streamer.rb:120:in `each'
    from /var/lib/gems/1.9.1/gems/mosql-0.4.3/lib/mosql/streamer.rb:120:in `block in initial_import'
    from /var/lib/gems/1.9.1/gems/mosql-0.4.3/lib/mosql/streamer.rb:108:in `each'
    from /var/lib/gems/1.9.1/gems/mosql-0.4.3/lib/mosql/streamer.rb:108:in `initial_import'
    from /var/lib/gems/1.9.1/gems/mosql-0.4.3/lib/mosql/streamer.rb:28:in `import'
    from /var/lib/gems/1.9.1/gems/mosql-0.4.3/lib/mosql/cli.rb:167:in `run'
    from /var/lib/gems/1.9.1/gems/mosql-0.4.3/lib/mosql/cli.rb:16:in `run'
    from /var/lib/gems/1.9.1/gems/mosql-0.4.3/bin/mosql:5:in `<top (required)>'
    from /usr/local/bin/mosql:23:in `load'
    from /usr/local/bin/mosql:23:in `<main>'
dcu commented 9 years ago

I just found the --unsafe option. I'll try again.