stripe-archive / mosql

MongoDB → PostgreSQL streaming replication
MIT License
1.63k stars 225 forks source link

Hanging on create table? #53

Closed AndreaCrotti closed 10 years ago

AndreaCrotti commented 10 years ago

I think it's just because I haven't understood how to use it yet, but I didn't manage to get the sync working.

Basically what I do is this:

$ mosql -c collections.yaml --mongo mongodb://user:password@localhost:27017/ --sql postgres://user:password@localhost/garage --skip-tail INFO MoSQL: Creating table 'engine_comment'...

And the configuration is very very simple:

alexandria:
  comments:
    :columns:
    - text: TEXT
    - status: TEXT

    :meta:
      :table: engine_comment
      :extra_props: true

So a couple of things I don't understand:

Any hint welcome I'm a bit lost.. thanks a lot

AndreaCrotti commented 10 years ago

I found the no-drop-tables option and now it does not hang, however:

  1. it still doesn't write anything to Postgres
  2. it still says "Creating table", but now it should not create it anymore right?

    vagrant@ubuntu-13:~$ mosql -c collections.yaml --mongo mongodb://user:password@localhost:27017/ --sql postgres://user:password@localhost/garage --skip-tail -v 3 --no-drop-tables INFO MoSQL: Creating table 'engine_comment'... INFO MoSQL: Mongd DB 'local' not found in config file. Skipping. INFO MoSQL: Mongd DB 'garage-test' not found in config file. Skipping. INFO MoSQL: Mongd DB 'activity2' not found in config file. Skipping. INFO MoSQL: Mongd DB 'activity' not found in config file. Skipping. INFO MoSQL: Importing for Mongo DB alexandria... INFO MoSQL: Mongd DB 'garage' not found in config file. Skipping. INFO MoSQL: Mongd DB 'admin' not found in config file. Skipping. INFO MoSQL: Mongd DB 'activity_hfi' not found in config file. Skipping. INFO MoSQL: Mongd DB 'chat' not found in config file. Skipping. INFO MoSQL: Mongd DB 'test' not found in config file. Skipping

Thanks a lot

nelhage commented 10 years ago

By default, MoSQL drops and re-creates all tables on an initial import, in order to ensure a clean import. You can pass --no-drop-tables to inhibit the dropping behavior, if necessary.

It will still print "Creating table", but with --no-drop-tables what that actually indicates is just that it's checking for existence and has found the table.

MoSQL's -v option does not take a verbosity level; Try -v -v -v to get more debug output, and hopefully that will reveal more detail. I suspect that the problem may be on your postgres server in some form -- can you check whether the query is hanging on that end?

AndreaCrotti commented 10 years ago

Yes but the thing is that I really don't want it to create the schema, I do that in other ways. Also because now I'm testing a small subset to see if it works, and it would generate a wrong schema..

Anyway I got it working I was just using the wrong collection name since I renamed! Alright then thanks for the answer.