Stellar looking for wrong database

[twolfson]

I was having trouble adding stellar to an internal project with respect to database connections being severed upon restore. I created a proof of concept to demonstrate the issue but have uncovered another one first.

The proof of concept can be found here:

https://gist.github.com/twolfson/d68a33c2d2dca3732691

Expected behavior: stellar should backup our database test successfully

Actual behavior: stellar is complaining that it cannot find database stellar

[twolfson]

The entire traceback is:

Traceback (most recent call last):
  File "/home/vagrant/env/bin/stellar", line 9, in <module>
    load_entry_point('stellar==0.4.1', 'console_scripts', 'stellar')()
  File "/home/vagrant/env/local/lib/python2.7/site-packages/stellar/command.py", line 264, in main
    stellar()
  File "/home/vagrant/env/local/lib/python2.7/site-packages/click/core.py", line 572, in __call__
    return self.main(*args, **kwargs)
  File "/home/vagrant/env/local/lib/python2.7/site-packages/click/core.py", line 552, in main
    rv = self.invoke(ctx)
  File "/home/vagrant/env/local/lib/python2.7/site-packages/click/core.py", line 893, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/vagrant/env/local/lib/python2.7/site-packages/click/core.py", line 744, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/vagrant/env/local/lib/python2.7/site-packages/click/core.py", line 388, in invoke
    return callback(*args, **kwargs)
  File "/home/vagrant/env/local/lib/python2.7/site-packages/stellar/command.py", line 60, in snapshot
    app = get_app()
  File "/home/vagrant/env/local/lib/python2.7/site-packages/stellar/command.py", line 28, in get_app
    app = Stellar()
  File "/home/vagrant/env/local/lib/python2.7/site-packages/stellar/app.py", line 45, in __init__
    self.init_database()
  File "/home/vagrant/env/local/lib/python2.7/site-packages/stellar/app.py", line 53, in init_database
    self.raw_conn = self.raw_db.connect()
  File "/home/vagrant/env/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1717, in connect
    return self._connection_cls(self, **kwargs)
  File "/home/vagrant/env/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 59, in __init__
    self.__connection = connection or engine.raw_connection()
  File "/home/vagrant/env/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1786, in raw_connection
    return self.pool.unique_connection()
  File "/home/vagrant/env/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 273, in unique_connection
    return _ConnectionFairy._checkout(self)
  File "/home/vagrant/env/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 630, in _checkout
    fairy = _ConnectionRecord.checkout(pool)
  File "/home/vagrant/env/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 433, in checkout
    rec = pool._do_get()
  File "/home/vagrant/env/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 949, in _do_get
    return self._create_connection()
  File "/home/vagrant/env/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 278, in _create_connection
    return _ConnectionRecord(self)
  File "/home/vagrant/env/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 404, in __init__
    self.connection = self.__connect()
  File "/home/vagrant/env/local/lib/python2.7/site-packages/sqlalchemy/pool.py", line 530, in __connect
    connection = self.__pool._creator()
  File "/home/vagrant/env/local/lib/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 95, in connect
    connection_invalidated=invalidated
  File "/home/vagrant/env/local/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 189, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb)
  File "/home/vagrant/env/local/lib/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 89, in connect
    return dialect.connect(*cargs, **cparams)
  File "/home/vagrant/env/local/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 376, in connect
    return self.dbapi.connect(*cargs, **cparams)
  File "/home/vagrant/env/local/lib/python2.7/site-packages/psycopg2/__init__.py", line 164, in connect
    conn = _connect(dsn, connection_factory=connection_factory, async=async)
sqlalchemy.exc.OperationalError: (OperationalError) FATAL:  database "stellar" does not exist
 None None

[quantus]

It seems like the connection to url postgresql://stellar:stellar@localhost:5432/made connection to database named stellar that didn't exist. I myself didn't run into this problem because I never defined the username or password in my config. If you change url value in stellar.yaml to postgresql://stellar:stellar@localhost:5432/template1 everything should work. The url is only used for getting raw connection to the database application and it shouldn't make any changes to template1 database.

I made change to init command to automatically add that template1 part to url when using postgresql.

[twolfson]

Everything looks great now =)

I have completed the gist to "test" PostgreSQL backups/restores.

https://gist.github.com/twolfson/d68a33c2d2dca3732691

The originally referenced commit in the gist is:

https://gist.github.com/twolfson/d68a33c2d2dca3732691/41fc80b3fac7542e0602b7b37ba8b75c32978fe8

[Teemu]

Your vagrant stuff is interesting and I wonder if that could be used to provide proper integration testing. As of now, we are relaying mostly on ad-hoc testing.

[twolfson]

The scripts should be reusable between Vagrant and Travis CI. I use a similar pattern for foundry plugins to encapsulate anything that could pollute the global environment.

https://github.com/twolfson/foundry-release-git/blob/1.1.0/package.json#L29

https://github.com/twolfson/foundry-release-git/blob/1.1.0/test/foundry-release-git_test.js#L82-L85