search

fragmenta / fragmenta-cms

A user-friendly CMS written in Go (golang)
http://fragmenta.eu
MIT License
586 stars 70 forks source link

Error when running fragmenta migrate from root of project #9

Open collinglass opened 8 years ago

collinglass commented 8 years ago

13:20:30 No database found 13:20:30 Running migration 2015-10-28-125605-Create-Database.sql 13:20:30 Running database creation migration: db/migrate/2015-10-28-125605-Create-Database.sql 13:20:30 ERROR loading sql migration:exit status 2 13:20:30 All further migrations cancelled

13:20:30 No migrations to perform at path ./db/migrate

kennygrant commented 8 years ago

Are you using Mysql? There is a known issue with migrations and mysql, which I'm working on a fix for (moving to using drivers rather than calling out to processes).

collinglass commented 8 years ago

No I am using postgres

kennygrant commented 8 years ago

OK thanks, I suspect assumptions are being made about your psql setup which don't hold true then. At present the assumption is you'll be running this locally first to setup and with permission to create databases with your local user - a production server would be set up with a backup seeded locally.

Can you give me the following details:

Does the username running fragmenta have permissions to create a database? If not which user would you use for that?

Can you run the migration yourself manually using another user and psql (it is just an sql file, so this should be easy to verify)?

I'd like to move migrations to using the user specified in the config at secrets/fragmenta.json but of course that user needs to be created in the db first, so there is one step of the bootstrap which the app can't run as the config db user. I may need to adjust the bootstrap process to require manual creation of the database first, but need to think about how best to do that.

FossPrime commented 8 years ago

I'm running this on a vanilla Debian 8.2 server. There seem to be quite a few requirements that aren't mentioned on the installation page, I installed golang, git and postgresql. GOPATH needs to be set ( as in .bashrc ). aswell as adding the $gopath/bin to the PATH.

export PATH=$PATH:$HOME/go/bin
export GOPATH=$HOME/go

Now I'm here.

Does the username running fragmenta have permissions to create a database? If not which user would you use for that?

I just installed it using 'apt-get install postgresql' so I'm assuming there are no users.

Can you run the migration yourself manually using another user and psql (it is just an sql file, so this should be easy to verify)?

Another user such as root? or are we talking another database user?

I'm completely unaware of what fragmenta-cli expects, clearly go and postgresql aren't enough. I'll follow http://www.techrepublic.com/blog/diy-it-guy/diy-a-postgresql-database-server-setup-anyone-can-handle/ and create a new sql user with global permissions, can it have a password?

UPDATE

I su - postgres and set up golang and path there. Doing that made fragmenta migrate work!! However now I get 

postgres@uberserver:/var/www/whip$ fragmenta server
18:46:08 
------
Fragmenta version: 1.3.1
------
18:46:08 Building server...
18:46:08 Running go fmt at ./...
18:46:08 Building server at /var/www/whip/bin/fragmenta-server-local
18:46:08 CMD go build -o /var/www/whip/bin/fragmenta-server-local server.go
server.go:8:2: import "/var/www/whip/src/app": cannot import absolute path
server.go:6:2: cannot find package "github.com/fragmenta/server" in any of:
    /usr/lib/go/src/pkg/github.com/fragmenta/server (from $GOROOT)
    /var/lib/postgresql/go/src/github.com/fragmenta/server (from $GOPATH)
18:46:08 Error running build exit status 1
18:46:08 Error building server: exit status 1

UPDATE #2

This works to make fragmenta migrate work on a vanilla postgres install.

sudo su - postgres
psql
CREATE USER larrypage WITH SUPERUSER;
\q
exit
fragmenta migrate

In summary,

UPDATE #3

I had made my sites in /var/www/ which is obviously not my $GOPATH/src/, strange requirement. We should mention that in docs. I dug that info up in #7

pmkhoa commented 8 years ago

What is the default user the fragmenta use to create database? My postgres listed me (khoapham) as the superuser, and I can manually run the migration script to create the database once I logged into the psql shell using 'khoapham'. But I can't run the fragmenta migration. Is there any way I can debug what user it use to create the database? Thanks!

kennygrant commented 8 years ago

@pmkhoa, for the bootstrap command it uses src/app/bootstrap.go

This invokes psql directly as the user used to run fragmenta, who is assumed to have superuser privs, so it sounds like it should work, however you might have invoked fragmenta as another user?

I have since reconsidered this and would prefer to use the driver to invoke commands, but this is tricky as most people won't give the server permission to set up databases. For now you might be best to just create the database manually with the sql file as you have done - after that migrations should work fine.

I need to look at this again as it is causing people issues when trying to do the initial migration and create the database. Thanks for reporting. I'm going to leave this one open for logging db creation issues.

LordOfTheRams commented 8 years ago

I have the same issue. To me it seems like this is a chicken egg problem. To be able to create a database, you first have to connect to postgres. Usually this database is called template1 in postgres. But the migrate code seems to try to connect to the database that it is creating:

                args := []string{"-d", config["db"], "-f", file}
                if strings.Contains(filename, createDatabaseMigrationName) {
                        args = []string{"-f", file}
                        log.Printf("Running database creation migration: %s", file)
                }

                // Execute this sql file against the database
                result, err := runCommand("psql", args...)

This generates psql -d frag_development -f db/migrate/2016-06-01-114929-Create-Database.sql

This will not work because frag_development does not yet exist. It should be psql -d template1 -f db/migrate/2016-06-01-114929-Create-Database.sql

After doing this migration shoud work. It is probably better to manually create the database and users, because you cannot expect the current user to be database superuser.

kennygrant commented 8 years ago

I agree it is probably a mistake to assume things about the setup of the database itself. This was intended to make the process painless, but makes some assumptions which don't hold true. So it's probably better to adjust the bootstrapping to happen after db creation. I'll look at this soon.

fbaube commented 6 years ago

I got the same error as the OP when trying to use sqlite3 with a new fragmenta 1.5.4 app. Unfortunately the discussion at https://github.com/fragmenta/fragmenta/issues/15 does not describe how to fix it. So, here goes.

1) Add the line import _ "github.com/mattn/go-sqlite3" to the file $GOPATH/src/github.com/fragmenta/fragmenta-app/src/app/setup.go 2) Under your go source tree, run fragmenta new app TheAppsName 3) In the file TheAppsName/secrets/fragmenta.json, change all three instances of "postgres" to "sqlite3" 4) Create an empty file TheAppsName/db/FragmentaTest_development.db 5) Skip the step fragmenta migrate ; just start fragmenta

HTH!

kennygrant commented 6 years ago

Thanks for the instructions fbaube for sqlite. I'm afraid I've neglected that somewhat as I'm using postgresql for all projects - ideally I'd like the simple setup to not require an external db as this is a definite pain point.

The problems with db setup are:

So if you see problems here, please check that your user has the correct permissions, and try running the bootstrap migrations yourself manually to set up the db etc and verify you can run them.

If you're using a different db, you do need to change import statements in order to initialise the drivers.