Any New function returns error "file does not exist" when file actually exists?

[kerak19]

Hello. I have strange error while using migrate.NewWithDatabaseInstance function. When i'm trying to invoke this function, i'm getting error "file does not exist". Problem is, my directory does exist.

Let's say my code does looks like this:

package main

import (
    "database/sql"
    "fmt"
    "log"
    "os"

    "github.com/jinzhu/gorm"
    _ "github.com/jinzhu/gorm/dialects/postgres"
    "github.com/mattes/migrate"
    "github.com/mattes/migrate/database/postgres"
    _ "github.com/mattes/migrate/source/file"
)

func migrations(db *sql.DB) error {
    driver, err := postgres.WithInstance(db, &postgres.Config{})
    if err != nil {
        return err
    }
    m, err := migrate.NewWithDatabaseInstance(
        "file://migrations",
        "postgres", driver)
    if err != nil {
        return err
    }

    return m.Up()
}

func main() {
    db, err := gorm.Open("postgres", "postgres://testuser:testpass@127.0.0.1:5555/testdb?sslmode=disable")
    if err != nil {
        log.Println(err)
        return
    }
    defer db.Close()

    f, err := os.Open("migrations")
    if err != nil {
        log.Println(err)
        return
    }
    stat, err := f.Stat()
    if err != nil {
        log.Println(err)
        return
    }
    fmt.Println(stat.Name())

    err = migrations(db.DB())
    if err != nil && err != migrate.ErrNoChange {
        log.Println(err)
        return
    }
}

ls -la shows this:

lukasz@kerak ~/go/src/github.com/kerak19/migratetest $ ls -la
total 16
drwxrwxr-x  3 lukasz lukasz 4096 Oct 13 21:14 .
drwxrwxr-x 10 lukasz lukasz 4096 Oct 13 21:10 ..
-rw-rw-r--  1 lukasz lukasz  997 Oct 13 21:14 main.go
drwxrwxr-x  2 lukasz lukasz 4096 Oct 13 21:14 migrations

And the output of running this code is:

lukasz@kerak ~/go/src/github.com/kerak19/migratetest $ go run main.go
migrations
2017/10/13 21:14:23 first /home/lukasz/go/src/github.com/kerak19/migratetest/migrations: file does not exist

What can be the cause of this error? Message looks like os.ErrNotExist, but i have no idea why it is returned...

w

[mattes]

there are no migrations in the directory.

[kerak19]

Hey, thanks for quick response. I don't think this error is caused by lack of migrations(there would be ErrNoChange), i've now created migrations using: migrate create -dir=migrations -ext=.sql hello and the error still occurs, even with migrations in directory. I've also filled them with queries, but still the same error.

@edit i've also tried to put absolute path, but with no luck...

lukasz@kerak ~/go/src/github.com/kerak19/migratetest/migrations $ pwd
/home/lukasz/go/src/github.com/kerak19/migratetest/migrations

lukasz@kerak ~/go/src/github.com/kerak19/migratetest $ go run main.go
file:///home/lukasz/go/src/github.com/kerak19/migratetest/migrations
migrations error: file does not exist
exit status 1

._.

@edit2 Alright, i have no idea what's happening, but i've just copied literally this same directory from other project and it works... But unfortunately only this one, every other directory i'm creating(even with sudo) returns me this same error as earlier.

drwxrwxr-x 2 lukasz lukasz     4096 Oct 13 22:08 migrations/
drwxrwxr-x 2 lukasz lukasz     4096 Oct 13 21:23 migrations.bak/

These two directories have absolutely no differences(migrations.bak is the one which works). And now i have no idea if this is some weird error in your package when opening file or it's just Linux playing with me.

[ankita25]

I am seeing the same error. I copied the migrations from a different project, still I see "Migration error: file does not exist". Any help will be appreciated.

[olivere]

Maybe it helps, but I found that this usually occurs when the schema_migrations table contains a version that has no backing file. That might happen when you e.g. switch between a feature branch and the master branch.

If you add and migrate the DB on the feature branch, schema_migrations gets updated. When you now switch back to master, the migration up/down files are gone, but schema_migrations.version is still at the newer version. When started now, migrate will not find the missing migration up/down files.

[chrisdavies]

Getting the same behavior.

[rpeshkov]

Faced the same issue right now. For me problem was quite simple - I didn't provide extension when created migration, however I figured this out only after 30 minutes of debugging the sources :)

What do you think about having default value for file extension when creating migration?

[kerak19]

@rpeshkov By extension do you mean to add .sql at the end of migration file? I haven't tried that, i already 'migrated' to sql-migrate

[rpeshkov]

@kerak19 Exactly. Without this extension migration files are not matching internal regex and as a result, migrate thinks that there are no migrations at all.

https://github.com/mattes/migrate/blob/master/source/parse.go#L18-L21

[haibin]

I'm getting the same error. I did use the extension .sql

[haibin]

I found that the error happens when I have 2 different migration directories. Once I combine them into one directory, I don't see this error anymore.

[17twenty]

$ migrate -path ./migrations/ -database postgres://dev:secret@localhost:5432/postgres?sslmode=disable up
1516166452/u assetledger (12.557438ms)
1516166459/u orderbook (26.277909ms)
1516167278/u initialize (36.312169ms)
$ migrate -path ./migrations/ -database postgres://dev:secret@localhost:5432/postgres?sslmode=disable up
no change
$ rm migrations/1516167278_initialize.*
$ migrate -path ./migrations/ -database postgres://dev:secret@localhost:5432/postgres?sslmode=disable up
error: file does not exist

My fix was trivial, restart my dockerised postgres container. The fix is to run the down scripts. Even if you don't have anything in them as they still create a db entry.

[kevholditch]

I had another version of this error where it worked fine locally when running my tests but when I deployed my program onto a docker container (alpine) I got this error. The migration files were definitely there. I couldn't solve this.