Failed to apply database migrations: _initial.json file not found in the migrations/definitions directory

[NEmbgen]

Describe the bug When starting the tauri app. The following error occurs.

To Reproduce Steps to reproduce the behavior:

When using the surrealdb_migrations package, I'm getting the error: Failed to apply database migrations: _initial.json file not found in the migrations/definitions directory. This is weird, since this file is neither mentioned or generated by the package itself.

My setup is like this:

const DB_DIR: Dir<'_> = include_dir!("$CARGO_MANIFEST_DIR/database/");

#[derive(Debug, Clone)]
pub struct Database {
    conn: Surreal<Db>,
}

impl Database {
    pub async fn init(path: PathBuf) -> Result<Self, surrealdb::Error> {
        let conn = Surreal::new::<File>(path).await?;

        conn.use_ns("default").use_db("default").await?;

        MigrationRunner::new(&conn).load_files(&DB_DIR)
            .up()
            .await
            .expect("Failed to apply database migrations");

        Ok(Database { conn })
    }
}

Is this a known error?

Expected behavior The migrations run normally

Information Please complete the following information and remove the unnecessary ones.

• SurrealDB version: 1.2.2
• surrealdb-migration version:1.2.2

[Odonno]

Hello @NEmbgen

The _initial.json and all relative json files in definitions folder are generated by the surrealdb-migrations apply command. These files are used to get snapshots of the current state of the schema for each migration which will have 2 notable usage:

• being able to detect if any change is not missing (by comparing the expected schema and the last generated one)
• being able to rollback correctly (to a previous migration)

Because the files are generated on a file system, it is not possible to generate them when used in code like you do. For the error to go away, you will have to apply the migrations manually on your machine with a local database instance.

[NEmbgen]

It still doesn't work sadly. I started a local instance of SurrealDB an pointed my .surrealdb file towards it.

PS C:\Users\[user]\Projects\Software\[project]\src-tauri> surrealdb-migrations apply
Error:
   0: initial_definition_filepath not found at: "./database/migrations/definitions\\_initial.json"
   1: The system cannot find the path specified. (os error 3)

Location:
   C:\Users\[user]\scoop\persist\rustup\.cargo\registry\src\index.crates.io-6f17d22bba15001f\surrealdb-migrations-1.2.2\src\io.rs:676

Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.

Creating an empty _initial.json file in the mentioned folder results in:

EOF while parsing a value at line 1 column 0

or

missing field schemas at line 1 column 2

if I add {} to the file.

Do you maybe have a sample _initial.json file I could modify myself to get over this initial hurdle?

[Odonno]

Ok. I have reproduced this. The creation of _initial.json is fine but the file does not seem to be detected when embedded. Will try to fix this asap.

[Odonno]

Well, it is the extra slash that makes it fail. Can you try this?

const DB_DIR: Dir<'_> = include_dir!("$CARGO_MANIFEST_DIR/database");

[lylejohnson]

I am seeing what I assume is the same error, just using the CLI build of the app.

migrator@00bbe9c5ca0f:/app$ surrealdb-migrations apply
Error:
   0: initial_definition_filepath not found at: "/opt/surrealdb-migrations/migrations/migrations/definitions/_initial.json"
   1: No such file or directory (os error 2)

Location:
   /usr/local/cargo/registry/src/[index.crates.io](http://index.crates.io/)-6f17d22bba15001f/surrealdb-migrations-1.2.3/src/[io.rs:676](http://io.rs:676/)

Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets.

[AlexandreRoba]

I'm also having a lot of difficulties running the migration when it is embedded. I'm getting the same error "Initial definition file not found" but the file is present and the migration run smooth when running from the CLI.

I added some logging:

    tracing::info!("Migrating database");
    tracing::info!("Schema folder:{:?}", DB_DIR);
    let runner = MigrationRunner::new(&db.client);
    let runner = runner.load_files(&DB_DIR);

    runner
        .up()
        .await
        .map_err(|e| format!("Failed to apply migration(s):{}", e.to_string()))

Returns the following:

2024-04-13T08:49:32.536+02:00 [Runtime]  INFO web_server::migrate: Migrating database
2024-04-13T08:49:32.536+02:00 [Runtime]  INFO web_server::migrate: Schema folder:Dir { path: "", entries: [Dir(Dir { path: "events", entries: [] }), Dir(Dir { path: "migrations", entries: [File(File { path: "migrations/20240207_154950_users.surql", contents: "<930 bytes>" }), File(File { path: "migrations/20240207_155525_partners.surql", contents: "<510 bytes>" }), Dir(Dir { path: "migrations/definitions", entries: [File(File { path: "migrations/definitions/_initial.json", contents: "<35 bytes>" })] })] }), Dir(Dir { path: "schemas", entries: [File(File { path: "schemas/director.surql", contents: "<1886 bytes>" }), File(File { path: "schemas/organisation.surql", contents: "<1058 bytes>" }), File(File { path: "schemas/partner.surql", contents: "<597 bytes>" }), File(File { path: "schemas/payslip.surql", contents: "<3457 bytes>" }), File(File { path: "schemas/scope.surql", contents: "<2417 bytes>" }), File(File { path: "schemas/script_migration.surql", contents: "<243 bytes>" }), File(File { path: "schemas/social_tax_payment.surql", contents: "<549 bytes>" }), File(File { path: "schemas/user.surql", contents: "<343 bytes>" })] })] }
thread 'tokio-runtime-worker' panicked at crates/server/web-server/src/main.rs:62:13:
Migration db error
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
2024-04-13T08:49:32.622+02:00 [Runtime] ERROR web_server: Could not migrate db:Failed to apply migration(s):Initial definition file not found
2024-04-13T08:49:32.623+02:00 [Runtime] loading service panicked: Migration db error

We can clearly see the file is present in the Dir.

NOTE: This was with version 1.2.3. I just saw you released 1.3.0. Tried with this one, same error :(. I had to force downgrade my version of surrealdb to 1.3.1 as 1.4.0 creates conflict.

[Odonno]

Sorry guys. It was indeed an issue when trying to detect the last definition file.

This is now fixed in v1.3.1.

[zvictor]

I still get this error on v2.0.0-preview.1 whenever the file schemas/script_migration.surql is not present. If it's mandatory to have this file, it should be better communicated.

surrealdb-migrations apply
Error:
   0: initial_definition_filepath not found at: "migrations/definitions/_initial.json"
   1: No such file or directory (os error 2)

Location:
   src/io.rs:676

Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets.