Provide a way to load migrations from a dir at runtime (not embedded)

[akkie]

Is your feature request related to a problem? Please describe.

I only want to run the migrations automatically during integration testing. Currently I have a variable DB_MIGRATIONS_DIR defined which points to my migrations directory which is called surrealdb. I need to set that variable locally, in in the GitHub pipelines.

Describe the solution you'd like I would like to get rid of that variable and use env::current_dir().unwrap().parent().unwrap().join("surrealdb") to get the path to my migrations directory.

[Odonno]

Why not set the path in a configuration file? see https://github.com/Odonno/surrealdb-migrations?tab=readme-ov-file#configuration

[akkie]

I have tried that, but this doesn't work for the integration tests.

[core]
    path: surrealdb

[db]
    address = "ws://surrealdb:8000/"
    username = "root"
    password = "root"
    ns = "app"
    db = "api"

Tried:

let config_file_path = env::current_dir().unwrap().parent().unwrap().join(".surrealdb");

MigrationRunner::new(&db.connection)
    .use_config_file(&config_file_path)
    .up()
    .await
    .expect("Failed to apply migrations");

And

let config_file_path = env::current_dir().unwrap().parent().unwrap().join(".surrealdb");

MigrationRunner::new(&db.connection)
    .up()
    .await
    .expect("Failed to apply migrations");

Both variants fail with Failed to apply migrations: Error listing schemas directory.

Running surrealdb-migrations apply works as expected, with the given config.

[Odonno]

I did this in an integration test file and it works perfectly:

use surrealdb::Result;
use surrealdb_migrations::MigrationRunner;
use surrealdb::engine::any::connect;
use surrealdb::opt::auth::Root;

#[tokio::test]
async fn test_one() -> Result<()> {
    let config_file_path = ".surrealdb";

    let db = connect("ws://localhost:8000").await
    .expect("Failed to connect");

    // Signin as a namespace, database, or root user
    db.signin(Root {
        username: "root",
        password: "root",
    }).await
    .expect("Failed to sign in");

    // Select a specific namespace / database
    db.use_ns("namespace").use_db("database").await
    .expect("Failed to use ns/db");

    MigrationRunner::new(&db)
        .use_config_file(&config_file_path)
        .up()
        .await
        .expect("Failed to apply migrations");

    Ok(())
}

Does it work for you?

[akkie]

Maybe the reason why it does not work for me is:

• I have a deeply nested structure for my integration tests
• I have a testing crate that contains the setup for SurrealDB and which I use in my other crates

I can provide a repo to reproduce the issue if that helps

[Odonno]

Yes, a repro could help.

[akkie]

Here is an example: https://github.com/akkie/surrealdb-migrations-issue

The issue occurs because of the workspace structure. First, I had the project in the root and it worked as expected, also with a nested test folder structure. After I have changed it to the workspace structure, the issue occurred.

[Odonno]

Well, this is because each Rust project is isolated/executed independently. When, you run your tests, the root folder will be app-main and not the root of your workspace.

If you put .surrealdb file and surrealdb folder in the app-main, it will work as expected.

If you want the surrealdb folder to stay in the root workspace. Change the path to ../surrealdb in your .surrealdb file.

[akkie]

I have multiple crate folders that contains tests. So the option would be to put a .surrealdb file in each folder. Changing the path in the file of the root folder isn't an option, because I also use the surrealdb-migrations command to apply migrations to my local DB.

[Odonno]

Yes, you can do that. A global config file in the root folder to apply migrations via cli and a config file on each Rust project for your tests. Does it solve your problem?

[akkie]

Yes, that worked. Thanks!