Mock when using a transaction (or not)

[hidwulf]

Hi,

As far as we can go we succeeded to mock everything but the Transaction.

Instead of passing the pool we used a top wrapper containing all struct using a db.

I'll try to provide an example of how we did things (feel free to correct, advise us on that too as we are on the learning curve of Rust).

Something like this:

pub struct DatabaseWrapper {
   pg_pool: Arc<PgPool>, // not use anymore except for maybe transaction.
   user_db: Arc<UserDB>,
   order_db: Arc<OrderDB>,
}
pub struct UserDB {
   pool: Arc<PgPool>,
}
trait UserDBFn for UserDB {
 async fn find_by_id(user_id: Uuid) ... 
}

So everything is fine in our services we can mock services and we can 100% mock the *db module containing only SQL stuff.

Example:

impl ServiceAFn for UserService {
  async fun me (user_id: Uuuid, b_wrapper: &DatabaseWrapper) -> Result ... {
   ...
   db_wrapper
      .get_user_db()
      .find_by_id(user_id)
      .await?;
   ...
  }
}

Easy mocked with something like this:

  lert mut user_db_mocked = MockUserDBFn::new()
  user_db_mocked
      .expect_find_by_id()
      .times(1)
      .returning(|_| {None});
  let mut db_wrapper_mocked = MockDatabaseWrapperT::new();
    db_wrapper_mocked
      .expect_get_user_db()
      .return_const(user_db_mocked);

So ok ... a lot's of boilerplate to me way more than we would like to have but we won't fight if it's usual.

The situation is we are facing a wall to do something similar for the transaction.

In a service:

let mut tx = db_wrapper.get_pool().begin().await?;
...
db_wrapper
      .get_user_db()
      .register_user(userStructForCreation, &mut tx).await?
...
tx.commit().await?;

1) So here we really don't know how to mock the transaction part, we tried to create also a wrapper around the transaction but as it is a mutable objet we didn't succeed of writing it ... (we used 'static as we saw it in your source code, tried something else but it doesn't work)

pub struct TransactionWrapper {
  pub tx: Arc<Transaction<'static, Postgres>>,
}
pub trait TransactionHelper {
  fn build<'c>(tx: Arc<Transaction<'static, Postgres>>) -> Self;
  async fn commit(&self) -> Result<(), ApiError>;
}
pub trait DatabaseWrapperT {
 ...
  async fn start_transaction(&self) -> Result<TransactionWrapper, ApiError>; // Use it's pool to create a tx wrapper and returning it
 ...
}

In the service

let transaction_wrapper = db_wrapper.start_transaction().await?;

It's ok for the build/start transaction but ... for the commit we didn't achieved to write something that compile (maybe/mostly) due to the lifetime or this complicated pattern. ==> Is there any way to write it (without discarding automock as we think lifetime cannot be put on trait)

2) I feel like we are on the edge about that, is it really not too overkill ? (in java very easy with injection, in Haskell easier with algebraic effect with Polysemy). We thought it could be a good idea to use Sqlite during test but we are loosing everything we loved sqlx for, query_as! is out and we had to write query like in 2000, parsing/accessing AnyRow manually with index (a nightmare for stability) so this is not an acceptable solution for us.

It seems don't have a lot of options here as we really don't want to start a dedicated db docker for test or using pg_temp.

Anyway thanks for reading this and feel free to provide any feedback or direction even if we have to discard everything I don't mind it will serve the adoption on sqlx in our company.

Regards

NB: we know it's not 100% releveant to sqlx, even if we think providing mock in the api could be interesting, it could be great in the end to have such "full" example in the repo for adoption/help.

[ReDestroyDeR]

Coming from a JVM stack I definetely miss the ability to mock infrastructure layer. It seems that SQLx is by design motivates only integration testing. I would love to hear feedback on this issue from the developers

I've heard that OOP design practices are obsolete in Rust stack, but anyway, is abstracting over traits is such an anti-pattern anyway? I'm used to developing with usage of DDD and Clean-Architecture Tactics. I wasn't able to find any recomendations regarding that style in Rust.

I did find this reply in a neighbour issue

https://github.com/launchbadge/sqlx/issues/1036#issuecomment-774153372

Maybe I'll come to something with it

fyi @launchbadge

Thank you

[bhoudebert]

In the end I really managed to build a sort of structs around it, it was not easy to get it but in the end it is just perfect like anything you would have in java.

What I used a struct wrapper around the DB struct (where you attach trail/impl call to sql), this wrapper has a mirror holding mock db struct. This wrapper must also have to be in charge of handling get_connection and everything related to a transaction.

The test wrapper has obvisouly peculiar mechanics to allow you to trigger different sql case like tx error etc, it also holds the different DB classic Mock with mockall.

Then you use #[cfg(not(test))] and #[cfg(test)] to import the right one.

It is important to notice that you just cannot have a big mock wrapper as it would be totally unusable through the testing phase as when you use Mock, they need to be immutable or implements Clone which destroy a mock context.

Major point is that you can find there and there valid solutions up until you need to use transaction (with commit and rollback), meaning, the struct that is doing sql have an extra parameter which is a connection or a transaction connection. Then, the issue appears outside the db component where you have the tx alongside classic other component, it will just still require a DB connection.

Having this now for more than 2 years in place in all our sqlx rust backends.

NB: if I can I will share it here later.

[ReDestroyDeR]

In the end I really managed to build a sort of structs around it, it was not easy to get it but in the end it is just perfect like anything you would have in java.

What I used a struct wrapper around the DB struct (where you attach trail/impl call to sql), this wrapper has a mirror holding mock db struct. This wrapper must also have to be in charge of handling get_connection and everything related to a transaction.

The test wrapper has obvisouly peculiar mechanics to allow you to trigger different sql case like tx error etc, it also holds the different DB classic Mock with mockall.

Then you use #[cfg(not(test))] and #[cfg(test)] to import the right one.

It is important to notice that you just cannot have a big mock wrapper as it would be totally unusable through the testing phase as when you use Mock, they need to be immutable or implements Clone which destroy a mock context.

Major point is that you can find there and there valid solutions up until you need to use transaction (with commit and rollback), meaning, the struct that is doing sql have an extra parameter which is a connection or a transaction connection. Then, the issue appears outside the db component where you have the tx alongside classic other component, it will just still require a DB connection.

Having this now for more than 2 years in place in all our sqlx rust backends.

NB: if I can I will share it here later.

I am currently using SQLx in my study project. I managed to split details of working with transcations in domain code via writing some basic traits.

This implementation will allow me to write unit-tests without any pain of spinning up a test database in a container for sole purpose of testing business rules

https://github.com/ReDestroyDeR/otus-highload-social-network/blob/d30298b23c225de0b03fce812177be0501ad5edf/src/pool.rs

I am only unsure of the lifetime that is present on transaction. All the pool methods that are creating transactions seem to have elided or 'static lifetimes. I couldn't yet figure that part out and for the sqlx::Pool<DB: sqlx::Database> implementation I did put a 'static lifetime. For my purpose it seems to be enough.

For production projects at work I am yet unsure if I ever would want to use this library. To do simple unit testing I would have to do so much infrastructural work. It honestly feels like a chore.