Combine multiple Options for reducing nested code

[linxydmhg]

. (Note, this is not really an issue, more of a question) Hello, so I have been using this library rather imperatively, rather than functionally because Im not too familiar with the concepts, but found good use for Option/Either.

However, I came upon some code that seemed rather unwieldy where I have multiple options that depend on other options and sometimes need to get a future to get the next option, as you can see here.

  return activeShiftOption.fold(
    () => ActiveShiftDetails.error('activeShift none'),
    (activeShift) async {
      final employeeOption = await database.getEmployee(activeShift.employeeId);
      return employeeOption.fold(
        () => ActiveShiftDetails.error('employee none'),
        (employee) async {
          final shiftOption = await database.getShift(activeShift.shiftId);
          return shiftOption.fold(
            () => ActiveShiftDetails.error('shift none'),
            (shift) async {
              final clockOption = await database.getClock(shift.clockIn);
              return clockOption.fold(
                () => ActiveShiftDetails.error('clock none'),
                (clock) async {
                  final checkOption =
                      await database.getCheck(clock.healthCheckId);
                  return checkOption.fold(
                    () => ActiveShiftDetails.error('check none'),
                    (check) async {
                      final userOption =
                          await userDatabase.getUser(clock.registeredById);
                      return userOption.fold(
                        () => ActiveShiftDetails.error('user none'),
                        (user) => ActiveShiftDetails.success(
                          employee: employee,
                          activeShift: activeShift,
                          shift: shift,
                          clockIn: clock,
                          check: check,
                          scannedByUser: user,
                        ),
                      );
                    },
                  );
                },
              );
            },
          );
        },
      );
    },
  );

As you can see there is a nesting of 6 option.folds. Is there some sort of cleaner way to do this with the library? I tried using the Option.map2,3 but there didn't seem to be any type safety. Any guidance/concepts that could help?

[ibcoleman]

Hi @linxydmhg!

I'm by no means an expert on any of this, so take this with a grain of salt, but here's the way I've been approaching things and it's been working pretty well in my current project. First, I think you want to use Either instead of Option, since you're basically trying to capture a multi-step validation process. The basic approach is chain a series of functions together, each returning a value of Either<ErrorCase, T>. If at any step in that pipeline of functions, an ErrorCase is encountered, it short-circuits execution of the pipeline.

Here's the code I came up with based on trial and error and spebbe's helpful response to my earlier head-scratching: (https://github.com/spebbe/dartz/issues/47#issuecomment-636086432).

    // ** THE EVALUATION MONAD **
    // The problem you're bumping up against is that "Monad's don't compose". But 
    // there's a "tool" for that. In Dartz there's what @spebbe has called an über-monad 
    // (it's a Reader Monad, it's a State Monad, it's a Writer Monad, etc...) Don't 
    // worry too much about it right now; suffice it to say it's a way of composing 
    // "pipeline" of monadic effects around Future and Either.
    //
    // The type signature is `EvaluationMonad<E, R, W, S>`, and the type params
    // stand for Error, Reader, Writer, State.
    //
    // So this über-monad, we're going to declare the Error to be ActiveShiftDetailsFailure,
    // the Reader to be Database interface, and the State to be Shift. (We're skipping
    // Writer, so we configure it as Unit)
    //
    // You can read more about Reader Monad elsewhere but from my understanding
    // is to think of it as a version of dependency injection.

    final M = EvaluationMonad<ActiveShiftDetailsFailure, Database, Unit, Unit>(UnitMi);

    // ** DEFINING INDIVIDUAL PIPELINE STEPS **
    //
    // Here we're going to define individual actions so the composed evaluation we're
    // going to define isn't overwhelming. Each one of these uses the EvaluationMonad
    // we declared to declare one stage of our evaluation pipeline.
    //
    // So, for example, the signature of `Database.getEmployee` looks like:
    //     Future<Either<ActiveShiftDetailsFailure, Employee>> getEmployee(employeeId)

    // We use the EvaluationMonad to create an Evaluation on the Database.getEmployee function.
    //
    // That makes the signature of `employeeEv`:
    //    `Evaluation<ActiveShiftDetailsFailure, Database, Unit, Unit, Employee> Function (String, Database)`
    //
    final employeeEv = (String employeeId, Database database) => M.liftFuture(database.getEmployee(employeeId)).bind(M.liftEither);
    final shiftEv = (String shiftId, Database database) => M.liftFuture(database.getShift(shiftId)).bind(M.liftEither);
    final clockEv = (String clockIn, Database database) => M.liftFuture(database.getClock(clockIn)).bind(M.liftEither);
    final checkEv = (String healthCheckId, Database database) => M.liftFuture(database.getCheck(healthCheckId)).bind(M.liftEither);
    final userEv = (String registeredById, Database database) => M.liftFuture(database.getUser(registeredById)).bind(M.liftEither);
    final detailsEv = (Shift shift, Employee employee, Clock clock, User user) => M.liftEither(right(ActiveShiftDetails(shift, employee, clock, user)));

    // ** DEFINING THE EVALUATION PIPELINE **
    // Now that we've got the individual pieces of the pipeline defined, here's our
    // initial Either with the Shift we want to evaluate. We're going to drop it
    // into our pipeline, and if everything goes well, a properly initialized ActiveShiftDetails
    // will come out the other end. This was kind of a mind-fsck for me jumping to FP: the 
    // idea that you're basically defining a "program" that doesn't actually get 
    // executed here, but which you explicitly call later as a kind of program-within-a-program.
    final Evaluation<ActiveShiftDetailsFailure, Database, Unit, Shift, ActiveShiftDetails> eval =
    M.get().bind((shift) => M.ask()
        .bind((database) => employeeEv(shift.employeeId, database)
        .bind((employee) => shiftEv(shift.shiftId, database)
        .bind((Shift newShift) => clockEv(shift.clockIn, database)
        .bind((Clock clock) => checkEv(clock.healthCheckId, database)
        .bind((Check check) => userEv(clock.registeredById, database)
        .bind((User user) => detailsEv(newShift, employee, clock, user))))))));

    // ** THE RUNTIME **
    // Ok, so up till now we haven't actually executed anything. Nothing up to now
    // has any state. You could declare everything previously as static functions.
    // So how do you execute/evaluate the "pipeline"?

    // A couple of different shifts we want to evaluate...
    final Shift aShift = Shift();
    final Shift aDifferentShift = Shift();

    // Here's where the magic happens. The .value() function is how we tell the evaluation
    // to execute. We have to pass in the Reader (i.e. the environment, dependencies, etc...)
    // and we have to pass in the State (i.e. the Shift we're going to evaluate)...
    final successFuture = eval.value(GoodDatabase(), aShift);
    final successE = await successFuture;

    expect(successE.isRight(), isTrue);
    successE.fold((l) => fail("Shouldn't Be Here!"), (r) => expect(r.shift.employeeId, equals("2222")));

    //
    // Now execute the pipeline with a completely different database & initial `Shift` value.
    final Either<ActiveShiftDetailsFailure, ActiveShiftDetails> failedE = await eval.value(
                   DatabaseWithClockFailure(), aDifferentShift);

    expect(failedE.isLeft(), isTrue);
    failedE.fold((l) => expect(l.msg, equals("No Clock Found!!")), (r) => fail("Shouldn't Be Here"));

    /******************************************/
    // Here's our database:
    abstract class Database {
      Future<Either<ActiveShiftDetailsFailure, Employee>> getEmployee(employeeId);
      Future<Either<ActiveShiftDetailsFailure, Shift>> getShift(shiftId);
      Future<Either<ActiveShiftDetailsFailure, Clock>> getClock(clockIn);
      Future<Either<ActiveShiftDetailsFailure, Check>> getCheck(healthCheckId);
      Future<Either<ActiveShiftDetailsFailure, User>> getUser(registeredById);
    }

Wanted to go into more detail here, because this is where things get confusing without some of the syntactic sugar other languages have.

get() lets you access the State. ask() lets you access the Reader. ( Why get, vs ask? I do not know. :) )

    M.get().bind((shift) => M.ask()
            .bind((database) => employeeEv(shift.employeeId, database)
            .bind((employee) => shiftEv(shift.shiftId, database)
            .pure(clockEv(shift.clockIn, database)).flatMap(id)
            .bind((Clock clock) => checkEv(clock.healthCheckId, database)
            .pure(userEv(clock.registeredById, database)).flatMap(id)
            .bind((User user) => detailsEv(shift, employee, clock, user))))));

So super-straightforward to access the State:

    M.get().bind((shift) => doSomeEvaluation(shift.employeeId));

Or to access the Reader context:

    M.ask().bind((database) => doSomeEvaluation(database));

But we want the scope to include both, so we need to nest the binds:

    M.get().bind((shift) =>
        M.ask().bind((database) =>
            employeeEv(shift.employeeId, database)
        )
    );

I realize that, depending on your familiarity with some of the underlying concepts, this can be a lot, and hopefully I've done more to untangle rather than confuse. Let me know if you have any questions! (Also, anyone with more knowledge than me please chime in and set me straight. ;) )

[spebbe]

That was a really great intro to Evaluation, @ibcoleman – thank you!

[ibcoleman]

Means a lot coming from you, @spebbe. Thanks.