Flips 3.0 Design Proposal

[smoothdeveloper]

Thinking some more about this, it would make sense to split the library into the domain / types parts (everything to build a Model object) and then have separate libraries for each solver which implement translation of Model back and forth with specific solver backends and provide the same simplified API as currently in the Solver module.

Using discriminated unions for the solver type and putting it in the Settings record is not the right approach.

The split between the three stages would remain :

• formulation
• runtime
• result extraction

but the runtime part would be opened up, while remaining simple for the simplest use cases:

// formulation
let formulation : Model =
   Model.create ...

// runtime
let solver = Flips.Solver.GoogleOrTools.Solver.createSolver Flips.Solver.GoogleOrTools.SolverSettings.basic
let result = Solver.solve formulation solver 

// result extraction
match result.Status with
| Optimal ->
  result.Decisions
  result.Objectives
| _ -> ...

For the specific solver settings, studying how Feliz model typed DSL over HTML format in a way which is convenient to use.

I think such API would benefit more advanced use case, also the benefit of splitting the assemblies / nuget packages, then you can reference only the model in projects dealing with formulation and handling the results back, and break down the dependencies in solver specific packages into Flips.Solver.* runtime packages.

Proposed assemblies:

• Flips everything up to Model
• Flips.Solver the abstraction defining the SolveResult and interface for running the solver with settings and mapping the results back to the Model
• Flips.Solver.GoogleOrTools backend using google-ortools, implementing the interfaces in Flips.Solver
• ...

[matthewcrews]

I really like this direction. I’d like to think about this and put together some examples of what the code would look like with the new layout. I want the “ergonomics” of the library to be really clean and composeable. Modeling SolverType as a DU was a mistake.

The other thing I keep top of mind is simplicity for a new developer. I’m wanting to keep simple things simple while enabling more complex business scenarios.

Would we want to include a default solver? Would the user need to download a single library or one for the base and one for the solver they want to use?

[matthewcrews]

Another thing I’m noting for my own sake. Exposing the Settings type was a mistake. Any user that builds that type using the Record construction syntax will have their code break if we add a field. Didn’t see that coming but needs to be fixed so we can continually add features without breaking code.

[smoothdeveloper]

@matthewcrews thanks for the feedback.

I’m wanting to keep simple things simple while enabling more complex business scenarios.

We can make sure of preserving the simplicity for end users, which is one highlight of this library and a fundamental design principle.

It would boil down to opening a new namespace, the same way usage of Flips.UnitsOfMeasure happens, the namespace would be the one bringing up the easy to use Solver module of a specific backend.

There is some attention to be put in case code wants to use several solver backends, so that the types/modules don't get name-erased but I think this would be viable route for the API to remain almost identical.

We can also follow consistent naming conventions to avoid name-erasing:

open Flips.Solver
open Flips.Solver.GoogleOrTools.Solver
open Flips.Solver.Optano.Solver
let ortoolsSolver = Solver.createWithGoogleOrToolsSettings SolverSettings.GoogleOrTools.basic
let optanoSolver = Solver.createWithOptanoSettings SolverSettings.Optano.basic

There may be some redundancy in the implementation of those backend specific modules, but it is implementation detail and can be ironed out after maintaining the specific backends on their own.

Would we want to include a default solver?

I would say no, some reasons:

• heavy weight dependency to bring all of the supported backends (especially in script references)
• in projects, transitive dependencies get pulled automatically via nuget or paket, if I reference Flips.Solver.GoogleOrTools it will pull all the right things to be ready to run.

We could even provide a compatibility solver module which implements the current way it is done.

[matthewcrews]

The more I think about this, the more I think the modeling of the problem and the solving of the problem should be made distinct. The implication is that the Solution type is not defined in Flips but instead in a different library. Flips.Solver for example. Flips.Solver is just a tiny project which declares the basic ISolver interface and the Solution type.

The Flips.Solver.Legacy project declares equivalent types to the old Flips types. The newer solvers can diverge and provide more bespoke instances.

[smoothdeveloper]

@matthewcrews I concur with your analysis.

Your approach is "deffer design decision" and Flips at it's core is excellent library to express the model and run it to get one shape of solution object with minimum moving parts.

The only public thing I see for now as identical in each solver implementation is a datastructure to map from solver backend variables, constraints, objectives types to those from Flips and the other way around.

The particular service done by this datastructure should roughly have the minimal contract of what the Solution module was doing.

Then we can have an implementation of Solution module which works with the shared common denominator solution which is object storing the model and with a mapping of last results from the solver.

There will be a bit of variation on how the solution object is defined and how the solver is handling the processing for each backend but we should still be able to generalize the "visiting of the model" part to ease the work of making a naive Flips model -> Solver backend implementation.

[smoothdeveloper]

For consideration also, it would be good to allow client code to define their own "polymorphic" Decision and LinearExpression objects, the reason it is useful is that you can have your own type storing extra domain / application specific information without dealing with an extra mapping in your application logic.

type MyVariable(lb, ub, name, myDomainData) =
  interface IDecision with
     member x.Type = Continous
     // ...

  member x.UserFriendlyName = $"the amount of {myDomainData.SomethingName} per {name} produced"

Does that make sense?

[matthewcrews]

Does that make sense?

Interesting. Yes, that does make sense. I'm thinking through the design implications as I look forward. Per our discussions, my mental model divides the process of optimization into the following activities:

• Build/Update the Model
• Solve/Resolve the Model
• Inspect the Solution

I say Build/Update and Solve/Resolve because this process can actually be a loop. You create an initial model, solve it, inspect the solution, update the model, and resolve. That is exactly what the Multi-Objective Function feature actually does, it's just hidden from the user.

I see the potential that would be enabled by defining interfaces that client code could implement. I'm curious if all the operators would work though. A lot of the code in Flips is just defining operators for how types interact (+, -, ==, <==, >==). I think seeing use cases would help guide that design.

I like the idea of keeping the stages and responsibilities separate and clean. The Flips library is about modeling the problem. The Solvers are for solving a model and inspecting the solution. There is a minimal ISolver interface which governs the base capabilities.

Something that needs to be addressed though is being able to update an existing model and re-solve. I have not thought about that other than for the Multi-Objective Function feature. It's not possible to remove or update a constraint at this time. That's not good if you want to support an iterative solve workflow. Is it something we want to support? Is it something we want to make simple? Right now a Model has a list of Constraint and that's really simple to work with. Needing to index into that would get ugly. What would the indexer for the Constraint even be?

I'm not sure this is something at this moment but it is something that I am mulling. Right now I like the simplicity of having a Solution which you use as a parameter to evaluate the results of LinearExpression or Objective. A Solution really is just the values of the decision variables the solver found. Another useful thing would be to surface the Shadow Prices of constraints as well.

[matthewcrews]

I've been working on separating concerns between Flips and Flips.Solver. I like the idea of defining interfaces for IDecision, ILinearExpression, IConstraint, and IModel. I see how it enables a lot of flexibility going forward. One concern I have going forward though is the reduction of LinearExpression. float math can get messy and the LinearExpression.Reduce function goes to a lot of effort to make float math behave deterministically. These kind of bugs can be really difficult to identify.

The place where this really matters is in testing for equality of LinearExpression. If the float math is not done properly, you can get some bizarre behaviors.

One of the challenges is what will the interface for ILinearExpression be? What would a Solver instance expect? Right now the code performs the reduction of the LinearExpression to a reduced form which performs the reduction in coefficients to that each Decision only has one coefficient. Is that something we would want so ensure in the ILinearExpression contract? I like the idea of lifting the abstraction into an interface.

[smoothdeveloper]

One of the challenges is what will the interface for ILinearExpression be?

Since it is unclear, if needed or to give a look, the first step would be to define it with same abilities as LinearExpression has, and seeing what happens.

For example, I've started to explore a bit defining interface for CPLEX settings, and on top of a "rigid" DU that I'm just using for now:

https://github.com/matthewcrews/flips/blob/340c2f0892e1e159ac05391508253001797f637e/Flips.Solver.Cplex/CplexSolver.fs#L159-L178

What would a Solver instance expect?

I recommend we start with what Flips define, the concrete Model and everything in it, later on we can bring interfaces with remaining compatible (so long Flips concrete types define it) if need be.

The solution object could be an interface with lookup function to get a decision by it's name, or an array of those, to help performing routine operations a model may do.

type ISolution =
  abstract GetDecisionValue : decision: DecisionName -> float
  abstract GetExpressionValue : expr: LinearExpression -> float
  abstract GetDecisionValueRange : decisions: DecisionName array -> float array
  abstract FillWithDecisionValues : decisions: DecisionName array -> buffer: float array -> startIndex: int -> unit

Flips.Solver may also define a simple implementation of it taking IReadOnlyDictionary<DecisionName,float> as input.

Those are just some proposals.

[TysonMN]

In the dev branch, I see a lot of InternalsVisibleTo being used to expose things to downstream projects. I don't recommend releasing like this. One reason is that it puts potential additional solvers outside this solution at a disadvantage because they have access to a smaller API.

Maybe this is just a temporary thing while lots of code is being moved around.

[matthewcrews]

Completely agreed. It was temporary thing to unblock some work until new interfaces were defined.

[smoothdeveloper]

@TysonMN also concur, it is temporary and I'd recommend just using internal namespace with things being public while things are in flux.

@matthewcrews, I think it would be good to gather an action plan to streamline the work on dev on the different areas. I'm myself not pushing on the external surface of the cplex solver API due to waiting where we are taking the breaking down of the API.

I think if more people can contribute bindings to solvers in "work in progress" mode in dev branch, it will give us more foot print to evaluate the things we can abstract or should leave on the solver side.

Similarly about the client code which defines the model and uses Flips, we should also introduce code which looks more like real world in term of size and workflow from the model to the domain output, even if just to get a better feel of what changes in the API imply against such code.

[TysonMN]

Proposed assemblies:

• Flips everything up to Model
• Flips.Solver the abstraction defining the SolveResult and interface for running the solver with settings and mapping the results back to the Model
• Flips.Solver.GoogleOrTools backend using google-ortools, implementing the interfaces in Flips.Solver
• ...

Even being new to Flips, I immediately see the benefit of having packages like Flips.Solver.GoogleOrTools. On the other hand, I don't see the advantage of separating Flips and Flips.Solver.

Would we want to include a default solver?

I would say no...

I agree with @smoothdeveloper.

As an example for comparison, I really like the logging library Serilog. However, almost every user of Serilog also depends on some sink, which is the term used by Serilog to describe the location where log events are sent. Want to log your events to the console? Then you should use Serilog.Sinks.Console. Want to log your events to a file? Then you should use Serilog.Sinks.File. Here is a list of sinks for Serilog. Many of them were created by the creator of Serilog, but some of them were created by others. My favorite such example is Serilog.Sinks.XUnit.

I think that is what you hope the future will look like for this library. The core Flips code is independent of a particular solver, some solver-specific packages are created in-house, and others in the community eventually create additional solver-specific packages.

As another comparison, there is no NuGet package called Serilog.Sinks. Instead, the interface for sinks is in the Serilog NuGet package. In the same way, I expect everything currently in Flips.Solver in the dev branch to also be in Flips.

I like the idea of defining interfaces for IDecision, ILinearExpression, IConstraint, and IModel. I see how it enables a lot of flexibility going forward.

I don't see that, but of course I am new to Flips. Would it be reasonable to first split out the solvers into their own NuGet packages and only then consider the additional advantage gained by adding any of these four interfaces?

One concern I have going forward though is the reduction of LinearExpression. float math can get messy and the LinearExpression.Reduce function goes to a lot of effort to make float math behave deterministically. These kind of bugs can be really difficult to identify.

The place where this really matters is in testing for equality of LinearExpression. If the float math is not done properly, you can get some bizarre behaviors.

That is what #141 is about. My guess is that there are values for which the properties under test fail because of rounding during floating point arithmetic but those values are either outside of the range currently being considered by the tests or have yet to be found by the tests.

type ISolution =
 ...
  abstract FillWithDecisionValues : decisions: DecisionName array -> buffer: float array -> startIndex: int -> unit

Is this mutation crucial for performance? If not, functional style would be to return another ISolution and make the behavior pure.

abstract FillWithDecisionValues : decisions: DecisionName array -> buffer: float array -> startIndex: int -> ISolution

[okkehattu]

I'm not sure this is something at this moment but it is something that I am mulling. Right now I like the simplicity of having a Solution which you use as a parameter to evaluate the results of LinearExpression or Objective. A Solution really is just the values of the decision variables the solver found. Another useful thing would be to surface the Shadow Prices of constraints as well.

I was thinking of adding an issue with a feature proposal, but I saw you had considered this already here. The shadow prices would be crucial for most of my use cases, so they would be really appreciated.

Something that needs to be addressed though is being able to update an existing model and re-solve. I have not thought about that other than for the Multi-Objective Function feature. It's not possible to remove or update a constraint at this time. That's not good if you want to support an iterative solve workflow. Is it something we want to support? Is it something we want to make simple? Right now a Model has a list of Constraint and that's really simple to work with. Needing to index into that would get ugly. What would the indexer for the Constraint even be?

Iterating a solution using an existing model by changing an individual parameter / coefficient might be useful as well, but that might require more drastic changes.

Just my three cents. Thanks for the effort so far, the planned updates seem promising!

[matthewcrews]

@okkehattu thank you for the thoughts. When I started this work the documentation for the GLOPS solver was sparse for .NET which is why Shadow Prices were not included. There's also the problem of the access pattern. A Shadow Price is associated with a Constraint and there's not real concept of a indexed collection of constraints.

Could you provide some example use cases of what you would like to do? That would help guide the design discussion. Flips was not originally designed with iterative solve scenarios in mind. Right now you need to rebuild/resolve the model each time. That's adequate for my uses cases but I am only one perspective.

[matthewcrews]

Even being new to Flips, I immediately see the benefit of having packages like Flips.Solver.GoogleOrTools. On the other hand, I don't see the advantage of separating Flips and Flips.Solver.

I'm wrestling with this same point. For now it's two different projects. It would be relatively simple to merge them. Any solver backend though would likely include Flips.Solver as a dependency so you would likely get it for free anyway.

[smoothdeveloper]

On the other hand, I don't see the advantage of separating Flips and Flips.Solver.

I expect everything currently in Flips.Solver in the dev branch to also be in Flips.

In my initial comment, I think I made that split for some of those reasons:

• the model and numerical parts of Flips are already quite solid and will probably be updated less frequently
• I expect most of the upcoming "involved code additions" to revolve around the "runtime" part, which match with Solver kind of activities, knowing there will be several backend, it made sense to have a place to accrete code that helps implementing the solvers, but those concerns are orthogonal to the other parts
• from client code perspective, it is nice option to be able to split the "formulation" and "extraction" parts from the "runtime/solving" one even in terms of dependencies
• as @matthewcrews says, when you use dependency manager to pull a solver, it doesn't really matter that the assembly is separate, it will bring both Flips and Flips.Solver

I don't really see any drawback with all those elements.

A Shadow Price is associated with a Constraint and there's not real concept of a indexed collection of constraints.

This brings back the requirement to store them in structure using the hashcode

Could you provide some example use cases of what you would like to do? That would help guide the design discussion. Flips was not originally designed with iterative solve scenarios in mind. Right now you need to rebuild/resolve the model each time. That's adequate for my uses cases but I am only one perspective.

Hopefully @okkehattu will have some ideas to share around the iterative solving approach, I'm also interested in that area in terms of extensibility, I don't remember fine details of models I've seen using those techniques, but it was looking a bit like that

• solve problem with smaller set of fundamental constraints
• check all the extra constraints failed, and depending the logic, adjust the decision in the formulation, or add more constraints
• loop until reaching solved in terms of both the solver and domain criteria

I know the shadow price is also used in my domain, having some of those concepts surfacing in Flips and a bit of documentation around the topic would be great.

[matthewcrews]

I am in agreement with @smoothdeveloper but I'm open to this being a discussion. I think the library/package separation really needs to be driven by real world use cases. I like the "simplicity" of having a package that is all about problem/model formulation and separate ones for interpreting the model. I think core Flips has done this fairly well at easing model formulation.

We still face a problem of wanting to remove/update/add an existing constraint/objective and re-solving. This gets a little complicated though because you want to update the Solver Model Instance as well, not just the Flips model. The Solver Model Instance is a separate representation from the Filps model. I don't have a good answer here. I think some real world use cases will really clarify what we want to be able to do.

What would you want to index a collection of constraints by? Do you just need the hashcode? That seems kind of clunky to me, but I don't know.

I think returning Shadow Prices can be returned via a function which takes a Solution as an input. For example:

let shadowPrice = Constraint.shadowPrice solution constraint

I like the idea of exposing additional functionality as functions because it makes extensibility going forward easier. F# really encourages the separation of types from functions that operate on those types and I tend to favor that. Again, there's are opinions but I'm open to being corrected with real world use cases.

[smoothdeveloper]

I'm open to this being a discussion.

Just to be clear, with what I stated, I also look for having more insight on the motives of split or keeping Flips.Solver together from others' perspective.

For now, it only felt related to nuget publishing and consumption, and in current dotnet eco system, I don't see the separate assembly/package as a problem.

The points about versioning also didn't come the time of the top comment of this issue, but in me looking at comments / inquiries about those considerations.

This gets a little complicated though because you want to update the Solver Model Instance as well, not just the Flips model. The Solver Model Instance is a separate representation from the Filps model. I don't have a good answer here. I think some real world use cases will really clarify what we want to be able to do.

@matthewcrews sketching few ideas around it:

We could develop a DSL for updating a "live" model, it would have a grammar of operations as a DU, that could be used to update the decision boundaries, add / remove constraints.

It could maybe even fit in the Model algebra of functions that exists right now and remain coherent.

The Solver Model Instance is a separate representation from the Filps model. I don't have a good answer here. I think some real world use cases will really clarify what we want to be able to do.

In the CPLEX backend, for now I have this:

https://github.com/fslaborg/flips/blob/e54f8c5211677ed874d0f925b17011bf4d4aa692/Flips.Solver.Cplex/CplexSolver.fs#L389-L397

this allows to potentially implement those operations, at least on the cplex side.

What would you want to index a collection of constraints by?

I may have misunderstood, but it felt that end result of extracting shadow prices would conceptually be a Map<Constraint,float>, or a dictionary, and that there was use for Constraint as key.

But for your questions, I don't think the questions that involve "what should we index X by?" are to be solved in Flips, because those decisions pertain to what makes sense in context of the client code. The question applies to the internal implementation details of course, but not so much for the public API surface.

I think returning Shadow Prices can be returned via a function which takes a Solution as an input. For example:

+1, this looks consistent to how extraction of decision values is done

I like the idea of exposing additional functionality as functions because it makes extensibility going forward easier. F# really encourages the separation of types from functions that operate on those types and I tend to favor that. Again, there's are opinions but I'm open to being corrected with real world use cases.

+1

[TysonMN]

I agree with @smoothdeveloper; I don't see any significant downside to separate Flips and Flips.Solver projects. It just isn't the choice that my intuition says is the right one at this stage of refactoring. I will keep this separation in mind as the refactoring continues.

[okkehattu]

Could you provide some example use cases of what you would like to do? That would help guide the design discussion. Flips was not originally designed with iterative solve scenarios in mind. Right now you need to rebuild/resolve the model each time. That's adequate for my uses cases but I am only one perspective.

I don't have a use case at this point. Could be useful for performance (or convenience) reasons, but I'm at least not seeing that as a problem so far and the iteration can easily be done on a higher level on the application side.

[matthewcrews]

Something that I started to formulate in my head is a way to support the iterative workflow of:

1. Build initial Model
2. Solve Model
3. Evaluate Solution
   • If solution good
     • Done
   • If not
     • Update model, return to 2

This would be a really useful way of iteratively exploring a model. Now, here's the problem, how do you support that?

Right now the Model type is a simple record with two fields:

type Model = 
    internal {
        Constraints : IConstraint list
        Objectives : IObjective list
    }

I can think of at least two approaches.

Brute Force

Change the Constraints field to be an indexed collection and have the index be the Constraint Name. Then add a function: Model.updateConstraint which would change the value the index is bound to and return a new Model. That's simple but here's a big problem, how do you make it efficient to update the underlying model in the Solver with the updated constraint? Do you you just have to build an entirely new model? If you want to incrementally change it, how do you keep track of what changed since you last solved it? Technically that would work but it would be a poor experience and unusable in the situations I'm thinking about.

Model becomes a diff list

Instead of having the Model type be a record, have it be a list of diffs. The final state of the model is just a fold over the changes to the initial model. Each change that is performed could be provided a version number that is just an incrementing int. I imagine it as something like this:

type ChangeIndex = ChangeIndex of int

type ModelChange =
    | AddObjective of Objective
    | AddConstraint of Constraint

type Model = {
    Name : string
    Changes : (ChangeIndex * ModelChange) list
}

Now when you go so solve a model, the solution you get back includes the model and the underlying Solver Model which would depend on the library.

type Solution = {
    DecisionValues : Map<Decision, float>
    Model : Model
    SolverModel : SolverModel
}

You can look at the first element of Solutuion.Model.Changes to get the last version number. Now, if you update or add constraints to the Model, you can instead call Solver.reSolve

let reSolve settings solution model =
    // The first thing you would do is look for the updates in `model` compared to the version in `solution`
    // and make the appropriate updates to the solver model

This a rough idea and I think it is Solver dependent. I know that CBC, GLOPS, and Gurobi can do this. I'm betting CPLEX can as well. You definitely don't want to use a List to store the changes in the Model type because you will want to traverse up and down the values. I'm not sure if there is an efficient way to do that but I haven't looked.

Also, when building the model you want to start with the initial change and then work forward in time. I'm sure there's a data structure for this. I just don't know if off the to of my head.

[TysonMN]

I can help with making things efficient including the use of good data structures. You and @smoothdeveloper can focus on converting a change in a Flips model to a change in a solver model for each type of solver.

[matthewcrews]

I guess my main concern when it comes to the data structures is that Flips adheres pretty closely to F# norms so immutability (or the perception of immutability, persistent data structures "cheat") is a key concern I have.

I also think that there should be different function for update* vs add*. An add* assumes that the thing being added was not a part of the model yet. An update* would check that it already existed. The question then becomes, "Do we validate at the point we update or do we wait until we call Solver.reSolve? My preference would be that errors are raised as close to when they occur as reasonable.

[TysonMN]

Part of functional programming is about preferring immutablity over mutablity, but not shunning it completely. Most of the time, immutablity suffices. If you have a use case that significantly benefits from mutablity, then it is ok to use it.

I have mixed feelings about separating vs combing add and update. I feel like separation is the default choice. Maybe callers want to fail when their function of choice is not possible. On the other hand, combing them yields an idemponent function, which is a great property for a function to have. I suspect the mixed feelings result from separation being correct in some cases and combining being correct in others.

[matthewcrews]

I suspect the mixed feelings result from separation being correct in some cases and combining being correct in others.

I too like the idea of having an idempotent function but I also think a lot about users making unintended mistakes and the hours of debugging. If you only have add*, you will never get an error when you are trying to update an existing constraint. That could be really dangerous because the math will likely still work. As a user, I would rather fail early.

Also, conceptually "adding" a constraint to actually update a constraint feels odd. I'd almost rather have a different word, like set, which suggests either adding or updating.

The downside with using set* is that conceptually it doesn't make sense with Objectives. The order that Objectives are added to a Model will dictate their priority for sequential optimization.

Another approach would have it be purely a functionality of the Solver library. Have reSolve not take a model. Have it just take the Solution and the type you want to update.

module Solution =

    module Constraint =

        let update (solution: Solution) (constraint: Constraint) = // ...

And then reSolve becomes part of the Solution module.

module Solution =

    let reSolve settings solution = // ...

Not sure I love that, but I think it's important we explore the solution space.

[smoothdeveloper]

@matthewcrews wrt to updating a model with different constraints, I think we should defer a bit on the solver agnostic API for those scenarios, until we play more with the underlying solvers.

I kind of remember I was looking into this in context of CPLEX, few excerpts with relevant conditionals as to update values that have been fed already to an instance:

https://github.com/fslaborg/flips/blob/0275069059435745b52ff5173f5ad3802bf4b376/Flips.Solver.Cplex/CplexSolver.fs#L66-L76

https://github.com/fslaborg/flips/blob/0275069059435745b52ff5173f5ad3802bf4b376/Flips.Solver.Cplex/CplexSolver.fs#L136-L148

https://github.com/fslaborg/flips/blob/0275069059435745b52ff5173f5ad3802bf4b376/Flips.Solver.Cplex/CplexSolver.fs#L19-L29

We can expect each solver to have their idiosyncrasies (or even idioties 🙂) and trying to tackle those concerns at the Flips main API level upfront is going to be untenable if we haven't played more with the backend implementations and came back with a good summary of the issues faced, and solid way to handle it even without Flips (HasDoNotReuseState in code above would be sane default with the rest remaining largely "undefined behaviour" for now).

Those are the areas where we will hit edge cases, hard to figure out bugs, etc. they will be "use at your own risk" for a while, and hence don't need to surface at the main API level, for now, IMO.

Following this, the same would apply for iteration logic, this stuff seems to be highly domain specific and will be hard to figure out, we can rely on documentation to touch on the matter until some great ideas emerge out of real world usages.

All that aside, I do like the idea of "model or solver state is a list of diff", kind of like event sourcing, but I'm not sure it is the best nor simplest we would provide for the main Model object as the default approach, this could be separate "debug audit trail" concern.

We should ideally aim to get Flips 3 with simplest split of solver backend implementation in the packaging and defining the few interfaces/asbstractions, tailored only at retaining the simple API for simple use cases, while having few escape hatches to handle with solver backend in proprietary fashion for advanced use case (see the cplex callback stuff I was experimenting a bit with).

And then learn from that and users feedback to extend the use case footprint in an even later version of the library.

[matthewcrews]

I'm closing this as there is now a Discussions area for this.