A File-format for MathOpt

[odow]

Here's a potentially naive question (considering I know nothing about conic):

Is it possible to save any problem (based on the current standard form with linear-quadratic functions, and the current sets) in the CBF format?

Because I was reading through the specification (incl. the new one http://cblib.zib.de/doc/format2.pdf) and it wasn't obvious to me how to have quadratic objectives/quadratic terms in the constraints.

It seems like if this is to be the standard form that unifies linear/conic etc, it would be nice to have one simple, easy to parse file-format that everybody agrees upon (unlike LP and MPS where nobody agrees on a canonical format and each reader/writer has weird limitations).

[chriscoey]

It's not possible with CBF, no. It was designed for LP/SOCP/SDP storage, and later was extended to also allow exponential cones (3 dimensional: z >= y exp(x/y)). It is affine expression in cone only, and Henrik will likely want to keep it conservative like that.

It is worth starting a new issue to discuss a storage format. I don't think it's high on the list of priorities but if there is a simple solution then perhaps it should be.

[odow]

Excellent. This is high on my list of priorities, but I didn't want to JuMP (if you will) the gun if the CBF can already do it.

[mlubin]

CBF is a good conceptual starting point, but it's designed for conic and doesn't support quadratic terms, as @chriscoey explained. I'd argue for keeping things simple to start out, and just serializing an Instance type using JLD(2) if you're looking for intermediate storage. Long-term storage of instances in MOI format doesn't seem like the best idea at this point since we could easily decide to change the structures of the sets or functions before MOI 1.0.

[odow]

I'm not looking to save MOI objects in julia. But some general mathematical optimisation problem in a language agnostic way. The question is how general to make that. MPS and LP both have limitations. It's seems that the MOI general form is quite nice so it seems like a good starting point.

[mlubin]

I agree that MOI would correspond well with a language-agnostic file format. It just seems premature to start storing instances in this format because MOI will be changing.

[mlubin]

As mentioned in https://github.com/JuliaOpt/MathOptInterface.jl/pull/132, I'm going to be playing around with serializing MOI instances. The goal is to be able to write small instances to a human readable text format and use these in JuMP's tests to make sure it constructs the correct model without needing a solver. They could also be used in solvers' tests. Currently thinking JSON.

[odow]

Yes I had a play with this a while ago. Json works very well. A list of constraints. Each constraint is an object from a predefined list of sets with parameters. I'm on the farm ATM but will be back at work and my computer tomorrow.

[mlubin]

I'd structure it with a predefined list of functions and a predefined list of sets that you can mix and match to keep it closer to the original MOI. If you have some code already written would be happy to take a look.

[odow]

I don't think I got around to writing any code but I started (very much the operative word) a document with some specs Mathematical_Optimization_Format.pdf

[mlubin]

I like it, can we chat more?

[odow]

You're on east coast time yeah? I can do anytime Tuesday or Wednesday EST. Don't mind waking up so pick whatever works. After Wednesday I'm going to Australia for a few days so it'll have to be next week

[joaquimg]

What about TOML? They are being used in pkg3.

[odow]

We've made some progress in https://github.com/odow/MathOptFormat.jl

Feedback solicited.

[joaquimg]

Where its better to talk? Here? There? A gitter room?

[odow]

Best to open an issue over there.

[odow]

MathOptFormat now supports MOI v0.6. I'm reasonably happy with the design. It is just a one-to-one mapping with MOI.

It is built on-top-of MOIU so it supports all the current MOI functions and sets. (See the model definition https://github.com/odow/MathOptFormat.jl/blob/6928f658b66df72d98b7b3c8f971a3be19c33188/src/MathOptFormat.jl#L13-L28)

The workflow for writing an arbitrary MOI model is

my_model = Gurobi.Optimizer()
# ... model definition ...
mof_model = MathOptFormat.Model()
MOI.copy_to(mof_model, my_model)
MOI.write_to_file(mof_model, "my_file_name")

The reading flow is

mof_model = MathOptFormat.Model()
MOI.read_from_file(mof_model, "my_file_name")
my_model = Gurobi.Optimizer()
MOI.copy_to(my_model, mof_model)

[blegat]

It would be nice to be able to directly write the Gurobi optimizer. We would need to dispatch on a third argument being the file extension though.

[blegat]

@odow Is this resolved ?

[odow]

What do you think about adding MOF as a submodule to MOI?

[rschwarz]

I guess that means submodule in the Julia sense, not git sense.