Serialize (and deserialize) a MathematicalProgram (e.g. to Yaml)

[RussTedrake]

For mathematical programs using only built-in types for costs and constraints (e.g. LinearCost, LinearConstraint, ...)... it should be possible to serialize an entire program to yaml... and load it again. There are existing serialization formats for optimization problems from other packages (e.g. ampl), but adding the yaml workflow will allow the same methods to additionally be used with other parts of Drake. For instance, we would like to serialize GraphOfConvexSets instances as a follow-on effort to this.

The relevant yaml serialization documentation is here.

Adding the serialization methods to the costs and constraints should be easy enough. The serialize method in MathematicalProgram knows enough of the type information for some of the types (e.g. it keeps separate lists for linear_constraints, linear_equality_constraints, etc) to support deserialization. However, there are other built-in types that we might like to serialize/deserialize which currently get dumped into "generic" costs/constraints. Examples include L2NormCost, ExpressionConstraint, etc. I think we would need to be a little more clever about deserializing to those types. cc @jwnimmer-tri in case he has any wisdom/recommendations.

cc @bernhardpg .

[jwnimmer-tri]

Is the MathOpt format (#17280) able to fully express what we need? If we can load/save to a standard format, that seems like it might be even better that a custom YAML format.

(Our YAML serialization infrastructure can read/write JSON files, too.)

[RussTedrake]

If we think about the GCS case, then the GCS class generates mathematical programs (in a few different forms)... but I'm pretty sure that the structure of the GCS problem itself does not fit into the MathOpt format.

GCS => MathProg => Solve could be GCS => MathProg => MathOpt and then MathOpt => MathProg ... but I don't think we can get MathOpt => GCS?

[jwnimmer-tri]

Ah. Serializing a GCS and the costs + constraints inside it is a different beast than serializing a MathematicalProgram. I'd overlooked the mention of a follow up to serialize the whole GCS. I agree that it GCS serialization is probably beyond what MathOpt can express.

For our cost and constraint serialization, possibly we should consider re-using the mnemonics (and/or syntax) from the MathOpt format, as a building block -- instead of the Drake class names per se. That might make it easier for better interop down the road.

Whatever we choose for the syntax, tactically I think we should probably not plan to add a Serialize member function to our existing classes like LinearConstraint or Binding. The way that a symbolic::Variable gets serialized is tricky (see #17068), and might depend on the context its used in. There's a good chance that GCS's private variables will be serialized differently than we'd do for the user's own variables. Keeping all of the serialization choices private to a single SaveGcs and LoadGcs helper functions is probably a safer bet, while we figure things out. We could always promote them to member functions later, if we choose. Best guess, right way to do this is to "(de)compile" the MP or GCS into a tree of vanilla helper structs, and then use the serialization tools on those private helper structs. That decouples the I/O format from the implementation choices, which should be an aid to maintenance. (Consider things like: how does a sparse matrix get serialized, versus how is it represented in the runtime evaluator's code.)