Add ReducedSpace{<:AbstractPredictor} layer

[odow]

A second option for #70

• [x] Needs tests

[odow]

Okay. I'm happy with this for now. There's an open question about what to do for layers which don't support a reduced-space formulation. For now, errors are thrown. But we could add a fallback to the full-space formulation. My only concern is that then this is silent and might not be what the user expected.

[Robbybp]

I like this approach, and I prefer raising an error to silently falling back to the full-space formulation.

[odow]

Yeah so still an open question for "how do I make only some of the layers in a pipeline reduced-space" but that can be a separate issue if we have a concrete use-case.

[Robbybp]

You can always do something like

f_full = Pipeline(Affine(A, b), ReLU())
f_part_reduced = Pipeline(ReducedSpace(f_full.layers[1]), f_full.layers[2])

right? You're talking about an easier way to do this conversion?

[odow]

Yeah I guess, I meant if you have a PyTorch model, how do you say "make the 2-N-1 layers reduced space".

But aggreed you can do this manually.

[odow]

Perhaps we just need https://github.com/lanl-ansi/MathOptAI.jl/issues/74, and then you can muck with the layers after the fact.