Gurobi / gurobipy-pandas

Convenience wrapper for building optimization models from pandas data
https://gurobipy-pandas.readthedocs.io
Apache License 2.0
92 stars 15 forks source link

Implement an MVar-backed extension array #49

Open simonbowly opened 1 year ago

simonbowly commented 1 year ago

The pieces are all there for this in gurobipy v10 and since everything would be a 1D operations performance should be very good. We would only enable the extension for gurobipy>=10, and would likely want a way to disable it in case unsupported pandas operations come up.

A tricky part would be name generation:

simonbowly commented 1 year ago

Put a rough implementation together on the extension-dtype branch. From 6848bc9, with gurobipy 10.0.0b1 (benchmark script creates 300k variables and 100k linear constraints):

> python bench.py 
Gurobi 10.0.0 (beta1) - expires 2022-11-21

===== Without Extension =====
                   x                  y                  z                     c                      lhs sense  rhs
0  <gurobi.Var x[0]>  <gurobi.Var y[0]>  <gurobi.Var z[0]>  <gurobi.Constr c[0]>  x[0] + y[0] + -1.0 z[0]     <  0.0
1  <gurobi.Var x[1]>  <gurobi.Var y[1]>  <gurobi.Var z[1]>  <gurobi.Constr c[1]>  x[1] + y[1] + -1.0 z[1]     <  0.0

===== With Extension =====
                   x                  y                  z                   c                      lhs sense  rhs
0  <gurobi.Var x[0]>  <gurobi.Var y[0]>  <gurobi.Var z[0]>  <gurobi.Constr R0>  x[0] + y[0] + -1.0 z[0]     < -0.0
1  <gurobi.Var x[1]>  <gurobi.Var y[1]>  <gurobi.Var z[1]>  <gurobi.Constr R1>  x[1] + y[1] + -1.0 z[1]     < -0.0

===== Create 100000 linear constraints =====
Without Extension: 4.58
With Extension:    1.13

As noted above, we get default constraint names. But the 4x speedup looks promising already (and no, the time spent generating names is not significant in the test).