calliope-project / calliope

A multi-scale energy systems modelling framework
https://www.callio.pe
Apache License 2.0
287 stars 93 forks source link

Account for python's optimize mode to enable faster model building #633

Open irm-codebase opened 3 months ago

irm-codebase commented 3 months ago

What can be improved?

We have the possibility of a speed upgrade by changing "if then raise" statements to assert cases in the backend. In cases were we know that a model is safe to run, we could call the -O option for python to skip these cases automatically.

We have these in several loops in the backend, so it could net a performance bump.

Version

v0.7

irm-codebase commented 3 months ago

Here is an example for the expression_parser.py file. The for is entirely for checking. Although in this case, maybe if __debug__ is better than assert.

    def as_array(self) -> xr.DataArray:  # noqa: D102, override
        lhs, rhs = self._eval("array")
        where = self.eval_attrs["where_array"]
        for side, arr in {"left": lhs, "right": rhs}.items():
            extra_dims = set(arr.dims).difference(set(where.dims))
            if extra_dims:
                raise BackendError(
                    f"({self.eval_attrs['equation_name']}, {self.instring}) | "
                    f"The {side}-hand side of the equation is indexed over dimensions not present in `foreach`: {extra_dims}"
                )
        lhs_where = lhs.broadcast_like(where)
        rhs_where = rhs.broadcast_like(where)

        match self.op:
            case "==":
                op = np.equal
            case "<=":
                op = np.less_equal
            case ">=":
                op = np.greater_equal
        constraint = op(lhs_where, rhs_where, where=where.values, dtype=np.object_)
        return xr.DataArray(constraint)
irm-codebase commented 3 months ago

Some hits of viable cases I found. Number on the left indicates "hits" (also national example). Couldn't find a way to make the debugger add them all together, sadly.

image