Load timeseries data directly from Pandas DataFrame

[brynpickering]

Problem description

We only allow timeseries data to be added to the model from CSV. Ideally we would also have the possibility of having e.g. resource: dataframe=df:column for any dataframe df loaded into the current instance of Python.

Steps to introduce functionality

• [ ] Decide on syntax for calling a dataframe

  • The most likely is to follow the same as ...: file=filename.csv:column with ...: dataframe=df:column
  • How should it handle not finding a dataframe / running the model in CLI?

• [ ] In model_run.py, when loading timeseries, apply the df directly, rather than loading it from CSV

• [ ] Apply relevant checks: timeseries is consistent, data is consistent, parameter is allowed to be in timeseries

[ahilbers]

This is a feature that I currently "hack around" by creating and then deleting a CSV file. I'd be keen to implement this feature properly. I was thinking of using an extra argument when calling calliope.Model. This would be a dictionary where the values are the relevant time series. For example, in the national scale example model, suppose I want to use timeseries data from some dataframe csp_resource. I would first create the following dictionary:

timseries_df_dict = {'csp_resource': csp_resource}

where csp_resource is some dataframe. I'd then use the dict as an argument when calling calliope.Model:

model = calliope.Model('path_to_my_model.yaml', timeseries_dict=timeseries_df_dict)

In techs.yaml, I'd use techs.csp.constraints.resource: dataframe:csp_resource where csp_resource refers to the key in timeseries_df_dict. I'd write some tests to make sure this happened succesfully. After this, the treatment of the data happens in the same workflow as if it had been read from CSV.

I wonder what you guys think of doing it this way. From @brynpickering 's description above, it seemed like he was thinking of reading in the dataframes directly from the main namespace instead of passing them as an argument into calliope.Model. I proposed using an argument since reading in global variables into imported functions is always a bit risky, and this method makes more explicit which timeseries you are using, and allows you to easily have different models with different timeseries.

If this sounds good to you, I can go ahead and implement it.

[brynpickering]

Fixed by #305