pvlib / pvlib-python

A set of documented functions for simulating the performance of photovoltaic energy systems.
https://pvlib-python.readthedocs.io
BSD 3-Clause "New" or "Revised" License
1.19k stars 998 forks source link

Add initial storage support #1333

Open Peque opened 3 years ago

Peque commented 3 years ago

Abstract

The storage support proposal for NumFOCUS Small Development Grants was recently accepted.

This issue aims to track progress of the development and discussions related to the proposal.

Goal

The goal of this project is to make a first step in supporting storage within pvlib. To do so, it aims to complete at least:

Phases

  1. Use case refinement phase a. Reading and understanding the state-of-the-art battery models and use cases b. Talking to stakeholders/experts to understand the most needed use case (PV installers, HW manufacturers and other professionals) c. Adjustment of the scope, first use case and detailed planning for the implementation phase
  2. Implementation of the first prototype a. Public discussion with pvlib maintainers for an initial candidate API b. Implementation of the first functions and tests c. Validation of the first use case with well-known data
  3. Integration a. Adjustments to the API for a ready-to-be-integrated version b. Project documentation update (API, storage, example code) c. Fix any issues or comments in the pull-request dis

Timeline

Peque commented 3 years ago

So, the first step is to understand the state-of-the-art battery models and use cases. I will start with:

Any other interesting readings are more than welcome! ๐Ÿ˜Š

Another goal is to talk to different stakeholders to understand the current market needs and technologies. Also, this would create awareness around pvlib and, maybe, attract more contributors. :rocket:

I will start with people I know I have a higher likelihood of response (people I'm already connected with in the Spanish/EU market):

If you know other people that you think it could be interesting to meet with, please share their contact or introduce us! It is always much easier to organize a meeting when you are introduced by someone or if you already know that person. :wink:

In particular, I think it could be interesting to meet with some hardware manufacturer. Also, meeting someone outside the EU would be good to avoid geographic bias.

btaute commented 3 years ago

With respect to further reading / reference material:

With respect to conversations:

Peque commented 3 years ago

@btaute Thanks for your suggestions and interesting pointers! :blush:

May I contact you using the Gmail address you use on your commits?

btaute commented 3 years ago

Ya, that would be great.

cwhanse commented 3 years ago

So, the first step is to understand the state-of-the-art battery models and use cases. I will start with:

There is a lot of battery-related content going into the next release of SAM, I believe. @janinefreeman perhaps you could point @Peque to a few of the key algorithms?

janinefreeman commented 3 years ago

Hi all! We have many updates to the battery models and dispatch algorithms going into SAM for the next release, and we would love to make sure the pvlib capabilities are closely aligned with SAM's capabilities. @Peque we're happy to meet with you to help figure out the best way to make this happen, and which algorithms may be of interest for your first steps. My colleagues Darice @dguittet and Brian @brtietz are great resources for the battery modeling pieces of SAM as well. @cwhanse can send an email intro as needed! Thanks!

Peque commented 3 years ago

Thanks a lot @janinefreeman, that would be great! :blush:

williamhobbs commented 2 years ago

I agree with @btaute about looking at EPRI's DER-VET/StorageVET. I could make email introductions with EPRI contacts if that has not already been done and is of interest. And of course I agree that SAM is an excellent resource.

I can be available to provide any input that might be useful. I have an electric utility and IPP/developer background. My pvlib experience is a little light, but I have been a pretty heavy SAM user over the years.

This is likely outside your current scope, but if future work will involve battery dispatch for PV output smoothing, https://github.com/epri-dev/PV-Ramp-Rate-Smoothing could be worth looking at. It will be in NREL SAM soon. A 2021 IEEE PVSC preprint should be available soon, or I can send a copy over email.

This is also likely outside your scope, but I'd be interested in discussing things such as modeling real-world considerations of needing both solar and load forecasts for for optimized dispatch if variable utility pricing is included. Hypothetical example: it's 10AM, the battery just charged with some excess energy, and now load drops below solar. Do you discharge it now when time of use rates are low, or store that energy until afternoon when TOU rates are high but miss out on capturing additional excess energy if it becomes available? If you have a forecast of excess energy between now and then, you can make a better decision.

cwhanse commented 2 years ago

I think functions that represent automated controls, like ramp rate smoothing, could be considered within the scope of pvlib. An algorithm to optimize storage dispatch based on forecasts and TOU rates strikes me as scope for another project.

Peque commented 2 years ago

I leave here a couple of notes I have taken during the use case refinement phase. With the intention of starting with the next phase (implementation of the first prototype) this month. :blush:

Market analysis

So, one of the goals for this initial phase was to analyze the market and talk to different stakeholders. During November I had the opportunity to go to a national professional fair about renewables. There, I had the opportunity to talk to PV installers, hardware distributors and hardware manufacturers.

With the idea of covering a first use-case for residential self-consumption (behind-the-meter, small PV, stationary storage), this is the feedback I was provided with:

This feedback may of course be biased, since we are talking about a national fair. It may extrapolate better to other countries in the EU market but worse to other markets around the world.

In particular, I was told that the use of DC-connected storage is not so common in other markets. For example, in the US it seems it is more common to use AC-connect batteries (i.e.: Tesla batteries). This may be influenced by the local regulations. In Spain, for example, you may get grants if you use DC-connected batteries, since they maximize self-consumption and you cannot charge them using grid power.

Models and code reuse

@cwhanse and @janinefreeman were very kind to organize a meeting in order to discuss this initial implementation and answer some questions. @dguittet and @brtietz (from NREL) were also present. They have been directly involved in the storage developments within SAM. :heart:

Due to the time constraints of this project we agreed that what makes sense (and is viable) is to integrate the capabilities of SAM (i.e.: battery models) by adding it as an optional dependency to pvlib. Cliff agreed that was okay and the goal of pvlib is always to support different/multiple models. So the idea here is to define a flexible API that could allow the integration of other models/dependencies/implementation in the future.

SAM implements 2 battery models:

The initial pvlib implementation will be focused on supporting a custom dispatch series (i.e.: delegate any energy flow optimizations). Although for this use case the BatteryStateful strikes as more flexible, performance may be an issue. These were the results of my performance tests, in both cases using a custom dispatch with a full-year hourly series (8760 data points):

Pending confirmation in case there could be other factors affecting the results, this could be due to the Python-to-C-to-Python interfacing. I noticed the .Controls.input_power parameter of the battery-stateful model needs to be numeric (i.e.: not an array/tuple), so I'm guessing iterating at the Python level is a must to run this model with PySAM.

Since the idea is to cover an initial use-case for residential self-consumption, simulations are expected to span over a period of multiple years (10+, up to 25). 15 seconds per year would mean minutes to wait for the results, which sounds unreasonable.

I will, therefore, start with the Battery model, even if ignoring or not using many of its ancillary models/capabilities for this initial implementation in pvlib.

This comes with another restriction: AC connection. Supporting DC-connected batteries with the Battery model would require changes in SAM's C++ code.

AC connection sounds reasonable for a first implementation in pvlib, since DC connection would require changes in the inverter model as well, to take into account inverter constraints when it comes to the interter-to-battery connection and power flow. For what I know, no inverter models in pvlib currently contemplate DC-connected storage (and SAM, using the Sandia model, does not either).

Other use cases

Of course this is just the first step in adding storage support, so other use cases can be added later on. @btaute @williamhobbs Be sure I will contact you for those involving larger storage for utility-scale or front-of-meter installations. :blush:

mikofski commented 2 years ago

Thanks @Peque, baby steps, this is just a starting point. We can use this as a launching point for future additions like DC-coupled or relaxing hourly time constraints. I have also been doing a lot more PV+BESS modeling lately so I may be able to contribute more. Another thing I would suggest is that you could also contribute to SAM/SSC as well. Perhaps that's out of scope for the small dev grant, but something to keep in the back of your mind.

Peque commented 2 years ago

Thanks for your support @mikofski!

I'm starting to fear the same (that at some point, changes to SCC may be required). While I have experience with Python programming, and even some with embedded C, I still think C++ is one of the most difficult programming languages. :joy:

Anyway, the team at NREL was very welcoming and open to external contributions. So that compensates the programming language barrier. :blush:

wholmgren commented 2 years ago

Thanks to everyone for their time and efforts so far! @Peque sorry if I missed this, but what is the value in wrapping pySAM? Or why not just use/improve PySAM directly? Perhaps a draft PR would make this obvious, so my apologies if I'm both too late and too early to these discussions.

From my perspective, a slow but reliable and reasonably complete python implementation would also be welcome. Once we're happy with the algorithms, I suspect numba could improve performance relatively easily.

adriesse commented 2 years ago

Thanks for the updates @Peque. At a very high level, what information would be the inputs and output to/from SAM?

adriesse commented 2 years ago

Apparently @wholmgren was typing while I was thinking similar things. Somethings really basic in native pvlib would be my preference, although I recognize there may be users who have different needs.

Peque commented 2 years ago

Thanks for sharing your thoughts!

@wholmgren I think pvlib and PySAM have different approaches. While pvlib aims to be flexible maintaining both a structured and OOP API and, at the same time, allows and supports multiple models to run simulations, PySAM is just a way of exposing the still relatively rigid API of SAM (and SAM's goal, for what I know, is not to be flexible and support many different models).

The goal of my initial proposal was to implement something in Python (when I say Python I mean all the scientific Python ecosystem that pvlib is already using like NumPy and Numba). At the time I didn't really understand the implications of that statement (battery models, lifetime models, losses, optimization algorithms for energy flow...). While I still think native Python models could be a great addition, after talking to NREL's team, I was recommended, due to the time constraints of this project, to go for an integration of the already-existing models.

I think this work, on its own, will be beneficial for pvlib, since it defines the API and paves the way for adding more models in the future, which could rely on different packages (i.e.: not SAM) or could be even (re)implemented in Python.

Improving PySAM, or SAM's modularity, may be a sub-product of this development, but I don't think it should be the end goal.

@adriesse At the moment, although this still needs to be discussed for the initial prototype, I see 2 modules. One module handles the storage models. The inputs would be the dispatch series (i.e.: energy flow in and out the battery) and the battery parameters. The output would be multiple series that could include: the energy flow in-and-out the battery, the energy "excess" (that could not flow in-or-out the battery due to the battery constraints), the state of the battery at the end of the cycles or the degradation series. Another module could be developed to handle basic dispatching rules. For example, I'd like to support in this initial use case a dispatching algorithm that maximizes self-consumption. The inputs would be the generation series and the load series. The output could be the series of the battery energy flow and the grid energy flow. This algorithm in particular could be easily implemented in Python without depending on (Py)SAM.

Anyway, I'd like to note that my intention is not to end my contributions after wrapping SAM's models. I'd love to help implementing native models with a more modular approach, but I'd like to comply with the initial planning (i.e.: have a prototype running by January and something integrated by March, with a limited dedication each month). If using existing models helps achieve that goal I think it is worth it. :blush:

NumFocus's SDG projects can span over one year I think, so there will be plenty of time for improvements. My agile-development mindset pushes me to try to honor dates/milestones first, and then iterate from there. I think if we try to start with native models, there is a higher risk of failure at the end of the project.

@cwhanse Would you like to share your thoughts?

If there is a better way to move forward, we are still in time! :smile:

cwhanse commented 2 years ago

To me the value of wrapping pySAM is to facilitate use of SAM capabilities, and to bring those capabilities into (what we hope will be) a common interface for battery/storage models. We don't yet have the interface but Peque's response to adreisse (above) outlines what could become the interface.

mikofski commented 2 years ago

A couple of notes:

  1. While having the flexibility to tinker with the code can be an advantage, I don't think pvlib's core philosophy values that over established, published, peer-reviewed, documented, and tested algorithms.
  2. Having a pvlib-PySAM interface to a battery model that is rigid isn't necessarily a bad thing and accomplishes the goal of the SDG project quite nicely. It is a low barrier that I bet will have it's share of complications, and if it proves to be very easy then we will have leftovers to start on more ambitions projects.
  3. The goal should match the customer pains, gains, and jobs to be done (see https://www.strategyzer.com/canvas/value-proposition-canvas). In that respect I humbly suggest that users' biggest pain, gain, and job to be done are all to model PV + storage. I believe that integrating the PySAM battery model into pvlib relieves their pain, creates gain, and gets the job done.
adriesse commented 2 years ago

A couple of thoughts:

  1. @mikofski I think pvlib-python offers rather more than the ability to "tinker with the code" ! (And I know you would agree.)
  2. @Peque For the interface/wrapper/api although you intend to target the annual+hourly model in SAM, perhaps you can still keep the pvlib-python portion generic enough to accommodate arbitrary timesteps and durations.
  3. @mikofski Who is our customer? Anyone who is in a hurry can just use SAM or any of the other tools that already offer storage (or pay a consultant/consulting firm).
Peque commented 2 years ago

@adriesse Yeah, that is the idea. Not providing an annual+hourly time series would throw an exception when dealing with this specific backend, but that should not generally be the case. :blush:

I think pvlib's customers could be all pvlib's users. Each start in GitHub, each fork, each citation... Everything counts towards a growing community. A bigger community also means higher chances of people contributing by opening issues, suggesting changes, creating pull requests or sending proposals for NumFocus' SDG program or any other program like Google Summer of Code. A growing community also increases the chances of companies investing back in pvlib (i.e.: by investing development time).

Anyone can use SAM. But I don't think SAM is the software everybody wants to use. In my case, for example, I could have used (Py)SAM to solve my needs (i.e.: run PV simulations with Python). I opted for pvlib though, even if it had its limitations like not having storage implemented, for a couple of reasons: flexibility, modularity, implementation language and API.

I consider myself a pvlib customer. When I wanted to simulate storage, I could have gone with SAM, but I thought (and I really hope! :joy:) I could contribute back to pvlib by adding a missing piece that is gaining importance every day.

I think the gain here is that pvlib would allow to easily incorporate new storage models. SAM backend will just be the first one. I imagine pvlib first implemented a single inverter model? (i.e.: Sandia) Which could have already been implemented in SAM by the time as well (I don't know), but now pvlib provides other models like yours, Anton. Justifying that addition to SSC, for example, could have been harder, since supporting multiple models may not be their goal and it may require many other changes in their core software SAM.

wholmgren commented 2 years ago

I think we need to ground this conversation in some pseudocode or simple scripts that actually use pysam with the battery feature as intended here. I didn't find any good examples with a few minutes of googling.

dguittet commented 2 years ago

Run speed

  • The battery model run in a couple hundred milliseconds
  • The battery-stateful took a couple milliseconds for each iteration (for each call to .execute()), which resulted in a total simulation time of over 15 seconds

@Peque Could you share the code you used for this comparison? I did a quick run with PySAM v2.2.4 using this attached script BatteryWithCustomDispatch.py.zip, which is a modification of the Example here. I found a less dramatic difference for a single year 8760 hr run: 0.13 vs 0.17 s, a 30% increase in time for the battery stateful module to do the basic simulation (not collecting data from each time step).

The idea of feeding an input array of control powers or control currents sounds like a good idea-- I think that could help speed things up while making it more usable. But since this is for efficiency, I wouldn't make this a priority for this project unless this change will make a significant impact on run time for the entire workflow. Like if there were something more complex in your usage of the stateful module that has led to a dramatic difference in performance.

If the performance is not a factor, IIRC the other criteria are time stepping & duration, DC vs AC, power flow accounting, input and output monitoring/recording.

Interfacing w/ Storage Models

If the goal is to establish an interface for a generic storage model that can eventually be coupled to a native Python model that uses Numba and so forth for speed, you could consider as your first step implementing the interface with a bag-of-coulombs battery model. And from there figuring out what inputs and outputs you need in order to meet your use case of self-consumption with a DC-connected residential battery.

Something simple like this may work, and can perhaps be massaged in order to be vectorized:

For a battery of given size (๐‘ƒ_๐‘)^ kW, (๐ธ_๐‘)^ kWh image

Eb : battery energy, kWh Pb : battery dispatch power, kW, > 0 for charging

Once you have water flowing through all the pipes, you can replace it with a PySAM battery model. That process will also give you the opportunity to document / standardize how to switch to a new battery model.

Peque commented 2 years ago

@dguittet Thanks a lot for your comment Darice! :blush:

My code looked like this:

import pprint as pp
import PySAM.BatteryStateful as battery

params = {
    # specify power input
    "control_mode": 1,
    "input_power": 0,
    # run second by second
    "dt_hr": 1 / 360,
    # Options: 0=LeadAcid,1=LiIon
    "chem": 1,
    # setup size
    "nominal_energy": 10,
    "nominal_voltage": 500,
    # initial conditions
    "initial_SOC": 50.000,
    "maximum_SOC": 95.000,
    "minimum_SOC": 5.000,
    # voltage parameters
    "voltage_choice": 0,
    "Vnom_default": 3.600,
    "resistance": 0.0001,
    "Vfull": 4.100,
    "Vexp": 4.050,
    "Vnom": 3.400,
    "Qfull": 2.250,
    "Qexp": 0.040,
    "Qnom": 2.000,
    "C_rate": 0.200,
    "Vcut": 2,
    # thermal parameters
    "mass": 507.000,
    "surface_area": 2.018,
    "Cp": 1004.000,
    "h": 20.000,
    "cap_vs_temp": [[-10, 60], [0, 80], [25, 1e2], [40, 1e2]],
    "T_room_init": 20,
    # cycling fade
    "life_model": 0,
    "cycling_matrix": [
        [20, 0, 1e2],
        [20, 5e3, 80],
        [20, 1e4, 60],
        [80, 0, 1e2],
        [80, 1e3, 80],
        [80, 2e3, 60],
    ],
    # calendar fade
    "calendar_choice": 1,
    "calendar_q0": 1.020,
    "calendar_a": 0.003,
    "calendar_b": -7280.000,
    "calendar_c": 930.000,
    "calendar_matrix": [[-3.1e231]],
    # no losses or replacements
    "loss_choice": 0,
    "replacement_option": 0,
}

b = battery.new()

for k, v in params.items():
    b.value(k, v)

b.setup()

start = default_timer()
for i in lifetime_dispatch:
    b.Controls.input_power = i
    b.execute(0)
print("time", default_timer() - start)

Got the parameter structure from an example as well. The issue seems to be with the time delta! Note how it was set to 1 / 360 (second timesteps). I didn't think the time delta could have an impact when maintaining the number of iterations, but it really does. Switching the dt_hr makes things run much faster, in accordance with your results. Is that expected? (the fact that a single iteration is significantly slower for smaller time steps)

Under these circumstances, I think there is no significant performance penalty for using the stateful model, so I will consider it for the first integration. I have quickly checked and storing intermediate state parameters (i.e.: accessing b.StatePack.P in each iteration) does not seem to have a significant impact in performance either. :tada:

janinefreeman commented 2 years ago

One very late-to-the-game comment from me: one consideration that @cwhanse and I have discussed is, how can we best avoid duplicating effort (e.g. recoding existing battery models from SAM into python)? There will obviously be some cases where that is unavoidable, but ideally the broader group of bright minds and talented programmers involved in pvlib/SAM/PySAM can get farther by leveraging each other's code. :)

Peque commented 2 years ago

ยกHappy New Year to everybody! :tada:

So, I opened a draft PR to start a discussion.

Any comments/suggestions are more than welcome! :blush:

A couple of thoughts:

adriesse commented 2 years ago

Hi @Peque,

I read through everything to get an idea of where you're headed, but was kind of waiting for the flurry of comments from others to begin before piping up. I do agree it would be best to start with discussing what's described in the documentation. Anyway, main point of this comment is to signal that your work has not gone unnoticed! :)

Anton

Peque commented 2 years ago

After the initial review, I pushed a couple of changes taking into account your comments. Thanks again for your feedback! :blush:

I have been thinking about the scope of this PR and where to head next.

For self-consumption use cases, dispatching optimization at the end requires taking into account the prices. Implementing these types of optimizations may be out-of-scope for pvlib. As noted in An Overview of the Automated Dispatch Controller Algorithms in the System Advisor Model, automated dispatch strategies with one day look-ahead may be too optimistic, and one day look-behind may have little predictive power depending on the use case. Also, while Heuristic Dispatch Based on Price Signals may be the way to go, that again may fall outside of pvlib's scope.

However, I could add an example to show how to code a simple custom dispatching for a self-consumption use case that limits discharging to specific hours of the day. This way users will know how to use other dispatching techniques on their own if needed.

If you agree, I could concentrate my efforts in things that may be better suited for inclusion in pvlib:

  1. Documentation on how to simulate batteries with data from the datasheet (in my experience datasheet information is more uneven than with inverters and, hence, it may otherwise be difficult to model commercially available batteries for newcomers).
  2. Addition of the basic functions for DC-connected battery simulations, progressively taking into account:
    • Output power limited by the inverter
    • DC-DC lower conversion losses
    • Clipping reduction

Perhaps point 2 could be a starting point for discussing a new inverter model (some inverters, for instance, impose limits on the power that they can deliver/withdraw to/from the battery).

What do you think?

mikofski commented 2 years ago

Sorry for not answering your question directly. IMHO pvlib should model storage as if the dispatch algorithm were known a priori with perfect foresight already therefore decoupling the energy calculation from any economic or energy optimization. I think this approach has several advantages:

  1. faster development and implementation because there are less design parameters and less problems to solve - it descopes and focuses development specifically on the battery energy model
  2. in principle should allow a solver to use this function as input, and then optimize for a given objective by iterating over the dispatch algorithm and manipulating the output as needed for LCOE, capex, opex, energy, etc.
  3. allow very simple dispatch algorithms to be implemented directly like firming, shifting, and end excess shifting where those dispatch schedules can be directly and explicitly inferred from the pv output.

does this help? Happy to chat more

PS: psuedo-example of #2 & #3 above:

from pvlib import bess
from scipy.optimize import minimize as sc_min

# bess.dcc_battery is a data class that has parameters like max charge & discharge rates and max capacity
# bess.calc_dcc_energy(pv_dc_energy, inverter, dispatch_schedule, dcc_battery_params)
#     calculates dcc energy flow & state of charge
# bess.calc_dcc_firming_schedule(pv_dc_energy, inverter, grid_limit, dcc_battery_parms) calculates a firming schedule
# bess.calc_dcc_shifting_schedule(pv_dc_energy, inverter, dispatch_starttime, dcc_battery_parms) calculates a firming schedule
# bess.calc_dcc_excess_schedule(pv_dc_energy, inverter, dispatch_starttime, dcc_battery_parms) calculates a firming schedule

# find the dispatch algorithm that maximizes 
# calculate pv_dc_energy or energy at MPP for every timestep given modules, irradiance, etc.

def total_bess_dcc_energy(pv_dc_energy, inverter, dispatch_schedule, dcc_battery_params):
    return sum(pv_dc_energy, inverter, dispatch_schedule, dcc_battery_params)

# use scipy minimize or something like it to find the largest total dcc energy
sc_min(
    f=lambda dispatch_schedule: -total_bess_dcc_energy(pv_dc_energy, inverter, dispatch_schedule, dcc_battery_params),
    args=dispatch_schedule)
Peque commented 2 years ago

After playing around a bit with the code, I think I will finally implement a first DC-connected battery model/solver that does not take into account the fact that the power provided by the battery would alter the efficiency of the inverter's output.

In example:

This is fat from perfect but could be a good first step and would not require the addition of a new inverter model (which could be considered afterwards). Does that sound good to you?

Also, in order to take into account the clipping losses, I see that current pvlib inverter models apply the clipping just before returning the AC power, but that clipped power is not returned. Would it be considered reasonable to update current inverter models to return both the output AC power and the clipped power? I for instance think that information about clipping losses could be relevant for other use cases (not just batteries). :blush:

Peque commented 2 years ago

Even if later than I would have anticipated, I pushed some changes to the associated PR (https://github.com/pvlib/pvlib-python/pull/1378).

Basically, I ended up discarding my own initial idea of creating the "simplest" DC-connected battery model/solver that didn't take into account the fact that the power provided by the battery would alter the efficiency of the inverter's output. The implementation I ended up with wasn't as simple as I thought and the result was less than ideal.

Instead, I went ahead with a new inverter model implementation that does take into account things like "smart" charging to avoid clipping losses (even when the custom dispatch series wasn't set to charge at that instant) and the fact that the battery power affects the efficiency of the inverter. To do this, I based the implementation on the sandia_multi model. Since I guess the result is not up to the standards required in pvlib for inclusion as an "official" inverter model (and is not based on any published paper), I left the implementation in the powerflow module, available only for those adventurers that want to dive into storage and DC-connected batteries. :smile:

You can have a look at the documentation generated in the PR if you'd like to provide some feedback.

I updated the PR to note that the code is up for review as well, in case someone wants to suggest changes to the API before inclusion.