mikofski commented 1 year ago

Is your feature request related to a problem? Please describe. In case weather file doesn't have RH or Pwat, I would like to be able to use Tdew to calculate spectral mismatch

Describe the solution you'd like This capability is already implemented in MetPy relative_humidity_from_dewpoint

Describe alternatives you've considered reimplement in pvlib? import metpy and call it? ignore b/c most weather files already include RH or Pwat, or uncertainty in chain of conversions too much?

Additional context I thought this topic came up somewhere else but I couldn't find it in google group, discussions, or issues/pr's

adriesse commented 1 year ago

I would prefer to duplicate very minor models/functions in pvlib rather than expand the list of dependencies.

AdamRJensen commented 1 year ago

I find using metpy annoying due to the requirement of specifying units; thus I agree that it would be nice to have these simple functions in pvlib.

wholmgren commented 1 year ago

I agree with Anton that these models and implementations are simple enough that it's better to just copy (with attribution/license) than to introduce a non-trivial dependency. I also agree with Adam that the metpy units requirement is annoying.

The dewpoint to relative humidity calculation is essentially exact for this purpose so I don't have any concerns about introducing additional uncertainty.

If we agree that we should add this function then I might have someone that would be interested in contributing a PR.

adriesse commented 1 year ago

I suggest looking up the equation in a textbook and writing a new function referencing that. I did this years ago using the meteonorm documentation as reference.

mikofski commented 1 year ago

A coworker shared this reference: http://www.bom.gov.au/climate/averages/climatology/relhum/calc-rh.pdf, which is a calculation of the ratio of vapor to saturation vapor pressures based on ambient and dew point temperatures

kurt-rhee commented 10 months ago

Another reference that may be useful for saturation vapor pressures. PlantPredict uses the AEKR coefficients over water for the magnus form equations.

https://www.osti.gov/servlets/purl/548871-PjpxAP/webviewable/

uakkoseo commented 4 months ago

I'm currently working on this. There are three different options. MetPy uses Bolton1980, @kurt-rhee mentions PlantPredict uses the AEKR, @mikofski pointed us to the document that uses Abbot 1985. It's unclear to me whether it is preferable to choose one approach over another. Any thoughts?

wholmgren commented 4 months ago

https://github.com/Unidata/MetPy/issues/508 also relevant. I suspect all of the above is fine for PV modeling applications.

AdamRJensen commented 4 months ago

I'm currently working on this. There are three different options. MetPy uses Bolton1980, @kurt-rhee mentions PlantPredict uses the AEKR, @mikofski pointed us to the document that uses Abbot 1985. It's unclear to me whether it is preferable to choose one approach over another. Any thoughts?

Personally I'd love to see all three of them. If they have similar input arguments they could all be bundled in one function that has a model argument, like this:

temp_dew_from_relative_humidity(relative_humidity, model='aekr')

adriesse commented 4 months ago

I would prefer to see just one good one rather than a range of options since this is peripheral to PV modelling.

kurt-rhee commented 4 months ago

@uakkoseo

I don't have a strong preference for either implementation, I believe that the model accuracy in either case must be quite rough. If it helps, here is the C# implementation of the equation mentioned above:

/* 
 * These magnus coefficients below are the AEKR coefficients which reduce overall errors the most between -40 and 50 degrees C according to research by NOAA:
 * https://www.osti.gov/servlets/purl/548871-PjpxAP/webviewable/
 * 
 * magnusA variable is not explicity needed since the value is cancelled out by a division operation, but the variable is kept here
 * for the sake of keeping the complete equation.
 * 
 */
double magnusA = 6.1094;
double magnusB = 17.625;
double magnusC = 243.04;

// e:  amount of water vapor in the air
var e = magnusA * Math.Exp((magnusB * Temperature) / (magnusC + Temperature));

// es:  amount of water vapr in the air if the air were entirely saturated at the same temperature and pressure
var es = magnusA * Math.Exp((magnusB * dewpoint) / (magnusC + dewpoint));

return 100 * (es / e);

uakkoseo commented 4 months ago

Thanks @kurt-rhee ! I think it is easy enough to bundle them all in one function and give the option to choose the model, unless one is sufficient or deemed better than others. From what I understand, all three of these models are used often, the uncertainties seem to be similar for different temperature ranges. The AEKR model has the largest range for temperature out of the three. If used in the stated temperature range Bolton and Abbot seem to be as applicable. (I understand as the issue @wholmgren linked also points to this as well)

wholmgren commented 4 months ago

More models = more problems. I agree with @adriesse that it's not worth it in this case. My inclination is to copy what MetPy has already implemented.

uakkoseo commented 4 months ago

I don't have a strong stance to not use MetPy but for PV modelling, AEKR might be the better of the two as it seems to work better above 35C (noting worse below -50C compared to MetPy).

markcampanelli commented 4 months ago

One approach could be to provide one reasonably validated model obeying a well defined interface with a mechanism that allows users to plug in their own model.

wholmgren commented 4 months ago

Copying in the error plot from that MetPy issue...

If I understand correctly, @uakkoseo and @kurt-rhee are proposing to implement the model in blue. Let's just do that and move on with life!

adriesse commented 4 months ago

proposing to implement the model in blue.

I have a lot of trouble figuring out from this discussion what's what. Is the blue line AEKR? The MetPy discussion links to a WMO report from 1966.

I would be content with a formulation that is currently used by some entity like WMO, ECMWF, NOAA, or even Meteonorm. If those entities use different formulations, then feel free to choose randomly from among them!

Let's just do that and move on with life!

uakkoseo commented 3 months ago

@adriesse thank you, I tried to find some guidance from the entities you listed above. I was able to find NOAA formula following this page to be using Tetens formula (MetPy uses Bolton, a modified version of Tetens with values from the more recent formula of Wexler) and Meteonorm has different coefficients as well using DWD 1979 (different than what is used in MetPy or AEKR equation). I hope the different formulations listed here can mean AEKR equation is okay to proceed with.

adriesse commented 3 months ago

I hope the different formulations listed here can mean AEKR equation is okay to proceed with.

The AEKR equation was published 27 years ago but not adopted by major institutions. Should pvlib?

kurt-rhee commented 2 weeks ago

Hey @uakkoseo if you haven't already gotten around to this, I can possibly contribute something in the next week or so.

uakkoseo commented 2 weeks ago

@kurt-rhee honestly I was a bit stuck on how to proceed on this! If you have a better grasp on it than me please contribute as I wasn't sure if there was a final agreement on the formula. I won't be able to get back to this earlier than next month or so otherwise.

kurt-rhee commented 1 week ago

Adding comments from the PR to this thread for continuity sake:

Reasoning

There are some occasions where dew point temperature is available, but relative humidity is not.
Relative humidity can be converted to precipitable water in pvlib via guemard94.
Precipitable water is used by first solar to calculate spectral correction factors.
This function would allow a user with dew point temperature to convert it to relative humidity.

Caveats

Caveat 1

In the issue thread: https://github.com/pvlib/pvlib-python/issues/1744 there is some controversy about the simpleness of this approach. Some users would prefer a more well known solution.
I would argue that because this approach was used in the creation of the First Solar spectral correction function, that it has historical significance and should therefore be included.
If any user should desire, they could add an alternative function and then create a controller function which allows the user to choose different relative humidity calculation methods, which is a pattern than exists already inside pvlib.

Caveat 2

I placed these functions inside of spectrum, but I could also see them living inside of pvlib.atmosphere.py where the gueymard94 function exists. I'd like the maintainers opinion before proceeding to update the docs and what's new file.

adriesse commented 1 week ago

@kurt-rhee Thanks for reactivating this issue.

I don't think there is any controversy about simpleness, nor is this a discussion about user preference. The content of pvlib needs to stand on firm foundations.

So, I did some research.

Calculating RH from the ratio of two saturation vapor pressures, $e_s$ is fine.

There are many different equations for $e_s$ and actually there is one already in pvlib--it's embedded in gueymard94_pw! The most common equation form for $e_s$ is the Magnus aka Tetens form. It is recommended by the WMO Handbook and used among others by: Meteonorm, DWD, Lufft, MetPy, NOAA, and of course PlantPredict. It seems appropriate for pvlib despite the fact that more accurate formulas exist.

The only question remaining is which set of three coefficients to use because there are many sets to choose from. Superficially they all look very similar, but due to the exponential function in the equation, small differences could become significant.

The coefficients shown in recent editions of the WMO Guide to Meteorological Instruments and Methods of Observation are: (6.112, 17.62, 243.12). The guide states that "These formulae are convenient for computation and sufficiently accurate for all normal meteorological applications" in the range -45 to 60 C. These values were first published in Sonntag 1990, but I can only find Sonntag 1994.

Based on my research, I recommend using these WMO coefficients to create a generic tdew_to_rh function for pvlib. The saturation vapor pressure equation could be a separate function also, and optionally, an inverse function rh_to_tdew could be created at the same time.

The question about historical significance could be assessed by further investigation (@kurt-rhee?):

Was Tdew used in publications by First Solar, or were RH data available?
Is the model spectral factor affected substantially by the choice of Magnus equation coefficients--in particular AEKR vs. WMO?

kurt-rhee commented 1 week ago

Ah I totally misunderstood the original asks in the conversation, I'm happy to change the default coefficients to the ones shown above and then change the formulation to the function to allow for users to change the input coefficients as they desire.

As for the last two questions:

My understanding is that First Solar tested with both Tdew and RH to ensure that either data set would give similar results.
Given how rough the model is for converting Tdew to RH, I think that many (reasonable) coefficient sets would give similar results. I did a bit of testing in my PlantPredict days and the input coefficient set didn't make a huge difference in the resulting energy model.

I'll create a new commit with the desired changes this week!

kandersolar commented 2 days ago

+1 to @adriesse's plan; basing our function on the WMO guide seems like the way to go to me. Anton, thank you for figuring out a good path forward here and breaking up the logjam!

pvlib / pvlib-python

implement Tdew to RH conversions #1744

Reasoning

Caveats

Caveat 1

Caveat 2