search

G6EJD / ESP32-e-Paper-Weather-Display

An ESP32 and 2.9", 4.2" or 7.5" ePaper Display reads Weather Underground data via their API and then displays the weather
Other
934 stars 203 forks source link

Weather Display's Imperial Units are out of scale for Pressure Graph #219

Closed ArtieRomero closed 12 months ago

ArtieRomero commented 12 months ago

The Weather Diasplay's three-day atmospheric pressure graph's units start at 30.0 and go to 31.0 in. Our air pressure in St. Louis, Missouri (U.S.A.) has been below 30.0 all summer with the result that no graph gets plotted. I cannot tell if the scale is supposed to auto-scale, but, regardless, can you take a look and perhaps have it auto-scale or use a lower pressure reading at the Y-axis origin? For reference, standard air pressure at sea level is 29.92 inches of mercury; a significant storm would result in the air pressure falling below even that, and also folks at higher elevations would naturally experience a lower air pressure. For reference, I am using the latest build, ver. 16.11, downloaded on July 17, 2023. Thanks

G6EJD commented 12 months ago

There are items you need to check:

  1. That the call to the graph function has Autoscale On, all the examples do. Yes it auto scales the graph.
  2. Monitoring the serial port will show the individual data points that eventually will be plotted, these should indicate the highest and lowest values.
  3. Make sure the location is correct.
ArtieRomero commented 12 months ago

David, thank you for responding. I checked all three items that you suggested and all seem in order (at least to me). Below I pasted-in the relevant information below for the three items. Note that I am using your code that was downloaded yesterday from GitHub and only made the required changes to the user credentials.

But perhaps more telling is that I experimented by solely changing the data units (on line 23 of the credential tab) from "I" (imperial) to "M" (metric), after which the updated display shows a perfectly executed graph. Any thoughts? Thanks, Artie

image

image

Checking the three items:

  1. Referencing code line #'s 395-398, this shows that autoscale is "on" for pressure:

// (x,y,width,height,MinValue, MaxValue, Title, Data Array, AutoScale, ChartMode) DrawGraph(gx + 0 * gap, gy, gwidth, gheight, 900, 1050, Units == "M" ? TXT_PRESSURE_HPA : TXT_PRESSURE_IN, pressure_readings, max_readings, autoscale_on, barchart_off);

DrawGraph(gx + 1 * gap, gy, gwidth, gheight, 10, 30, Units == "M" ? TXT_TEMPERATURE_C : TXT_TEMPERATURE_F, temperature_readings, max_readings, autoscale_on, barchart_off);

DrawGraph(gx + 2 * gap, gy, gwidth, gheight, 0, 100, TXT_HUMIDITY_PERCENT, humidity_readings, max_readings, autoscale_off, barchart_off);

  1. For each time period reported by the API, the Serial Monitor shows a pressure reading. E.g.: Receiving Forecast period - Period-0-------------- DTim: 1689649200 Temp: 78.03 TLow: 69.03 THig: 78.03 Pres: 1014.00 Humi: 73.00 For0: Rain

Icon: 10n Desc: light rain CCov: 21 WSpd: 3.09 WDir: 136.00 Rain: 0.60 Snow: 0.00 Pop: 0.78 Peri: 2023-07-18 03:00:00

Period-1-------------- DTim: 1689660000 Temp: 72.79 TLow: 67.91 THig: 72.79 Pres: 1014.00 Humi: 80.00 For0: Clouds

Icon: 02n Desc: few clouds CCov: 23 WSpd: 3.94 WDir: 104.00 Rain: 0.00 Snow: 0.00 Pop: 0.54 Peri: 2023-07-18 06:00:00

Period-2-------------- etc.

  1. I created several of these to send to my kids and brothers who live all over the U.S., including major cities (e.g. Seattle, Washington). Each of the displays works perfectly to report the weather forecast for the respective cities... except that the 3-day pressure graph seems to have its y-axis frozen at 30.0-31.0 inches for each display (when using imperial units). In all cases, the pressure forecast seems to be for less than 30 inches, which is apparently why the graph figure does not show a forecast curve.
G6EJD commented 12 months ago

I’ve checked the code and can’t find any issues, which location are you using that exhibits this issue?

ArtieRomero commented 12 months ago

I am using the following locations in the U.S. for the several displays that I created (see below). While they each display their respective weather for their locales, they all seem to have the 3-day pressure graphs locked at 30.0-31.0 inches for the y-axis.

Perhaps another clue is that I first created a prototype in early February and took a photo of it, and looking back at the photo, I see that its pressure graph was autoscaling properly. However, a few weeks ago, after I created the other two displays and inserted them into frames, I a) reinstalled your weather display program from GitHub to the Arduino IDE and installed/reinstalled it in all of the units, b) I registered for a new Openweathermap.org API account under a new email address and used API account codes from this newer account for the three displays, and c) used the newest versions of the several dependent library sketches. That is when I noticed that the Pressure graph was not working properly.

However, before contacting you, for b) and c), I went back and reinstalled the new Weather DIsplay code afresh and installed the earlier versions of the dependent libraries and then changed them back again - this did not make any difference to the behavior of the Pressure graph.

The locales that I am using: Wildwood, MO (Central time) Seattle, WA (Pacific time) Sarasota, FL (Eastern time)

Cheers, Artie

On Thu, Jul 20, 2023 at 3:33 AM G6EJD @.***> wrote:

I’ve checked the code and can’t find any issues, which location are you using that exhibits this issue?

— Reply to this email directly, view it on GitHub https://github.com/G6EJD/ESP32-e-Paper-Weather-Display/issues/219#issuecomment-1643506913, or unsubscribe https://github.com/notifications/unsubscribe-auth/BBK2GRJOEAEVOQKJMOZBYMLXRDUNJANCNFSM6AAAAAA2NWJ6PE . You are receiving this because you authored the thread.Message ID: @.***>

G6EJD commented 12 months ago

OK, I’ll check them out, noting not every location has weather data on OWM.

ArtieRomero commented 12 months ago

David,

Some items to note: a) When I switch the measurement units from "I" (imperial) to "M" (metric) in the credentials tab of the Weather Display program, I see the 3-day pressure displayed as a fully graphed curve (in metric units, of course). So the OWM source must be providing atmospheric pressure information for my locale. I.e. the issue seems to reside in the Weather Display program. Maybe try switching your display's units "I" to see what happens to the pressure graph?

b) I mentioned that back in February, when first getting the Weather Display up and running as a prototype, I did have the 3-day pressure graph working, including auto-scaling the y-axis.
If you look at the old photo pasted in below, taken in February, you can see that the pressure graph is working perfectly, including autoscaling the y-axis (in this photo the y-axis goes from 29.0-31.0). In my current display, the y-axis seems to be stuck at 30.0-31.0 with the result that the local weather's <30 inch ambient pressure graph is not appearing.

Referring to this working February display's image and comparing it to the current display that is not working: -the local, Wildwood, MO, is the same locale that I am currently monitoring -the Weather Display software, ver. 16.11, is the same version that I am currently using (although I did download a new copy from GitHub a few weeks ago that is installed in the problematic display) The only thing that is different is that I am using a different OWM API key # now that was generated from a new email account registration

image

Thanks for your help. Cheers, Artie

ArtieRomero commented 12 months ago

David, please read the previous input that I provide in the previous message, where I indicate that when I uploaded code back in February the pressure graph was autoscaling properly (see photo above, which shows a y-axis from 29.0-31.0).

I just deleted all-things related to Arduino IDE and sketches from my PC and performed a fresh install of the IDE, the Lolin D32 board definitions, the weather sketch, and dependent libraries. Everything. Then, to reinstall the weather display program, I located in my computer's downloads folder the original zipped copy from GitHub that I downloaded in late January and used this. I used the same versions of the dependent libraries that I originally used (I recorded this in my notes) and also used the OWM API key that I originally used. Overkill, I know, but I went back to where I knew the pressure autoscale was working (see photo in above message).

The result after flashing this to my ePaper display? The autoscale is still not working and the y-axis still reads from 30.0-31.0. I can see just a little bit of the forecast pressure curve peeking above 30.0 inches later in the week, so the API is indeed sending the forecast.

HOWEVER, I did find the interesting results below when playing with code line 395 in the "DisplayForecastSection" ( the code line is:
if (Units == "I") pressure_readings[r] = WxForecast[r].Pressure * 0.02953; else pressure_readings[r] = WxForecast[r].Pressure; )

The value 0.02953 is used in the code to convert the pressure's hPa units to inches. Here is the result as I incrementally changed the original 0.02953 value to smaller and then greater values:

It appears that there is something weird going on with the autoscaling, where it will not go below 30.0, but will set the upper end higher.

Cheers, Artie

G6EJD commented 12 months ago

OK, I have conducted extensive testing and cannot see any issues with the code. For your location just now, I get: 0=30.002 1=30.032 2=30.032 3=29.973 4=29.943 5=29.973 6=29.973 7=29.973 8=29.973 9=30.002 10=29.973 11=29.914 12=29.884 13=29.884 14=29.914 15=29.914 16=29.943 17=29.973 18=29.943 19=29.884 20=29.855 21=29.914 22=29.943 23=29.943 And in the graph window, there are 72 pixels allocated to display a range of 30.002 to 29.9943, that's a difference of 0.0077 inHg and so the pressure appears as a straight-line, which is does on mine.

The other compounding problem is OWM only issue air pressure in integer hPa so when converted to inHG you lose a lot of resolution, maybe time for the US to become metric!

Are you using an ESP32? and no other derivative that may be short on memory.

ArtieRomero commented 12 months ago

I am using the Lolin D32.
What is interesting is that simply multiplying the OWM's pressure, given in hPa, by a value somewhat larger than the proper 0.02953 conversion to inches (e.g. multiplying by 0.03100) results in a proper autoscaling of the graph.

I suppose a work-around for me might be to simply make this a permanent change in the code for my display so that the curve is properly graphed (auto-scrolled y-axis and the curve fully showing in the graph). Much of the value of the 3-day graphs is the visual observation of trends.

Total agreement from me about the U.S. needing to go metric (I hold a Ph.D. in chemistry and am no stranger to metric!).

G6EJD commented 12 months ago

Which compiler versions are you using?

ArtieRomero commented 12 months ago

How do I determine the compiler version?

ArtieRomero commented 12 months ago

I am using Arduino IDE 2.1.1, the current release

ArtieRomero commented 12 months ago

I just tried replacing the conversion value for hPa to inches with a value where it is doubled. The result is that the pressure graph has a y-axis from 60.0-61.0. And the graphed curve barely shows above the origin and even then, just briefly. The remainder of the time the curve is below the x-axis.

So doubling to obtain a full curve displayed won't work.

G6EJD commented 12 months ago

Your IDE displays the version on the top line. The ESP32 add-on is found in Boards, Boards Manager, then type ESP32 in the search box

ArtieRomero commented 12 months ago

The IDE version is the newest, ver. 2.1.1.
And I am using the Lolin D32 board and selected this in the Boards Manager.

I have a copy of the older IDE version (ver. 1.8.19) installed on another computer and tonight I will download the Weather Display code from GitHub and go through the set-up to flash it to the display. I will let you know if that makes a diffrence.

Other than that, all I can think of is to use older versions of the other libraries that the Weather Display code calls.

What is perplexing is that, as the photo from February shows, the pressure graph was behaving properly when I was initially prototyping the Weather Display. To my knowledge, when I recently repeated the set-up from scratch (but using the same Arduino IDE version that was previously installed) I freshly downloaded same the Weather Display code version from GitHub and installed the same version libraries (I noted the versions when prototyping in February). I have also now gone back and installed the most recent library versions (where there has been an update). Nothing changes.

G6EJD commented 12 months ago

The common theme is the drawgraph function that has never been changed, so I think you’re trying to display air pressure changes that in the real world are irrelevant. The code is correctly converted pressure in hPa to inHG then finding g the minima and maxima, but as OWM only publish in hPa the down conversion to inHG loses resolution by a factor of about 30, also what the drawgraph function tries to do is add some display headroom at both the top and bottom, but these are integer values so this will cause further compression of change. It looks like the pressure in your region is very steady.

ArtieRomero commented 12 months ago

Hmm. From your study and comment, I am now wondering if the autoscaler is working ok and it is just that there is little variation in the pressure for the scale being used AND that the autoscale builds the curve off of the x-axis line. So while it appears that there is only a little bit of curve being plotted, it is really just that most of the curve is laying on top of the x-axis line. It appears (?) that the minimum scale for autoscaling is one integer unit (thus the range shown is 30.0-31.0).

Option 1: Can the pressure graph be stitched over to metric? Since the metric mPa scale works so well, is it possible to just that metric scale for pressure but keep the imperial units for all other weather data?

Option 2: Use a fixed y-axis scale for pressure in imperial units, possibly going from 28.5-31.0? Separately, wondering if a fixed inches scale would work better, it would result in a narrow amplitude for the pressure curve. I searched online and the lowest pressure recorded in the lower 48 states is 27.17 during Hurricane Katrina (the eye of the Cat. 5 hurricane), and the highest value is 31.42 in Montana during an incredibly cold winter. If these extremes are allowed to go "off-scale" and we focus on more normal pressure, including in the mountain and coastal cities, a range of 28.5-31 seems acceptable (with 28.5 being the lower end of pressure for the eye of a cat. 2 hurricane. Do you think a 1.5 unit scale would give enough resolution for normal daily pressure variations?

On Fri, Jul 21, 2023 at 5:56 AM G6EJD @.***> wrote:

OK, I have conducted extensive testing and cannot see any issues with the code. For your location just now, I get: 0=30.002 1=30.032 2=30.032 3=29.973 4=29.943 5=29.973 6=29.973 7=29.973 8=29.973 9=30.002 10=29.973 11=29.914 12=29.884 13=29.884 14=29.914 15=29.914 16=29.943 17=29.973 18=29.943 19=29.884 20=29.855 21=29.914 22=29.943 23=29.943 And in the graph window, there are 72 pixels allocated to display a range of 30.002 to 29.9943, that's a difference of 0.0077 inHg and so the pressure appears as a straight-line, which is does on mine.

Are you using an ESP32? and no other derivative that may be short on memory.

— Reply to this email directly, view it on GitHub https://github.com/G6EJD/ESP32-e-Paper-Weather-Display/issues/219#issuecomment-1645394244, or unsubscribe https://github.com/notifications/unsubscribe-auth/BBK2GRIQO6H66G7TPOQX5PTXRJN4HANCNFSM6AAAAAA2NWJ6PE . You are receiving this because you authored the thread.Message ID: @.***>

ArtieRomero commented 12 months ago

Agreed. I think that the program is working properly, it is just being fed with a too-narrow range of data to provide a visible curve. It would be better, though, if the curve was plotted more toward the center of the graph instead of on top of the x-axis.

What do you think about the comments that I just made to improve the readability of the graph for us folks in imperial-units land? 1) set this particular graph to metric units with the result that the graphed curve looks good and amply shows the pressure variation 2) stick with imperial units and fix the scale from 28.5-31.0 (or something similar) so that the curve is not plotted on the x-axis and most weather conditions remain on-scale?

On Fri, Jul 21, 2023 at 11:39 AM G6EJD @.***> wrote:

The common theme is the drawgraph function that has never been changed, so I think you’re trying to display air pressure changes that in the real world are irrelevant. The code is correctly converted pressure in hPa to inHG then finding g the minima and maxima, but as OWM only publish in hPa the down conversion to inHG loses resolution by a factor of about 30, also what the drawgraph function tries to do is add some display headroom at both the top and bottom, but these are integer values so this will cause further compression of change. It looks like the pressure in your region is very steady.

— Reply to this email directly, view it on GitHub https://github.com/G6EJD/ESP32-e-Paper-Weather-Display/issues/219#issuecomment-1645968069, or unsubscribe https://github.com/notifications/unsubscribe-auth/BBK2GRJVLS4ZLTD24MJBVM3XRKWCNANCNFSM6AAAAAA2NWJ6PE . You are receiving this because you authored the thread.Message ID: @.***>

G6EJD commented 12 months ago

I've been doing further checks: These are the values received from OWM 0=30.032 1=30.002 2=29.973 3=29.943 4=29.943 5=29.943 6=29.973 7=29.973 8=29.943 9=29.884 10=29.884 11=29.884 12=29.884 13=29.884 14=29.943 15=29.973 16=29.943 17=29.884 18=29.884 19=29.914 20=29.943 21=29.973 22=30.032 23=30.032 Drawing graph... Then it determines maxima and minima Y Graph scale maximum=30.50 Y Graph scale minimum=30.00 Then it scales the data to fit the screen space x = 46.00 y = 443.39 x = 52.00 y = 447.64 x = 59.00 y = 448.00 x = 65.00 y = 448.00 x = 72.00 y = 448.00 x = 78.00 y = 448.00 x = 85.00 y = 448.00 x = 91.00 y = 448.00 x = 98.00 y = 448.00 x = 104.00 y = 448.00 x = 111.00 y = 448.00 x = 117.00 y = 448.00 x = 124.00 y = 448.00 x = 130.00 y = 448.00 x = 137.00 y = 448.00 x = 143.00 y = 448.00 x = 150.00 y = 448.00 x = 156.00 y = 448.00 x = 163.00 y = 448.00 x = 169.00 y = 448.00 x = 176.00 y = 448.00 x = 182.00 y = 448.00 x = 189.00 y = 443.39 x = 196.00 y = 443.39 Then it calculates the y-axis units labels. Y-axis spacing values: 30.5 Y-axis spacing values: 30.4 Y-axis spacing values: 30.3 Y-axis spacing values: 30.2 Y-axis spacing values: 30.1 Y-axis spacing values: 30.0

It adds 0.5 to the range.

You can switch to Metric pressure units like this: In DisplayForecastSection: if (Units == "I") pressure_readings[r] = WxForecast[r].Pressure;

DrawGraph(gx + 0 * gap, gy, gwidth, gheight, 900, 1050, TXT_PRESSURE_HPA, pressure_readings, max_readings, autoscale_on, barchart_off);

ArtieRomero commented 12 months ago

David,

I will go ahead and use the metric version code that you provided. I think the change and direction of the pressure is as (or more) valuable than the precise value at a moment in time, and this is displayed well in metric units..

It is interesting that the display, back in February, showed a well-formed graph curve, but it had greater variation during the 3-day forecast than we are presently experiencing (hot, humid, with no significant large weather systems). In the spring we get a lot of tornados (the entire St. Louis, MO metro region, including the outer suburbs, is outfitted with tornado sirens). So the bottom line is that the pressure right now is uneventful and the graph does represent this, it is just hard to read with the curve being buried on the x-axis.

Thanks for helping me understand this and for providing the metric code. Cheers, Artie

On Fri, Jul 21, 2023 at 12:37 PM G6EJD @.***> wrote:

I've been doing further checks: These are the values received from OWM 0=30.032 1=30.002 2=29.973 3=29.943 4=29.943 5=29.943 6=29.973 7=29.973 8=29.943 9=29.884 10=29.884 11=29.884 12=29.884 13=29.884 14=29.943 15=29.973 16=29.943 17=29.884 18=29.884 19=29.914 20=29.943 21=29.973 22=30.032 23=30.032 Drawing graph... Then it determines maxima and minima Y Graph scale maximum=30.50 Y Graph scale minimum=30.00 Then it scales the data to fit the screen space x = 46.00 y = 443.39 x = 52.00 y = 447.64 x = 59.00 y = 448.00 x = 65.00 y = 448.00 x = 72.00 y = 448.00 x = 78.00 y = 448.00 x = 85.00 y = 448.00 x = 91.00 y = 448.00 x = 98.00 y = 448.00 x = 104.00 y = 448.00 x = 111.00 y = 448.00 x = 117.00 y = 448.00 x = 124.00 y = 448.00 x = 130.00 y = 448.00 x = 137.00 y = 448.00 x = 143.00 y = 448.00 x = 150.00 y = 448.00 x = 156.00 y = 448.00 x = 163.00 y = 448.00 x = 169.00 y = 448.00 x = 176.00 y = 448.00 x = 182.00 y = 448.00 x = 189.00 y = 443.39 x = 196.00 y = 443.39 Then it calculates the y-axis units labels. Y-axis spacing values: 30.5 Y-axis spacing values: 30.4 Y-axis spacing values: 30.3 Y-axis spacing values: 30.2 Y-axis spacing values: 30.1 Y-axis spacing values: 30.0

It adds 0.5 to the range.

You can switch to Metric pressure units like this: In DisplayForecastSection: if (Units == "I") pressure_readings[r] = WxForecast[r].Pressure;

DrawGraph(gx + 0 * gap, gy, gwidth, gheight, 900, 1050, TXT_PRESSURE_HPA, pressure_readings, max_readings, autoscale_on, barchart_off);

— Reply to this email directly, view it on GitHub https://github.com/G6EJD/ESP32-e-Paper-Weather-Display/issues/219#issuecomment-1646035984, or unsubscribe https://github.com/notifications/unsubscribe-auth/BBK2GRKWB6FZLL5CEUKBFMTXRK43ZANCNFSM6AAAAAA2NWJ6PE . You are receiving this because you authored the thread.Message ID: @.***>

G6EJD commented 12 months ago

OK, understood, you’re battling with screen space and resolution and inHG is not well suited to detecting change. Things would be different if OWM issued decimal pressure values.

ArtieRomero commented 11 months ago

David,

I promised to pass along what I learned from troubleshooting the Weather Displays that I made that would not update.

I made 4 of these, keeping one and sending the others to my distant brothers. They are identical and were flashed with the same sketch, back to back, with the only difference being the edits that I made to the owm_credentials sketch page. My unit works perfectly, as does one of the gifted units. The other two brothers' units did not work with their respective home WiFis.

For one brother, he sent his unit back and I reflashed it and returned it to him and it is still not working. My other brother with a non-working unit is more tech-savvy and we decided to troubleshoot by phone. But first I emaile him a copy-paste of the own_credentials page used in his unit (as a Word doc) and he verified that his WiFi logon credentials were correct.

Talking this over the phone and with him using a very old Windows laptop with the Weather Display connected, he installed Arduino IDE and the ESP32 board libraries, selecting Lolin D32, and we used Arduino's WiFiscan to verify that its ESP32 was able to see the 2.4 GHz WiFi signal.

I had already mailed him a memory key containing the sketch file of your weather display program. We then downloaded from the internet and installed the other code libraries that your Weather Display requires, but not your GitHub program. The latter was because he wanted to just copy it from the memory key to avoid the required setup changes in the owm_credentials sketch page.

He then dug out my configured copy of your Weather Display "sketch" that I sent him on a memory key and copied this and pasted it into a blank sketch on the Arduino IDE. After flashing this to his Weather Display device, it worked. It has now been automatically updated several times using battery power.

Curious is that the fully configured program and board information that I flashed to the Weather Display that I sent him, which did not work, has been re-flashed to my own otherwise identical Weather Display while I was troubleshooting what could be wrong with his, and my Weather Display has always worked. And one brother's worked immediately upon turning it on. So the issue seems to be not uniformly impactful across users' WiFis. I now have 3/4 Weather Displays working using what would appear to be identical set-up steps.

The upshot seems to be an issue in the ESP32 board definitions or in one of the dependent libraries that were installed. Perhaps a corrupted file. But it did not evidence itself using my home Wifi or on the gifted display that worked without issue. For the one that remains non-working, I am going to have the brother send it back and I am going to use a new ThinkPad and start from scratch and send the device back to him and see if it will work and am optimistic.