search

rszimm / sprinklers_pi

Sprinkling System Control Program for the Raspberry Pi
GNU General Public License v2.0
310 stars 100 forks source link

WUnderground Failure #58

Closed Skispcs closed 7 years ago

Skispcs commented 8 years ago

A system that was working has recently started to get errors, "Invalid Response from WUnderground server!" when running diagnostic. Using a PWS and the settings page is configured with the IP 104.72.15.199. api.wunderground.com does not resolve to that IP but then it has not from the first time I have installed the program tail /var/log/sprinklers_pi after running diagnostic shows the entry: Bad Wunderground Response.

I assume that the WUnderground has changed something....

nhorvath commented 8 years ago

I suggest you update your ip to whatever you currently get for api.wunderground.com They probably just switched servers around.

On Thu, Apr 28, 2016 at 5:54 PM, Skispcs notifications@github.com wrote:

A system that was working has recently started to get errors, "Invalid Response from WUnderground server!" when running diagnostic. Using a PWS and the settings page is configured with the IP 104.72.15.199. api.wunderground.com does not resolve to that IP but then it has not from the first time I have installed the program tail /var/log/sprinklers_pi after running diagnostic shows the entry: Bad Wunderground Response.

I assume that the WUnderground has changed something....

— You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub https://github.com/rszimm/sprinklers_pi/issues/58

Skispcs commented 8 years ago

Thank you for the response. I tried that and I get the same symptoms. Pinging api.wundeground.com currently gives me 23.46.166.236 and using that IP also fails.

marksev1 commented 8 years ago

Why can't this system simply be pointed to wundergrounds url? Is that planned as an enhancement?

nhorvath commented 8 years ago

Doing a DNS lookup in C is unnecessarily complicated. No one has had time.

On Wed, May 11, 2016 at 5:55 AM, marksev1 notifications@github.com wrote:

Why can't this system simply be pointed to wundergrounds url? Is that planned as an enhancement?

— You are receiving this because you commented. Reply to this email directly or view it on GitHub https://github.com/rszimm/sprinklers_pi/issues/58#issuecomment-218414388

marksev1 commented 8 years ago

So why are some people having issues then? I'd like to be prepared in advance before i start this project.

2016-05-11 14:32 GMT+02:00 Nick Horvath notifications@github.com:

Doing a DNS lookup in C is unnecessarily complicated. No one has had time.

On Wed, May 11, 2016 at 5:55 AM, marksev1 notifications@github.com wrote:

Why can't this system simply be pointed to wundergrounds url? Is that planned as an enhancement?

— You are receiving this because you commented. Reply to this email directly or view it on GitHub < https://github.com/rszimm/sprinklers_pi/issues/58#issuecomment-218414388>

— You are receiving this because you commented. Reply to this email directly or view it on GitHub https://github.com/rszimm/sprinklers_pi/issues/58#issuecomment-218445445

nhorvath commented 8 years ago

I've been using 38.102.136.138 for years without issue. The problem is api.wunderground.com resolves in round robin to many IP addresses. Some of those addresses go away occasionally.

On Wed, May 11, 2016 at 8:39 AM, marksev1 notifications@github.com wrote:

So why are some people having issues then? I'd like to be prepared in advance before i start this project.

2016-05-11 14:32 GMT+02:00 Nick Horvath notifications@github.com:

Doing a DNS lookup in C is unnecessarily complicated. No one has had time.

On Wed, May 11, 2016 at 5:55 AM, marksev1 notifications@github.com wrote:

Why can't this system simply be pointed to wundergrounds url? Is that planned as an enhancement?

— You are receiving this because you commented. Reply to this email directly or view it on GitHub < https://github.com/rszimm/sprinklers_pi/issues/58#issuecomment-218414388

— You are receiving this because you commented. Reply to this email directly or view it on GitHub < https://github.com/rszimm/sprinklers_pi/issues/58#issuecomment-218445445>

— You are receiving this because you commented. Reply to this email directly or view it on GitHub https://github.com/rszimm/sprinklers_pi/issues/58#issuecomment-218446903

Skispcs commented 8 years ago

The problem is api.wunderground.com resolves in round robin to many IP addresses. Some of those addresses go away occasionally.

I respectfully disagree that the problem is an IP address resolution issue. I have taken the IP that responds to the ping at api.wunderground.com and inserted it into the json request url documented in the api documentation and I get a valid response in the browser. I them put that IP in the program and get "Invalid response" in sprinkler_pi.

e.g. If I put 38.102.136.138 in the program and check it will fail. If enter the same IP in this URL from the api documentation and it works. http://38.102.136.138/api/214745275e67dafb/conditions/q/pws:KVAVIRGI39.json

Therefore the problem is NOT the IP address but it is either the request or parsing the response. Looking at my wunderground analytics page does not show many requests so I am assuming for now that it is the request going out to the api that is the problem.

Skispcs commented 8 years ago

Doing a DNS lookup in C is unnecessarily complicated.

getaddrinfo() http://www.beej.us/guide/bgnet/output/html/singlepage/bgnet.html#getaddrinfo

geovanni2 commented 8 years ago

I just got my Pi connected to wunderground. nslookup api.wunderground.com = 104.73.96.110. Make sure you typed your personal API key correctly.

madisdev commented 7 years ago

I think this issue is related to the Akamai dns servers. If I test the request in the browser it works with the DNS name, but does not with the IP. I think Akamai is inspecting the host name sent across the wire and only accepts host names not IP's. I will look at the code and see if its something I can contribute.

This is the error I get when using IP. "The requested URL "[no URL]", is invalid."

I also verified it doesn't work in postman with setting Host header.

nhorvath commented 7 years ago

Ideally it would make a proper http request to api.wunderground.com but it's complicated to do that in C code. A compromise might be just adding a header for Host: api.wunderground.com which should satisfy akamai.

On Thu, Apr 20, 2017 at 2:11 PM, madisdev notifications@github.com wrote:

I think this issue is related to the Akamai dns servers. If I test the request in the browser it works with the DNS name, but does not with the IP. I think Akamai is inspecting the host name sent across the wire and only accepts host names not IP's. I will look at the code and see if its something I can contribute.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/rszimm/sprinklers_pi/issues/58#issuecomment-295839947, or mute the thread https://github.com/notifications/unsubscribe-auth/AAKE3C6bzcs-mjuQloN70WKfj4PnKriZks5rx5_GgaJpZM4ISXeh .

madisdev commented 7 years ago

I tried that through postman, but it still gave the same error. Its weird to say the least.

nhorvath commented 7 years ago

It's possible that the ip you have is no longer assigned to wunderground... are you using a fresh ip from dig or ping api.wunderground.com?

On Thu, Apr 20, 2017 at 2:32 PM, madisdev notifications@github.com wrote:

I tried that through postman, but it still gave the same error. Its weird to say the least.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/rszimm/sprinklers_pi/issues/58#issuecomment-295848166, or mute the thread https://github.com/notifications/unsubscribe-auth/AAKE3IoGDrEqQLOJjYGuK1X481zqTVcwks5rx6SrgaJpZM4ISXeh .

madisdev commented 7 years ago

Ok so using the IP. Its weird it works when I use the ZIP in the sprinklers_pi UI, but not the PWS.

nhorvath commented 7 years ago

Strange because I use pws... I'll check which ip later.

On Apr 20, 2017 3:40 PM, "madisdev" notifications@github.com wrote:

Ok so using the IP. Its weird it works when I use the ZIP in the sprinklers_pi UI, but not the PWS.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/rszimm/sprinklers_pi/issues/58#issuecomment-295875400, or mute the thread https://github.com/notifications/unsubscribe-auth/AAKE3J61mCRzgKDEA47Rjj4I9oNALL1pks5rx7S5gaJpZM4ISXeh .

Skispcs commented 7 years ago

I get the same behavior, using the zip code, I get weather data. When I enter the PWS, I get nothing and the diagnostics fail with error "Invalid Response from WUnderground server!".

nhorvath commented 7 years ago

I am using pws with 172.224.165.103

On Apr 20, 2017 6:29 PM, "Skispcs" notifications@github.com wrote:

I get the same behavior, using the zip code, I get weather data. When I enter the PWS, I get nothing and the diagnostics fail with error "Invalid Response from WUnderground server!".

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/rszimm/sprinklers_pi/issues/58#issuecomment-295944831, or mute the thread https://github.com/notifications/unsubscribe-auth/AAKE3Dv_zlV3ogjfBxl66qPBTzbt9Rm1ks5rx9xIgaJpZM4ISXeh .

Lynnpaskett commented 7 years ago

I too am having the error but what i see is that the API Key is 32 long and the app only uses 16 am i missing something here?

nhorvath commented 7 years ago

Both of my keys, one which I just generated are 16 characters. Please double check your key settings on http://api.wunderground.com/weather/api Generate (Purchase for $0) a Developer level "Anvil Plan" key.

On Thu, May 18, 2017 at 10:54 PM, Lynnpaskett notifications@github.com wrote:

I too am having the error but what i see is that the API Key is 32 long and the app only uses 16 am i missing something here?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/rszimm/sprinklers_pi/issues/58#issuecomment-302595391, or mute the thread https://github.com/notifications/unsubscribe-auth/AAKE3Nw9Fi8IFSRDbNPKPk1NaYGEy5xZks5r7QRSgaJpZM4ISXeh .

Lynnpaskett commented 7 years ago

Some how i got to the OpenWeathermap api key and it is 32 you are so right my bad thanks But it is still giving the errror :(

nhorvath commented 7 years ago

Does the pws you have selected work when you use the test page in the api docs?

On May 18, 2017 11:47 PM, "Lynnpaskett" notifications@github.com wrote:

Some how i got to the OpenWeathermap api key and it is 32 you are so right my bad thanks But it is still giving the errror :(

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/rszimm/sprinklers_pi/issues/58#issuecomment-302601488, or mute the thread https://github.com/notifications/unsubscribe-auth/AAKE3KTScB-sgpriSPwNUITHvt8SvuE6ks5r7RDngaJpZM4ISXeh .

Lynnpaskett commented 7 years ago

I do not have a pws . the app was working fine without it until recently

nhorvath commented 7 years ago

So you are using zip code?

On May 19, 2017 8:46 AM, "Lynnpaskett" notifications@github.com wrote:

I do not have a pws . the app was working fine without it until recently

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/rszimm/sprinklers_pi/issues/58#issuecomment-302693267, or mute the thread https://github.com/notifications/unsubscribe-auth/AAKE3G2Aqtx1NLmNmeed3FZsGChasbLEks5r7Y8xgaJpZM4ISXeh .

nhorvath commented 7 years ago

What ip are you using?

On May 19, 2017 8:51 AM, "Nick Horvath" nhorvath@gmail.com wrote:

So you are using zip code?

On May 19, 2017 8:46 AM, "Lynnpaskett" notifications@github.com wrote:

I do not have a pws . the app was working fine without it until recently

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/rszimm/sprinklers_pi/issues/58#issuecomment-302693267, or mute the thread https://github.com/notifications/unsubscribe-auth/AAKE3G2Aqtx1NLmNmeed3FZsGChasbLEks5r7Y8xgaJpZM4ISXeh .

Lynnpaskett commented 7 years ago

Yes zip and 3.202.233.120 but i just now did nslookup api.wunderground.com and got 184.85.205.78 tried it but still no good Invalid Response from WUnderground server!

nhorvath commented 7 years ago

Try the IP i listed a few posts up

On Fri, May 19, 2017 at 8:58 AM, Lynnpaskett notifications@github.com wrote:

Yes zip and 3.202.233.120 but i just now did nslookup api.wunderground.com and got 184.85.205.78 tried it but still no good

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/rszimm/sprinklers_pi/issues/58#issuecomment-302695932, or mute the thread https://github.com/notifications/unsubscribe-auth/AAKE3Pa4RqB8hVEp50Nu-rW_zGLD45Umks5r7ZH-gaJpZM4ISXeh .

Lynnpaskett commented 7 years ago

172.224.165.103 still error

madisdev commented 7 years ago

Hello. When I had the same issue with the PWS I dug into the code a little. From my investigation it seemed to be an issue with weatherundergrounds CDN. They don't seem to like the web request for some reason.

So I was looking at the spec for HTTP GET(https://www.w3.org/Protocols/rfc2616/rfc2616-sec5.html) specifically section 5.1.2.And this part in particular "The absoluteURI form is REQUIRED when the request is being made to a proxy." Maybe we should change the GET to send across the full uri instead of just the host? I can test this out tonight if noone else can.

Again I'm not criticizing the code. Just suggesting possible fix.

madisdev commented 7 years ago

94

See my pull request, that should resolve IP resolution and proxy issues.