URL that breaks JSON result only on our own private Amazon instance

[soulgalore]

Hi Pat, I have a problem with two URLs that makes the JSON API being empty on our Amazon instance. When I test it on WebPageTest.org it works fine.

The URLs: https://pt.m.wikipedia.org/wiki/Brasil https://pt.m.wikipedia.org/wiki/Brasil?mobileaction=beta

On our local instance, we send the request, then the result page looks good: http://wpt.wmftest.org/result/160224_0X_6B/

but the JSON is empty: http://wpt.wmftest.org/jsonResult.php?test=160224_0X_6B

I tested on an EC2 instance for WPT.org and there it looks fine: http://www.webpagetest.org/result/160224_VH_RNV/ http://www.webpagetest.org/jsonResult.php?test=160224_VH_RNV

I've been trying to see if I could find something in the log for the server, but the only error l find is a problem with EC2 being down a couple of weeks ago. But it could be I'm looking in the wrong place (/var/www/webpagetest/www/log). Can you send me in the right direction where I should look? Or maybe you know what could be the problem? All our other URLs works fine.

Best Peter

[padibona]

I believe this is the same issue we are running into using the latest build using Azure. The results that WPT create are there, but the JSON embedded in the results are missing, rendering our HAR Storage unable to capture the .HAR. I'll get you a URL ASAP.

Pete On Feb 24, 2016 8:14 AM, "Peter Hedenskog" notifications@github.com wrote:

Hi Pat, I have a problem with two URLs that makes the JSON API being empty on our Amazon instance. When I test it on WebPageTest.org it works fine.

The URLs: https://pt.m.wikipedia.org/wiki/Brasil https://pt.m.wikipedia.org/wiki/Brasil?mobileaction=beta

On our local instance, we send the request, then the result page looks good: http://wpt.wmftest.org/result/160224_0X_6B/

but the JSON is empty: http://wpt.wmftest.org/jsonResult.php?test=160224_0X_6B

I tested on an EC2 instance for WPT.org and there it looks fine: http://www.webpagetest.org/result/160224_VH_RNV/ http://www.webpagetest.org/jsonResult.php?test=160224_VH_RNV

I've been trying to see if I could find something in the log for the server, but the only error l find is a problem with EC2 being down a couple of weeks ago. But it could be I'm looking in the wrong place (/var/www/webpagetest/www/log). Can you send me in the right direction where I should look? Or maybe you know what could be the problem? All our other URLs works fine.

Best Peter

— Reply to this email directly or view it on GitHub https://github.com/WPO-Foundation/webpagetest/issues/569.

[padibona]

OK, I just tested this to ensure I don't waste your time. We are kicking off API calls to this URL: https://store.kaspersky.com/store/kasperus/en_US/buy/productID.275345500/quantity.1 from our Private WPT instance. After the test has completed we try and go to the /jsonResult.php?test=insert_test_number_here and we get an empty file.

We are using Azure and have no other issues with WPT other than this, sweet tool.

Is this the same thing you are encountering Peter?

Thanks,

Pete On Feb 24, 2016 9:06 AM, "Peter DiBona" padibona@gmail.com wrote:

I believe this is the same issue we are running into using the latest build using Azure. The results that WPT create are there, but the JSON embedded in the results are missing, rendering our HAR Storage unable to capture the .HAR. I'll get you a URL ASAP.

Pete On Feb 24, 2016 8:14 AM, "Peter Hedenskog" notifications@github.com wrote:

Hi Pat, I have a problem with two URLs that makes the JSON API being empty on our Amazon instance. When I test it on WebPageTest.org it works fine.

The URLs: https://pt.m.wikipedia.org/wiki/Brasil https://pt.m.wikipedia.org/wiki/Brasil?mobileaction=beta

On our local instance, we send the request, then the result page looks good: http://wpt.wmftest.org/result/160224_0X_6B/

but the JSON is empty: http://wpt.wmftest.org/jsonResult.php?test=160224_0X_6B

I tested on an EC2 instance for WPT.org and there it looks fine: http://www.webpagetest.org/result/160224_VH_RNV/ http://www.webpagetest.org/jsonResult.php?test=160224_VH_RNV

I've been trying to see if I could find something in the log for the server, but the only error l find is a problem with EC2 being down a couple of weeks ago. But it could be I'm looking in the wrong place (/var/www/webpagetest/www/log). Can you send me in the right direction where I should look? Or maybe you know what could be the problem? All our other URLs works fine.

Best Peter

— Reply to this email directly or view it on GitHub https://github.com/WPO-Foundation/webpagetest/issues/569.

[soulgalore]

@padibona yep it looks like the same. One thing: We run a lot of tests and all the other works, it is just these two URLs and we just added them.

[leotsem]

I'm getting the same empty json response when trying to retrieve results of any test run at theguardian.com. Works fine on other URLs.

Agent AMI: ami-a3a81dd0

Job information: Test ID=160320_BB_3 url=http://www.theguardian.com/ Capture Video=1 runs=5 bwIn=5000 bwOut=1000 latency=28 plr=0 browser=Chrome pngScreenShot=1 imageQuality=100 bodies=1 orientation=default

[pmeenan]

Don't know if it will solve the issue but I just pushed some logic to make the json_encode fall back to a user-mode lib that is more resilient in cases where the built-in one fails (same code that was used for the HAR export code).

[pmeenan]

Here was the change: https://github.com/WPO-Foundation/webpagetest/commit/d806312a6b92be903fab6ff6c8814909c4da57f5

[soulgalore]

Cool I'll make a test run tomorrow and check if its work.

[soulgalore]

I get the same on our private instance, how can I be sure that it has the latest updates? I'm not sure I fully understands the update flow.

[pmeenan]

you can check common_lib.inc to see if json_response() includes the modified code.

If you are running the server AMI it is supposed to do a "git pull origin master" hourly but it's possible something is causing that to fail.

[leotsem]

I can confirm https://github.com/WPO-Foundation/webpagetest/commit/d806312a6b92be903fab6ff6c8814909c4da57f5 fixed my problem with http://theguardian.com/.

My server AMI (based on ami-fcfd6194) didn't update it self so I had to manually run git pull origin master. Thanks @pmeenan

[soulgalore]

Yes it works now for us too, thanks!