search

percy / percy-capybara

Visual testing for Capybara with Percy.
https://docs.percy.io/docs/capybara
MIT License
45 stars 23 forks source link

Images missing in snapshots even though webserver logs indicate they are being loaded #44

Closed rsimha closed 7 years ago

rsimha commented 7 years ago

We on the ampproject/amphtml team use Percy for visual diff testing. We're having trouble with the Percy / Capybara / Phantomjs stack, where webpage images are missing in all our visual diff snapshots. However, the webserver logs indicate that the images are being loaded, even though they don't appear in the screenshots.

I've added waits between loading the page and taking the snapshot, but this doesn't seem to make any difference.

More details can be found in https://github.com/ampproject/amphtml/issues/10156, including webserver logs interspersed with percy / capybara logs. I'd appreciate any help in getting to the bottom of this.

Thanks!

rsimha commented 7 years ago

In case it helps, here are detailed logs with debugging enabled for Percy and Phantomjs:

ruby build-system/tasks/visual-diff.rb --percy_debug --phantomjs_debug
[12:55:55] Using gulpfile ~/src/amphtml/gulpfile.js
[12:55:55] Starting 'serve'...
[12:55:55] Serving unminified js
[12:55:55] Run `gulp build` then go to http://localhost:8000/examples/article.amp.html
[12:55:55] Finished 'serve' after 2.6 ms
[12:55:55] Webserver started at http://localhost:8000
HEAD / 200 27.498 ms - 12481
HEAD / 200 5.249 ms - 12481
[percy][DEBUG] Using filesystem_loader to discover assets.
[percy][DEBUG] Build resource: /amp-by-example/amp-by-example.html
[percy][DEBUG] Build resource: /amp-by-example/amp_by_example_logo.svg
[percy][DEBUG] Build resource: /article.amp/img/sea@1x.jpg
[percy][DEBUG] Build resource: /article.amp/img/hero@1x.jpg
[percy][DEBUG] Build resource: /article.amp/img/sample.jpg
[percy][DEBUG] Build resource: /article.amp/img/hero@2x.jpg
[percy][DEBUG] Build resource: /article.amp/img/sea@2x.jpg
[percy][DEBUG] Build resource: /article.amp/article.amp.html
[percy][DEBUG] All build resources loaded (0.009923793s)
[percy][DEBUG] Build 278235 created
[percy][DEBUG] Parallel test environment: not detected
2017-07-06T12:55:57 [DEBUG] CookieJar - Created but will not store cookies (use option '--cookies-file=<filename>' to enable persistent cookie storage)
2017-07-06T12:55:57 [DEBUG] Set  "http"  proxy to:  "" : 1080
2017-07-06T12:55:57 [DEBUG] Phantom - execute: Configuration
2017-07-06T12:55:57 [DEBUG]      0 objectName : ""
2017-07-06T12:55:57 [DEBUG]      1 cookiesFile : ""
2017-07-06T12:55:57 [DEBUG]      2 diskCacheEnabled : "false"
2017-07-06T12:55:57 [DEBUG]      3 maxDiskCacheSize : "-1"
2017-07-06T12:55:57 [DEBUG]      4 diskCachePath : ""
2017-07-06T12:55:57 [DEBUG]      5 ignoreSslErrors : "true"
2017-07-06T12:55:57 [DEBUG]      6 localUrlAccessEnabled : "true"
2017-07-06T12:55:57 [DEBUG]      7 localToRemoteUrlAccessEnabled : "false"
2017-07-06T12:55:57 [DEBUG]      8 outputEncoding : "UTF-8"
2017-07-06T12:55:57 [DEBUG]      9 proxyType : "http"
2017-07-06T12:55:57 [DEBUG]      10 proxy : ":1080"
2017-07-06T12:55:57 [DEBUG]      11 proxyAuth : ":"
2017-07-06T12:55:57 [DEBUG]      12 scriptEncoding : "UTF-8"
2017-07-06T12:55:57 [DEBUG]      13 webSecurityEnabled : "true"
2017-07-06T12:55:57 [DEBUG]      14 offlineStoragePath : ""
2017-07-06T12:55:57 [DEBUG]      15 localStoragePath : ""
2017-07-06T12:55:57 [DEBUG]      16 localStorageDefaultQuota : "-1"
2017-07-06T12:55:57 [DEBUG]      17 offlineStorageDefaultQuota : "-1"
2017-07-06T12:55:57 [DEBUG]      18 printDebugMessages : "true"
2017-07-06T12:55:57 [DEBUG]      19 javascriptCanOpenWindows : "true"
2017-07-06T12:55:57 [DEBUG]      20 javascriptCanCloseWindows : "true"
2017-07-06T12:55:57 [DEBUG]      21 sslProtocol : "tlsv1"
2017-07-06T12:55:57 [DEBUG]      22 sslCiphers : "ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES128-SHA:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-RC4-SHA:ECDHE-RSA-RC4-SHA:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES128-SHA:AES256-SHA:DES-CBC3-SHA:RC4-SHA:RC4-MD5"
2017-07-06T12:55:57 [DEBUG]      23 sslCertificatesPath : ""
2017-07-06T12:55:57 [DEBUG]      24 sslClientCertificateFile : ""
2017-07-06T12:55:57 [DEBUG]      25 sslClientKeyFile : ""
2017-07-06T12:55:57 [DEBUG]      26 sslClientKeyPassphrase : ""
2017-07-06T12:55:57 [DEBUG]      27 webdriver : ":"
2017-07-06T12:55:57 [DEBUG]      28 webdriverLogFile : ""
2017-07-06T12:55:57 [DEBUG]      29 webdriverLogLevel : "INFO"
2017-07-06T12:55:57 [DEBUG]      30 webdriverSeleniumGridHub : ""
2017-07-06T12:55:57 [DEBUG] Phantom - execute: Script & Arguments
2017-07-06T12:55:57 [DEBUG]      script: "/usr/local/google/home/rsimha/.rvm/gems/ruby-2.4.0/gems/poltergeist-1.15.0/lib/capybara/poltergeist/client/compiled/main.js"
2017-07-06T12:55:57 [DEBUG]      0 arg: "33270"
2017-07-06T12:55:57 [DEBUG]      1 arg: "1024"
2017-07-06T12:55:57 [DEBUG]      2 arg: "768"
2017-07-06T12:55:57 [DEBUG]      3 arg: "127.0.0.1"
2017-07-06T12:55:57 [DEBUG] Phantom - execute: Starting normal mode
2017-07-06T12:55:57 [DEBUG] WebPage - setupFrame ""
2017-07-06T12:55:57 [DEBUG] FileSystem - _open: ":/modules/fs.js" QMap(("mode", QVariant(QString, "r")))
2017-07-06T12:55:57 [DEBUG] FileSystem - _open: ":/modules/system.js" QMap(("mode", QVariant(QString, "r")))
2017-07-06T12:55:57 [DEBUG] FileSystem - _open: ":/modules/webpage.js" QMap(("mode", QVariant(QString, "r")))
2017-07-06T12:55:57 [DEBUG] Phantom - injectJs: "/usr/local/google/home/rsimha/.rvm/gems/ruby-2.4.0/gems/poltergeist-1.15.0/lib/capybara/poltergeist/client/compiled/web_page.js"
2017-07-06T12:55:57 [DEBUG] Phantom - injectJs: "/usr/local/google/home/rsimha/.rvm/gems/ruby-2.4.0/gems/poltergeist-1.15.0/lib/capybara/poltergeist/client/compiled/node.js"
2017-07-06T12:55:57 [DEBUG] Phantom - injectJs: "/usr/local/google/home/rsimha/.rvm/gems/ruby-2.4.0/gems/poltergeist-1.15.0/lib/capybara/poltergeist/client/compiled/connection.js"
2017-07-06T12:55:57 [DEBUG] Phantom - injectJs: "/usr/local/google/home/rsimha/.rvm/gems/ruby-2.4.0/gems/poltergeist-1.15.0/lib/capybara/poltergeist/client/compiled/cmd.js"
2017-07-06T12:55:57 [DEBUG] Phantom - injectJs: "/usr/local/google/home/rsimha/.rvm/gems/ruby-2.4.0/gems/poltergeist-1.15.0/lib/capybara/poltergeist/client/compiled/browser.js"
2017-07-06T12:55:57 [DEBUG] WebpageCallbacks - getJsConfirmCallback
2017-07-06T12:55:57 [DEBUG] WebpageCallbacks - getJsConfirmCallback
2017-07-06T12:55:57 [DEBUG] WebPage - updateLoadingProgress: 10
2017-07-06T12:55:57 [DEBUG] WebPage - updateLoadingProgress: 13
2017-07-06T12:55:57 [DEBUG] WebPage - updateLoadingProgress: 16
2017-07-06T12:55:57 [DEBUG] WebPage - updateLoadingProgress: 18
GET /examples/visual-tests/amp-by-example/amp-by-example.html 200 15.152 ms - 116573
2017-07-06T12:55:57 [DEBUG] WebPage - updateLoadingProgress: 21
2017-07-06T12:55:57 [DEBUG] WebPage - updateLoadingProgress: 24
2017-07-06T12:55:57 [DEBUG] WebPage - updateLoadingProgress: 26
2017-07-06T12:55:57 [DEBUG] WebPage - updateLoadingProgress: 28
2017-07-06T12:55:57 [DEBUG] WebPage - updateLoadingProgress: 31
2017-07-06T12:55:57 [DEBUG] WebPage - updateLoadingProgress: 33
2017-07-06T12:55:57 [DEBUG] WebPage - updateLoadingProgress: 35
2017-07-06T12:55:57 [DEBUG] WebPage - updateLoadingProgress: 37
GET /dist/v0/amp-selector-0.1.max.js 200 11.119 ms - 189182
2017-07-06T12:55:57 [DEBUG] WebPage - updateLoadingProgress: 39
2017-07-06T12:55:57 [DEBUG] WebPage - setupFrame ""
GET /dist/v0/amp-sidebar-0.1.max.js 200 11.668 ms - 208400
2017-07-06T12:55:57 [DEBUG] WebPage - evaluateJavaScript "(function() { return (function () {\n      return typeof __poltergeist;\n    })(); })()"
2017-07-06T12:55:57 [DEBUG] WebPage - evaluateJavaScript result QVariant(QString, "undefined")
GET /dist/v0/amp-install-serviceworker-0.1.max.js 200 10.630 ms - 206187
GET /dist/v0/amp-analytics-0.1.max.js 200 10.566 ms - 496751
2017-07-06T12:55:57 [DEBUG] WebPage - updateLoadingProgress: 41
2017-07-06T12:55:57 [DEBUG] WebPage - updateLoadingProgress: 43
2017-07-06T12:55:57 [DEBUG] WebPage - updateLoadingProgress: 45
GET /dist/v0/amp-accordion-0.1.max.js 200 3.664 ms - 106968
GET /dist/amp.js 200 8.914 ms - 1185023
2017-07-06T12:55:57 [DEBUG] WebPage - updateLoadingProgress: 47
2017-07-06T12:55:57 [DEBUG] WebPage - updateLoadingProgress: 49
Powered by AMP ⚡ HTML – Version 1498511723360 http://localhost:8000/examples/visual-tests/amp-by-example/amp-by-example.html
GET /examples/visual-tests/amp-by-example/amp_by_example_logo.svg 200 0.994 ms - 1876
2017-07-06T12:55:57 [DEBUG] WebPage - updateLoadingProgress: 50
2017-07-06T12:55:57 [DEBUG] WebPage - updateLoadingProgress: 52
2017-07-06T12:55:57 [DEBUG] WebPage - updateLoadingProgress: 58
2017-07-06T12:55:57 [DEBUG] WebPage - updateLoadingProgress: 61
2017-07-06T12:55:57 [DEBUG] WebPage - updateLoadingProgress: 65
2017-07-06T12:55:57 [DEBUG] WebPage - updateLoadingProgress: 67
2017-07-06T12:55:57 [DEBUG] WebPage - updateLoadingProgress: 72
2017-07-06T12:55:57 [DEBUG] WebPage - updateLoadingProgress: 76
2017-07-06T12:55:57 [DEBUG] WebPage - updateLoadingProgress: 81
2017-07-06T12:55:57 [DEBUG] WebPage - updateLoadingProgress: 84
2017-07-06T12:55:57 [DEBUG] WebPage - updateLoadingProgress: 100
2017-07-06T12:55:57 [DEBUG] WebPage - setupFrame ""
Visited examples/visual-tests/amp-by-example/amp-by-example.html...
Snapshotting...
2017-07-06T12:56:02 [DEBUG] WebPage - evaluateJavaScript "(function() { return (function () {\n      return typeof __poltergeist;\n    })(); })()"
2017-07-06T12:56:02 [DEBUG] WebPage - evaluateJavaScript result QVariant(QString, "object")
2017-07-06T12:56:02 [DEBUG] WebPage - evaluateJavaScript "(function() { return (function() { var page_id = arguments[0]; var args = []; for(var i=1; i < arguments.length; i++){ if ((typeof(arguments[i]) == 'object') && (typeof(arguments[i]['ELEMENT']) == 'object')){ args.push(window.__poltergeist.get(arguments[i]['ELEMENT']['id']).element); } else { args.push(arguments[i]) } } var _result = (function (name, args) {\n      return __poltergeist.externalCall(name, args);\n    }).apply(this, args); return window.__poltergeist.wrapResults(_result, page_id); })(1,\"find\",[\"css\",\".i-amphtml-loader-dot\"]); })()"
2017-07-06T12:56:02 [DEBUG] WebPage - evaluateJavaScript result QVariant(QVariantMap, QMap(("value", QVariant(QVariantList, ()))))
[percy][DEBUG] Using filesystem_loader to discover assets.
[percy][DEBUG] Snapshot started (name: "Amp By Example")
2017-07-06T12:56:02 [DEBUG] WebPage - evaluateJavaScript "(function() { return (function () {\n      return typeof __poltergeist;\n    })(); })()"
2017-07-06T12:56:02 [DEBUG] WebPage - evaluateJavaScript result QVariant(QString, "object")
2017-07-06T12:56:02 [DEBUG] WebPage - evaluateJavaScript "(function() { return (function() { var page_id = arguments[0]; var args = []; for(var i=1; i < arguments.length; i++){ if ((typeof(arguments[i]) == 'object') && (typeof(arguments[i]['ELEMENT']) == 'object')){ args.push(window.__poltergeist.get(arguments[i]['ELEMENT']['id']).element); } else { args.push(arguments[i]) } } var _result = (function (name, args) {\n      return __poltergeist.externalCall(name, args);\n    }).apply(this, args); return window.__poltergeist.wrapResults(_result, page_id); })(1,\"currentUrl\",[]); })()"
2017-07-06T12:56:02 [DEBUG] WebPage - evaluateJavaScript result QVariant(QVariantMap, QMap(("value", QVariant(QString, "http://localhost:8000/examples/visual-tests/amp-by-example/amp-by-example.html"))))
[percy][DEBUG] Snapshot resource: /examples/visual-tests/amp-by-example/amp-by-example.html
[percy][DEBUG] All snapshot resources loaded (0.054529548s)
[percy][DEBUG] All snapshot resources uploaded (0.156652138s)
2017-07-06T12:56:03 [DEBUG] WebPage - evaluateJavaScript "(function() { return (function () {\n      return typeof __poltergeist;\n    })(); })()"
2017-07-06T12:56:03 [DEBUG] WebPage - evaluateJavaScript result QVariant(QString, "object")
2017-07-06T12:56:03 [DEBUG] WebPage - evaluateJavaScript "(function() { return (function() { var page_id = arguments[0]; var args = []; for(var i=1; i < arguments.length; i++){ if ((typeof(arguments[i]) == 'object') && (typeof(arguments[i]['ELEMENT']) == 'object')){ args.push(window.__poltergeist.get(arguments[i]['ELEMENT']['id']).element); } else { args.push(arguments[i]) } } var _result = (function (name, args) {\n      return __poltergeist.externalCall(name, args);\n    }).apply(this, args); return window.__poltergeist.wrapResults(_result, page_id); })(1,\"currentUrl\",[]); })()"
2017-07-06T12:56:03 [DEBUG] WebPage - evaluateJavaScript result QVariant(QVariantMap, QMap(("value", QVariant(QString, "http://localhost:8000/examples/visual-tests/amp-by-example/amp-by-example.html"))))
2017-07-06T12:56:03 [DEBUG] WebPage - updateLoadingProgress: 10
GET /examples/visual-tests/article.amp/article.amp.html 200 3.043 ms - 14144
2017-07-06T12:56:03 [DEBUG] WebPage - updateLoadingProgress: 33
2017-07-06T12:56:03 [DEBUG] WebPage - setupFrame ""
2017-07-06T12:56:03 [DEBUG] WebPage - evaluateJavaScript "(function() { return (function () {\n      return typeof __poltergeist;\n    })(); })()"
2017-07-06T12:56:03 [DEBUG] WebPage - evaluateJavaScript result QVariant(QString, "undefined")
2017-07-06T12:56:03 [DEBUG] WebPage - updateLoadingProgress: 35
2017-07-06T12:56:03 [DEBUG] WebPage - updateLoadingProgress: 37
2017-07-06T12:56:03 [DEBUG] WebPage - updateLoadingProgress: 39
2017-07-06T12:56:03 [DEBUG] WebPage - updateLoadingProgress: 41
2017-07-06T12:56:03 [DEBUG] WebPage - updateLoadingProgress: 43
2017-07-06T12:56:03 [DEBUG] WebPage - updateLoadingProgress: 45
GET /dist/v0/amp-lightbox-0.1.max.js 200 6.500 ms - 248652
GET /dist/v0/amp-sidebar-0.1.max.js 200 6.480 ms - 208400
GET /dist/v0/amp-app-banner-0.1.max.js 200 6.672 ms - 212754
GET /dist/amp.js 200 1.205 ms - 1185023
GET /dist/v0/amp-ad-0.1.max.js 200 5.767 ms - 676432
GET /dist/v0/amp-analytics-0.1.max.js 200 6.677 ms - 496751
2017-07-06T12:56:03 [DEBUG] WebPage - updateLoadingProgress: 47
2017-07-06T12:56:03 [DEBUG] WebPage - updateLoadingProgress: 49
2017-07-06T12:56:03 [DEBUG] WebPage - updateLoadingProgress: 100
Powered by AMP ⚡ HTML – Version 1498511723360 http://localhost:8000/examples/visual-tests/article.amp/article.amp.html
2017-07-06T12:56:03 [DEBUG] WebPage - setupFrame ""
GET /examples/visual-tests/article.amp/img/hero@1x.jpg 200 0.823 ms - 79160
2017-07-06T12:56:03 [DEBUG] WebPage - setupFrame ""
GET /examples/visual-tests/article.amp/img/sea@1x.jpg 200 2.199 ms - 63914
GET /examples/visual-tests/article.amp/img/sample.jpg 200 2.399 ms - 28505
Visited examples/visual-tests/article.amp/article.amp.html...
2017-07-06T12:56:03 [DEBUG] WebPage - setupFrame ""
2017-07-06T12:56:03 [DEBUG] Network - Resource request error: QNetworkReply::NetworkError(ContentNotFoundError) ( "Error downloading https://pagead2.googlesyndication.com/amp_preconnect_polyfill_404_or_other_error_expected._Do_not_worry_about_it?1499360040000 - server replied: Not Found" ) URL: "https://pagead2.googlesyndication.com/amp_preconnect_polyfill_404_or_other_error_expected._Do_not_worry_about_it?1499360040000"
2017-07-06T12:56:03 [DEBUG] Network - Resource request error: QNetworkReply::NetworkError(HostNotFoundError) ( "Host ads.localhost not found" ) URL: "http://ads.localhost:8000/amp_preconnect_polyfill_404_or_other_error_expected._Do_not_worry_about_it?1499360040000"
2017-07-06T12:56:03 [DEBUG] Network - Resource request error: QNetworkReply::NetworkError(ContentNotFoundError) ( "Error downloading https://googleads.g.doubleclick.net/amp_preconnect_polyfill_404_or_other_error_expected._Do_not_worry_about_it?1499360040000 - server replied: Not Found" ) URL: "https://googleads.g.doubleclick.net/amp_preconnect_polyfill_404_or_other_error_expected._Do_not_worry_about_it?1499360040000"
2017-07-06T12:56:03 [DEBUG] Network - Resource request error: QNetworkReply::NetworkError(ContentOperationNotPermittedError) ( "Error downloading https://c.amazon-adsystem.com/amp_preconnect_polyfill_404_or_other_error_expected._Do_not_worry_about_it?1499360040000 - server replied: Forbidden" ) URL: "https://c.amazon-adsystem.com/amp_preconnect_polyfill_404_or_other_error_expected._Do_not_worry_about_it?1499360040000"
2017-07-06T12:56:04 [DEBUG] WebPage - setupFrame "{\"host\":\"ads.localhost\",\"type\":\"a9\",\"count\":1,\"attributes\":{\"aax_size\":\"300x250\",\"aax_pubname\":\"abc123\",\"aax_src\":\"302\",\"ampSlotIndex\":\"0\",\"width\":300,\"height\":250,\"_context\":{\"ampcontextVersion\":\"1498511723360\",\"ampcontextFilepath\":\"https://cdn.ampproject.org/1498511723360/ampcontext-v0.js\",\"sourceUrl\":\"http://localhost:8000/examples/visual-tests/article.amp/article.amp.html\",\"referrer\":\"\",\"canonicalUrl\":\"https://medium.com/p/cb7f223fad86\",\"pageViewId\":\"2748\",\"location\":{\"href\":\"http://localhost:8000/examples/visual-tests/article.amp/article.amp.html\"},\"startTime\":1499360164220,\"tagName\":\"AMP-AD\",\"mode\":{\"localDev\":true,\"development\":false,\"minified\":false,\"lite\":false,\"test\":false,\"version\":\"1498511723360\",\"rtvVersion\":\"1498511723360\"},\"canary\":false,\"hidden\":false,\"initialLayoutRect\":{\"left\":15,\"top\":1262,\"width\":300,\"height\":250},\"initialIntersection\":{\"time\":1209.6243969999998,\"rootBounds\":{\"left\":0,\"top\":0,\"width\":1024,\"height\":768,\"bottom\":768,\"right\":1024,\"x\":0,\"y\":0},\"boundingClientRect\":{\"left\":15,\"top\":1262,\"width\":300,\"height\":250,\"bottom\":1512,\"right\":315,\"x\":15,\"y\":1262},\"intersectionRect\":{\"left\":0,\"top\":0,\"width\":0,\"height\":0,\"bottom\":0,\"right\":0,\"x\":0,\"y\":0},\"intersectionRatio\":0},\"domFingerprint\":\"1674970725\",\"experimentToggles\":{\"a4aProfilingRate\":true},\"sentinel\":\"0-1787760248827559524\",\"clientId\":null,\"container\":null},\"type\":\"a9\"}}"
2017-07-06T12:56:04 [DEBUG] WebPage - updateLoadingProgress: 10
2017-07-06T12:56:04 [DEBUG] Network - Resource request error: QNetworkReply::NetworkError(HostNotFoundError) ( "Host ads.localhost not found" ) URL: "http://ads.localhost:8000/dist.3p/current/frame.max.html"
2017-07-06T12:56:04 [DEBUG] WebPage - updateLoadingProgress: 100
Snapshotting...
2017-07-06T12:56:08 [DEBUG] WebPage - evaluateJavaScript "(function() { return (function () {\n      return typeof __poltergeist;\n    })(); })()"
2017-07-06T12:56:08 [DEBUG] WebPage - evaluateJavaScript result QVariant(QString, "object")
2017-07-06T12:56:08 [DEBUG] WebPage - evaluateJavaScript "(function() { return (function() { var page_id = arguments[0]; var args = []; for(var i=1; i < arguments.length; i++){ if ((typeof(arguments[i]) == 'object') && (typeof(arguments[i]['ELEMENT']) == 'object')){ args.push(window.__poltergeist.get(arguments[i]['ELEMENT']['id']).element); } else { args.push(arguments[i]) } } var _result = (function (name, args) {\n      return __poltergeist.externalCall(name, args);\n    }).apply(this, args); return window.__poltergeist.wrapResults(_result, page_id); })(2,\"find\",[\"css\",\".i-amphtml-loader-dot\"]); })()"
2017-07-06T12:56:08 [DEBUG] WebPage - evaluateJavaScript result QVariant(QVariantMap, QMap(("value", QVariant(QVariantList, ()))))
[percy][DEBUG] Using filesystem_loader to discover assets.
[percy][DEBUG] Snapshot started (name: "AMP Article")
2017-07-06T12:56:08 [DEBUG] WebPage - evaluateJavaScript "(function() { return (function () {\n      return typeof __poltergeist;\n    })(); })()"
2017-07-06T12:56:08 [DEBUG] WebPage - evaluateJavaScript result QVariant(QString, "object")
2017-07-06T12:56:08 [DEBUG] WebPage - evaluateJavaScript "(function() { return (function() { var page_id = arguments[0]; var args = []; for(var i=1; i < arguments.length; i++){ if ((typeof(arguments[i]) == 'object') && (typeof(arguments[i]['ELEMENT']) == 'object')){ args.push(window.__poltergeist.get(arguments[i]['ELEMENT']['id']).element); } else { args.push(arguments[i]) } } var _result = (function (name, args) {\n      return __poltergeist.externalCall(name, args);\n    }).apply(this, args); return window.__poltergeist.wrapResults(_result, page_id); })(2,\"currentUrl\",[]); })()"
2017-07-06T12:56:08 [DEBUG] WebPage - evaluateJavaScript result QVariant(QVariantMap, QMap(("value", QVariant(QString, "http://localhost:8000/examples/visual-tests/article.amp/article.amp.html"))))
[percy][DEBUG] Snapshot resource: /examples/visual-tests/article.amp/article.amp.html
[percy][DEBUG] All snapshot resources loaded (0.005843576s)
[percy][DEBUG] All snapshot resources uploaded (1.116251422s)

[percy] Visual diffs are now processing: https://percy.io/ampproject/amphtml/builds/278235

Done! Percy snapshots are now processing...
--> https://percy.io/ampproject/amphtml/builds/278235
[12:56:09] Shutting down server
rsimha commented 7 years ago

/to @timhaines @fotinakis

rsimha commented 7 years ago

Our project is open source, and can be cloned at https://github.com/ampproject/amphtml.

Once you have the code, you can reproduce the issue from your local repository by running the following from the amphtml directory:

  1. gulp build
  2. ruby build-system/tasks/visual-diff.rb

If you'd like detailed logs from percy / phantomjs, you can run ruby build-system/tasks/visual-diff.rb --percy_debug --phantomjs_debug during step 2.

rsimha commented 7 years ago

In order to see what these pages are supposed to look like with their images, you can run the following from the amphtml directory:

  1. gulp build
  2. gulp serve
  3. Open a browser and navigate to http://localhost:8000/examples/visual-tests/amp-by-example/amp-by-example.html or http://localhost:8000/examples/visual-tests/article.amp/article.amp.html
timhaines commented 7 years ago

@rsimha-amp thank you for the great instructions here! I'm going to dig into this.

timhaines commented 7 years ago

@rsimha-amp I sent you PR #10293 on the AMPhtml project that fixes your assets_base_url. I've tested it and the images load in Percy now.

I'm going to close this issue now, but please feel welcome to follow up with me if I can help further.

rsimha commented 7 years ago

@timhaines, thanks for the quick work. I tried your change locally, and I'm not sure it works. I'm seeing several 404 errors and in addition to the images not loading correctly, the page doesn't fully load either.

I'll follow up in the PR comments.

timhaines commented 7 years ago

@rsimha-amp Oh, that sounds troublesome. How do I reproduce the 404s?

rsimha commented 7 years ago

@timhaines, I'm very sorry, I jumped the gun in my previous comment. Your PR did work on my local machine after I did a clean build, so I'm going to approve your PR. Thank you very much for the help!

timhaines commented 7 years ago

@rsimha-amp 👍 Excellent to hear. Thanks Raghu.