Updated handling of git checkouts to ensure the #ref matches the
origin ref; makes sure cloneOrFetch behavior actually updates the
repo HEAD locally if origin changes.
Better test reporting, so you can see what ran and what failed etc.
This was very obscure before.
We don't run tests for a repo if the requested branch doesn't exist.
In fact we totally remove the repo.
Bower resolves version numbers based on the cloned repo's bower.json
version values.
Repo references in the bower.json for tattoo_workspace include SHA1
for the cloned repos so we have better accounting for what was run.
Error messaging for bad branch references (in verbose mode) now shows
after all cloning is done so the progress bars can complete.
Total number of repos being tested is reported in the progress bar
line, so no surprises.
Sort the output of test results by repo name. Nice if you're running
75 tests, as-in tattoo PolymerElements/*#preview-2.0
You can provide wct-flags like --wct-flags="--local safari" now, whereas before --local was being parsed as a tattoo arg and blew things up.
Check out this nice run of all the iron elements with 2.0-preview branches:
$ tattoo "PolymerElements/iron-*#2.0-preview" --wct-flags="--local chrome --color"
Fetching repo names for wildcard search [=============================================] 100%
Getting Repo details from GitHub... [=============================================] 100%
Cloning/Updating repos... [=============================================] 100%
Installing dependencies with bower... [=============================================] 100%
Testing 37 repo(s)... [=============================================] 100%
---
PolymerElements/iron-behaviors#2.0-preview output:
Installing and starting Selenium server for local browsers
Selenium server running on port 50473
Web server running on port 2000 and serving from /Users/brendanb/src/github.com/polymer/tattoo/tattoo_workspace/iron-behaviors
chrome 54 Beginning tests via http://localhost:2000/components/iron-behaviors/generated-index.html?cli_browser_id=0
chrome failed to maximize
chrome 54 ✖ test/active-state.html » active-state » nested paper-input inside button » space in light child input does not trigger a button click event
Cannot read property 'focus' of undefined
<unknown> at HTMLElement._focusBlurHandler at /components/paper-input/paper-input-behavior.html:433:0
<unknown> at HTMLElement.wrapperFn at /components/shadydom/shadydom.min.js:2780:0
<unknown> at fireHandlers at /components/shadydom/shadydom.min.js:2674:0
<unknown> at retargetNonBubblingEvent at /components/shadydom/shadydom.min.js:2695:0
<unknown> at <unknown> at /components/shadydom/shadydom.min.js:2857:0
<unknown> at Context.<anonymous> at active-state.html:250:0
chrome 54 Tests failed: 1 failed tests
Test run ended in failure: 1 failed tests
chrome 54 (55/0/1)
Error: 1 failed tests
---
PolymerElements/iron-form#2.0-preview output:
Installing and starting Selenium server for local browsers
Selenium server running on port 51154
Web server running on port 2000 and serving from /Users/brendanb/src/github.com/polymer/tattoo/tattoo_workspace/iron-form
chrome 54 Beginning tests via http://localhost:2000/components/iron-form/generated-index.html?cli_browser_id=0
chrome failed to maximize
chrome 54 ✖ test/basic.html » resetting » can reset a form
expected '{"input1":"input1","input2":"","check1":"on","radio1":"on","papercheck1":"on","paper1":"paper1","paper2":""}' to equal '{"input1":"input1","input2":"input2++","check2":"on","radio2":"on","papercheck2":"on","paper1":"paper1++","paper2":"paper2++"}'
<unknown> at HTMLElement.<anonymous> at basic.html:541:0
<unknown> at <unknown> at /components/polymer/src/elements/legacy-element.html:537:0
chrome 54 ✖ test/basic.html » resetting » can reset a form
expected '{"input1":"input1","input2":"","check1":"on","radio1":"on","papercheck1":"on","paper1":"paper1","paper2":""}' to equal '{"input1":"input1","input2":"input2++","check2":"on","radio2":"on","papercheck2":"on","paper1":"paper1++","paper2":"paper2++"}'
<unknown> at HTMLElement.<anonymous> at basic.html:541:0
<unknown> at <unknown> at /components/polymer/src/elements/legacy-element.html:537:0
chrome 54 Tests failed: 2 failed tests
Test run ended in failure: 2 failed tests
chrome 54 (34/0/2)
Error: 2 failed tests
---
PolymerElements/iron-list#2.0-preview output:
Installing and starting Selenium server for local browsers
Selenium server running on port 50088
Web server running on port 2000 and serving from /Users/brendanb/src/github.com/polymer/tattoo/tattoo_workspace/iron-list
chrome 54 Beginning tests via http://localhost:2000/components/iron-list/generated-index.html?cli_browser_id=0
chrome failed to maximize
chrome 54 Tests failed: Error thrown outside of test function: the string "console.error: unimplemented" was thrown, throw an Error :)
HTMLElement.createFrom at /components/test-fixture/test-fixture.html:207
HTMLElement.<anonymous> at /components/test-fixture/test-fixture.html:175
HTMLElement.forElements at /components/test-fixture/test-fixture.html:269
HTMLElement.create at /components/test-fixture/test-fixture.html:173
Context.<anonymous> at basic.html:37
Test run ended in failure: Error thrown outside of test function: the string "console.error: unimplemented" was thrown, throw an Error :)
HTMLElement.createFrom at /components/test-fixture/test-fixture.html:207
HTMLElement.<anonymous> at /components/test-fixture/test-fixture.html:175
HTMLElement.forElements at /components/test-fixture/test-fixture.html:269
HTMLElement.create at /components/test-fixture/test-fixture.html:173
Context.<anonymous> at basic.html:37
chrome 54 (1/0/0 error)
Error: Error thrown outside of test function: the string "console.error: unimplemented" was thrown, throw an Error :)
HTMLElement.createFrom at /components/test-fixture/test-fixture.html:207
HTMLElement.<anonymous> at /components/test-fixture/test-fixture.html:175
HTMLElement.forElements at /components/test-fixture/test-fixture.html:269
HTMLElement.create at /components/test-fixture/test-fixture.html:173
Context.<anonymous> at basic.html:37
---
PASSED: PolymerElements/iron-a11y-announcer#2.0-preview
PASSED: PolymerElements/iron-a11y-keys-behavior#2.0-preview
PASSED: PolymerElements/iron-ajax#2.0-preview
PASSED: PolymerElements/iron-autogrow-textarea#2.0-preview
PASSED: PolymerElements/iron-checked-element-behavior#2.0-preview
PASSED: PolymerElements/iron-collapse#2.0-preview
PASSED: PolymerElements/iron-component-page#2.0-preview
PASSED: PolymerElements/iron-demo-helpers#2.0-preview PASSED: PolymerElements/iron-doc-viewer#2.0-preview
PASSED: PolymerElements/iron-dropdown#2.0-preview
PASSED: PolymerElements/iron-fit-behavior#2.0-preview
PASSED: PolymerElements/iron-flex-layout#2.0-preview
PASSED: PolymerElements/iron-form-element-behavior#2.0-preview
PASSED: PolymerElements/iron-icon#2.0-preview
PASSED: PolymerElements/iron-iconset#2.0-preview
PASSED: PolymerElements/iron-iconset-svg#2.0-preview
PASSED: PolymerElements/iron-image#2.0-preview
PASSED: PolymerElements/iron-input#2.0-preview
PASSED: PolymerElements/iron-label#2.0-preview
PASSED: PolymerElements/iron-location#2.0-preview
PASSED: PolymerElements/iron-media-query#2.0-preview
PASSED: PolymerElements/iron-menu-behavior#2.0-preview
PASSED: PolymerElements/iron-meta#2.0-preview
PASSED: PolymerElements/iron-overlay-behavior#2.0-preview
PASSED: PolymerElements/iron-pages#2.0-preview
PASSED: PolymerElements/iron-range-behavior#2.0-preview
PASSED: PolymerElements/iron-resizable-behavior#2.0-preview
PASSED: PolymerElements/iron-scroll-target-behavior#2.0-preview
PASSED: PolymerElements/iron-scroll-threshold#2.0-preview
PASSED: PolymerElements/iron-selector#2.0-preview
PASSED: PolymerElements/iron-test-helpers#2.0-preview
PASSED: PolymerElements/iron-validatable-behavior#2.0-preview
PASSED: PolymerElements/iron-validator-behavior#2.0-preview
SKIPPED: PolymerElements/iron-icons#2.0-preview
FAILED: PolymerElements/iron-behaviors#2.0-preview
FAILED: PolymerElements/iron-form#2.0-preview
FAILED: PolymerElements/iron-list#2.0-preview
33 / 36 tests passed. 1 skipped.
tattoo: 225579.742ms
usergenic
commented
7 years ago
Updated handling of git checkouts to ensure the #ref matches the origin ref; makes sure cloneOrFetch behavior actually updates the repo HEAD locally if origin changes.
Better test reporting, so you can see what ran and what failed etc. This was very obscure before.
We don't run tests for a repo if the requested branch doesn't exist. In fact we totally remove the repo.
Bower resolves version numbers based on the cloned repo's bower.json version values.
Repo references in the bower.json for tattoo_workspace include SHA1 for the cloned repos so we have better accounting for what was run.
Error messaging for bad branch references (in verbose mode) now shows after all cloning is done so the progress bars can complete.
Total number of repos being tested is reported in the progress bar line, so no surprises.
Sort the output of test results by repo name. Nice if you're running 75 tests, as-in
tattoo PolymerElements/*#preview-2.0You can provide wct-flags like
--wct-flags="--local safari"now, whereas before--localwas being parsed as a tattoo arg and blew things up.Check out this nice run of all the iron elements with 2.0-preview branches: