Introduce acceptance tests

dideler / toggle-youtube-comments

:cinema: :zipper_mouth_face: Chrome extension to hide or show YouTube comments

https://chrome.google.com/webstore/detail/toggle-youtube-comments/ampjnmipdgicjjidohpkidhghakoidfm

MIT License

14 stars 2 forks source link

Introduce acceptance tests #9

Open dideler opened 6 years ago

dideler commented 6 years ago

As a maintainer I want to run UI tests before publishing the extension In order to quickly detect breaking changes and avoid regressions

Scenarios:

Toggle button appears when hard navigating to a video
Toggle button appears when navigating from home page to a video
Toggle button appears when navigating from a video to a video
Toggle button appears when logged in and hard navigating to a video with comments favourited by the channel owner (see #33)
Toggle button does not appear when viewing a live video with chat

Example of how some of the feature tests could look:

Given the preferred site is set to
| YouTube UI |
| Old        |
| New        |
When I navigate to
| Page Navigation |
| Home -> Video   |
| Video           |
| Live Video      |
Then I do not see comments
When I click button to "show comments"
And I scroll down the page
Then I see comments
When I click button to "hide comments"
Then I do not see comments

Are we missing scenarios for edge cases? E.g. videos with comments disabled

dideler commented 6 years ago

Let's give headless Chrome a try. Here's a good tutorial: https://developers.google.com/web/updates/2017/06/headless-karma-mocha-chai

dideler commented 6 years ago

Loading extensions in Chrome Headless is not yet supported. Easiest way to test the extension might be to use Puppeteer, although we won't be able to run Chrome in headless mode.

Though I usually prefer CircleCI when using a hosted CI service, TravisCI looks more straightforward to setup with Puppeteer.

1natsu172 commented 6 years ago

I think so, it's time to introduce automatic test (E2E test). I actively want to support it, but unfortunately I don't have much knowledge about the E2E test. I tried it on another project but I had no knowledge so I put it on hold… 😔

Does @dideler have knowledge of test using Headless browser? If so, I want you to take the lead and introduce it! Of course I also think want to support you and will re-learn on this occasion.

So it seems that somehow hacks are necessary for both CircleCI and TravisCI. 🤔

dideler commented 6 years ago

Happy to take the lead on this! I've done automated UI testing with webdrivers such as Watir, Selenium, and Capybara Webkit, but I haven't used the official Chrome Headless yet. Though as mentioned in https://github.com/dideler/toggle-youtube-comments/issues/9#issuecomment-363932488, it doesn't look like we'll be able to run headless and load the extension :confused: .

I'll start putting something together this weekend.

1natsu172 commented 6 years ago

Thanks for taking the lead. And you have a very much experience in the Headless browser test! 👏
Good luck starting from the weekend.

So, about Puppeter can't run the Chrome extension, It seems to be avoiding it using xvfb. 🤔