search

web-push-libs / web-push-php

Web Push library for PHP
MIT License
1.69k stars 294 forks source link

Travis tests are failing #274

Closed reedy closed 2 years ago

reedy commented 4 years ago

It seems all recent commits and PR are resulting in test failures from travis. Someone should fix them

There were 14 errors:
1) PushServiceTest::testBrowsers with data set #0 ('firefox', 'stable', array())
Exception: web-push-testing-service error: {"error":{"id":"unable_to_get_subscription","message":"An issue occured while attempting to get the subscription: Error retrieving push subscription."}}
/home/travis/build/web-push-libs/web-push-php/tests/PushServiceTest.php:265
/home/travis/build/web-push-libs/web-push-php/tests/PushServiceTest.php:156
/home/travis/build/web-push-libs/web-push-php/tests/PushServiceTest.php:101
/home/travis/build/web-push-libs/web-push-php/tests/PushServiceTest.php:119
2) PushServiceTest::testBrowsers with data set #1 ('firefox', 'beta', array())
Exception: web-push-testing-service error: {"error":{"id":"unable_to_get_subscription","message":"An issue occured while attempting to get the subscription: Error retrieving push subscription."}}
/home/travis/build/web-push-libs/web-push-php/tests/PushServiceTest.php:265
/home/travis/build/web-push-libs/web-push-php/tests/PushServiceTest.php:156
/home/travis/build/web-push-libs/web-push-php/tests/PushServiceTest.php:101
/home/travis/build/web-push-libs/web-push-php/tests/PushServiceTest.php:119
3) PushServiceTest::testBrowsers with data set #2 ('chrome', 'stable', array('AIzaSyBAU0VfXoskxUSg81K5VgLgw...ZWe6tA'))
Exception: web-push-testing-service error: {"error":{"id":"webdriver_issue","message":"An issue occured while attempting to get the subscription: session not created: Chrome version must be between 71 and 75\n  (Driver info: chromedriver=2.46.628388 (4a34a70827ac54148e092aafb70504c4ea7ae926),platform=Linux 4.4.0-101-generic x86_64)"}}
/home/travis/build/web-push-libs/web-push-php/tests/PushServiceTest.php:265
/home/travis/build/web-push-libs/web-push-php/tests/PushServiceTest.php:156
/home/travis/build/web-push-libs/web-push-php/tests/PushServiceTest.php:101
/home/travis/build/web-push-libs/web-push-php/tests/PushServiceTest.php:119
4) PushServiceTest::testBrowsers with data set #3 ('chrome', 'beta', array('AIzaSyBAU0VfXoskxUSg81K5VgLgw...ZWe6tA'))
Exception: web-push-testing-service error: {"error":{"id":"webdriver_issue","message":"An issue occured while attempting to get the subscription: session not created: Chrome version must be between 71 and 75\n  (Driver info: chromedriver=2.46.628388 (4a34a70827ac54148e092aafb70504c4ea7ae926),platform=Linux 4.4.0-101-generic x86_64)"}}
/home/travis/build/web-push-libs/web-push-php/tests/PushServiceTest.php:265
/home/travis/build/web-push-libs/web-push-php/tests/PushServiceTest.php:156
/home/travis/build/web-push-libs/web-push-php/tests/PushServiceTest.php:101
/home/travis/build/web-push-libs/web-push-php/tests/PushServiceTest.php:119
5) PushServiceTest::testBrowsers with data set #4 ('firefox', 'stable', array('AIzaSyBAU0VfXoskxUSg81K5VgLgw...ZWe6tA'))
Exception: web-push-testing-service error: {"error":{"id":"unable_to_get_subscription","message":"An issue occured while attempting to get the subscription: Error retrieving push subscription."}}
/home/travis/build/web-push-libs/web-push-php/tests/PushServiceTest.php:265
/home/travis/build/web-push-libs/web-push-php/tests/PushServiceTest.php:156
/home/travis/build/web-push-libs/web-push-php/tests/PushServiceTest.php:101
/home/travis/build/web-push-libs/web-push-php/tests/PushServiceTest.php:119
6) PushServiceTest::testBrowsers with data set #5 ('firefox', 'beta', array('AIzaSyBAU0VfXoskxUSg81K5VgLgw...ZWe6tA'))
Exception: web-push-testing-service error: {"error":{"id":"unable_to_get_subscription","message":"An issue occured while attempting to get the subscription: Error retrieving push subscription."}}
/home/travis/build/web-push-libs/web-push-php/tests/PushServiceTest.php:265
/home/travis/build/web-push-libs/web-push-php/tests/PushServiceTest.php:156
/home/travis/build/web-push-libs/web-push-php/tests/PushServiceTest.php:101
/home/travis/build/web-push-libs/web-push-php/tests/PushServiceTest.php:119
7) PushServiceTest::testBrowsers with data set #6 ('chrome', 'stable', array(array('http://test.com', 'BA6jvk34k6YjElHQ6S0oZwmrsqHdC...lLQHDk', '-3CdhFOqjzixgAbUSa0Zv9zi-dwDV...BxSFPQ')))
Exception: web-push-testing-service error: {"error":{"id":"webdriver_issue","message":"An issue occured while attempting to get the subscription: session not created: Chrome version must be between 71 and 75\n  (Driver info: chromedriver=2.46.628388 (4a34a70827ac54148e092aafb70504c4ea7ae926),platform=Linux 4.4.0-101-generic x86_64)"}}
/home/travis/build/web-push-libs/web-push-php/tests/PushServiceTest.php:265
/home/travis/build/web-push-libs/web-push-php/tests/PushServiceTest.php:156
/home/travis/build/web-push-libs/web-push-php/tests/PushServiceTest.php:101
/home/travis/build/web-push-libs/web-push-php/tests/PushServiceTest.php:119
8) PushServiceTest::testBrowsers with data set #7 ('chrome', 'beta', array(array('http://test.com', 'BA6jvk34k6YjElHQ6S0oZwmrsqHdC...lLQHDk', '-3CdhFOqjzixgAbUSa0Zv9zi-dwDV...BxSFPQ')))
Exception: web-push-testing-service error: {"error":{"id":"webdriver_issue","message":"An issue occured while attempting to get the subscription: session not created: Chrome version must be between 71 and 75\n  (Driver info: chromedriver=2.46.628388 (4a34a70827ac54148e092aafb70504c4ea7ae926),platform=Linux 4.4.0-101-generic x86_64)"}}
/home/travis/build/web-push-libs/web-push-php/tests/PushServiceTest.php:265
/home/travis/build/web-push-libs/web-push-php/tests/PushServiceTest.php:156
/home/travis/build/web-push-libs/web-push-php/tests/PushServiceTest.php:101
/home/travis/build/web-push-libs/web-push-php/tests/PushServiceTest.php:119
9) PushServiceTest::testBrowsers with data set #8 ('firefox', 'stable', array(array('http://test.com', 'BA6jvk34k6YjElHQ6S0oZwmrsqHdC...lLQHDk', '-3CdhFOqjzixgAbUSa0Zv9zi-dwDV...BxSFPQ')))
Exception: web-push-testing-service error: {"error":{"id":"unable_to_get_subscription","message":"An issue occured while attempting to get the subscription: Error retrieving push subscription."}}
/home/travis/build/web-push-libs/web-push-php/tests/PushServiceTest.php:265
/home/travis/build/web-push-libs/web-push-php/tests/PushServiceTest.php:156
/home/travis/build/web-push-libs/web-push-php/tests/PushServiceTest.php:101
/home/travis/build/web-push-libs/web-push-php/tests/PushServiceTest.php:119
10) PushServiceTest::testBrowsers with data set #9 ('firefox', 'beta', array(array('http://test.com', 'BA6jvk34k6YjElHQ6S0oZwmrsqHdC...lLQHDk', '-3CdhFOqjzixgAbUSa0Zv9zi-dwDV...BxSFPQ')))
Exception: web-push-testing-service error: {"error":{"id":"unable_to_get_subscription","message":"An issue occured while attempting to get the subscription: Error retrieving push subscription."}}
/home/travis/build/web-push-libs/web-push-php/tests/PushServiceTest.php:265
/home/travis/build/web-push-libs/web-push-php/tests/PushServiceTest.php:156
/home/travis/build/web-push-libs/web-push-php/tests/PushServiceTest.php:101
/home/travis/build/web-push-libs/web-push-php/tests/PushServiceTest.php:119
11) PushServiceTest::testBrowsers with data set #10 ('chrome', 'stable', array('AIzaSyBAU0VfXoskxUSg81K5VgLgw...ZWe6tA', array('http://test.com', 'BA6jvk34k6YjElHQ6S0oZwmrsqHdC...lLQHDk', '-3CdhFOqjzixgAbUSa0Zv9zi-dwDV...BxSFPQ')))
Exception: web-push-testing-service error: {"error":{"id":"webdriver_issue","message":"An issue occured while attempting to get the subscription: session not created: Chrome version must be between 71 and 75\n  (Driver info: chromedriver=2.46.628388 (4a34a70827ac54148e092aafb70504c4ea7ae926),platform=Linux 4.4.0-101-generic x86_64)"}}
/home/travis/build/web-push-libs/web-push-php/tests/PushServiceTest.php:265
/home/travis/build/web-push-libs/web-push-php/tests/PushServiceTest.php:156
/home/travis/build/web-push-libs/web-push-php/tests/PushServiceTest.php:101
/home/travis/build/web-push-libs/web-push-php/tests/PushServiceTest.php:119
12) PushServiceTest::testBrowsers with data set #11 ('chrome', 'beta', array('AIzaSyBAU0VfXoskxUSg81K5VgLgw...ZWe6tA', array('http://test.com', 'BA6jvk34k6YjElHQ6S0oZwmrsqHdC...lLQHDk', '-3CdhFOqjzixgAbUSa0Zv9zi-dwDV...BxSFPQ')))
Exception: web-push-testing-service error: {"error":{"id":"webdriver_issue","message":"An issue occured while attempting to get the subscription: session not created: Chrome version must be between 71 and 75\n  (Driver info: chromedriver=2.46.628388 (4a34a70827ac54148e092aafb70504c4ea7ae926),platform=Linux 4.4.0-101-generic x86_64)"}}
/home/travis/build/web-push-libs/web-push-php/tests/PushServiceTest.php:265
/home/travis/build/web-push-libs/web-push-php/tests/PushServiceTest.php:156
/home/travis/build/web-push-libs/web-push-php/tests/PushServiceTest.php:101
/home/travis/build/web-push-libs/web-push-php/tests/PushServiceTest.php:119
13) PushServiceTest::testBrowsers with data set #12 ('firefox', 'stable', array('AIzaSyBAU0VfXoskxUSg81K5VgLgw...ZWe6tA', array('http://test.com', 'BA6jvk34k6YjElHQ6S0oZwmrsqHdC...lLQHDk', '-3CdhFOqjzixgAbUSa0Zv9zi-dwDV...BxSFPQ')))
Exception: web-push-testing-service error: {"error":{"id":"unable_to_get_subscription","message":"An issue occured while attempting to get the subscription: Error retrieving push subscription."}}
/home/travis/build/web-push-libs/web-push-php/tests/PushServiceTest.php:265
/home/travis/build/web-push-libs/web-push-php/tests/PushServiceTest.php:156
/home/travis/build/web-push-libs/web-push-php/tests/PushServiceTest.php:101
/home/travis/build/web-push-libs/web-push-php/tests/PushServiceTest.php:119
14) PushServiceTest::testBrowsers with data set #13 ('firefox', 'beta', array('AIzaSyBAU0VfXoskxUSg81K5VgLgw...ZWe6tA', array('http://test.com', 'BA6jvk34k6YjElHQ6S0oZwmrsqHdC...lLQHDk', '-3CdhFOqjzixgAbUSa0Zv9zi-dwDV...BxSFPQ')))
Exception: web-push-testing-service error: {"error":{"id":"unable_to_get_subscription","message":"An issue occured while attempting to get the subscription: Error retrieving push subscription."}}
/home/travis/build/web-push-libs/web-push-php/tests/PushServiceTest.php:265
/home/travis/build/web-push-libs/web-push-php/tests/PushServiceTest.php:156
/home/travis/build/web-push-libs/web-push-php/tests/PushServiceTest.php:101
/home/travis/build/web-push-libs/web-push-php/tests/PushServiceTest.php:119
baer95 commented 4 years ago

Looks like this is related to https://github.com/GoogleChromeLabs/web-push-testing-service and @Minishlink wanted to work on this last year, but did not get to it... he would probably welcome help though :)

https://github.com/web-push-libs/web-push-php/pull/228#issuecomment-500021751

reedy commented 4 years ago

It does seem that https://github.com/web-push-libs/web-push-php/commit/682b99f8a11c723099cf918e18350ee6b68f450e at least skips the tests from https://github.com/web-push-libs/web-push-php/pull/228#issuecomment-498856174

Minishlink commented 4 years ago

Yeah web-push-testing-service is hard to fix.. I tried back in February https://github.com/web-push-libs/web-push-php/tree/try-fix-test-once-more but no luck.

I don't see many solutions, either one of these:

  1. We fix web-push-testing-service completely, but it's very time consuming and I really don't have the time right now
  2. We fix web-push-testing-service but only for stable version of Chrome, looks a bit more promising, I may have some time next week or the week after
  3. We don't fix web-push-testing-service, and every PR should be accompanied by a video that shows that the PR changes work as expected, and that core feature (sending a simple push and receiving) works : I can accept PR quickly then but this introduces a burden on contributors

In any way, I can't accept PRs without being sure that core feature doesn't break...

If somebody wants to help, that would be highly appreciated

BR0kEN- commented 4 years ago

@Minishlink the best possible solution I see for the moment is to disable the tests that use the https://github.com/GoogleChromeLabs/web-push-testing-service. This will unblock all other tests and allow accepting unrelated to Web Push PRs like the one I've created - https://github.com/web-push-libs/web-push-php/pull/268

BR0kEN- commented 4 years ago

Also, it seems that it's worth to mention in README.md's head section that none of bug fixes/feature reqeusts will be accepted until the issue with tests is resolved so the community may not waste their time on contributing or can do that to their forks directly (as I was forced to do). Or otherwise, more people will be aware that help needed.