on the wire tests fails

[projekt01]

The over_the_wire.txt test fails on the initial request too not only on the later request with a query string. Probably google changed the rules.

Probably we should skip/remove the on the wire tests at all because it's not what the testbrowser is used to do.

D:\git\zope\zope.testbrowser>bin\test.exe -pv1
Running tests at level 1
Running zope.testrunner.layer.UnitTests tests:
  Set up zope.testrunner.layer.UnitTests in 0.000 seconds.
  Running:
    29/42 (69.0%) ... e\zope.testbrowser\src\zope\testbrowser\over_the_wire.txt

Failure in test d:\git\zope\zope.testbrowser\src\zope\testbrowser\over_the_wire.txt
Failed doctest test for over_the_wire.txt
  File "d:\git\zope\zope.testbrowser\src\zope\testbrowser\over_the_wire.txt", line 0

----------------------------------------------------------------------
File "d:\git\zope\zope.testbrowser\src\zope\testbrowser\over_the_wire.txt", line 21, in over_the_wire.txt
Failed example:
    browser.open('http://google.com/ncr')
Exception raised:
    Traceback (most recent call last):
      File "C:\Python27\lib\doctest.py", line 1289, in __run
        compileflags, 1) in test.globs
      File "<doctest over_the_wire.txt[3]>", line 1, in <module>
        browser.open('http://google.com/ncr')
      File "d:\git\zope\zope.testbrowser\src\zope\testbrowser\browser.py", line 244, in open
        self._processRequest(url, make_request)
      File "d:\git\zope\zope.testbrowser\src\zope\testbrowser\browser.py", line 266, in _processRequest
        resp = make_request(reqargs)
      File "d:\git\zope\zope.testbrowser\src\zope\testbrowser\browser.py", line 242, in <lambda>
        make_request = lambda args: self.testapp.get(url, **args)
      File "d:\home\.buildout\eggs\webtest-2.0.14-py2.7.egg\webtest\app.py", line 212, in get
        expect_errors=expect_errors)
      File "d:\git\zope\zope.testbrowser\src\zope\testbrowser\browser.py", line 81, in do_request
        self._assertAllowed(req.url)
      File "d:\git\zope\zope.testbrowser\src\zope\testbrowser\browser.py", line 78, in _assertAllowed
        raise RobotExclusionError(url, 403, msg, [], None)
    RobotExclusionError: HTTP Error 403: request disallowed by robots.txt

  Ran 42 tests with 1 failures, 0 errors and 0 skipped in 1.626 seconds.
Tearing down left over layers:
  Tear down zope.testrunner.layer.UnitTests in 0.000 seconds.

Tests with failures:
   d:\git\zope\zope.testbrowser\src\zope\testbrowser\over_the_wire.txt

[mgedmin]

Our tests should not depend on google.

If we want on-the-wire tests, we can spawn a subprocess running a hello world app with wsgiref.

[mgedmin]

"Fixed" in ad659fae4287a25d1127b68182ea64b1807becc2.

[tseaver]

https://mail.zope.org/pipermail/zope-tests/2016-April/093664.html

[florianpilz]

@tseaver Still an issue?

[tseaver]

@florianpilz The error seems to have cleared on 2016-05-15, but I cannot see any commit here which would explain that. Furthermore, the zope.testbrowser_py_270_32 builder is now gone from winbot, which makes sleuthing for it hard (maybe that is when the builder was dropped?)

The real underlying issue (making an across-the-wire request to a third-party service in a unit test) is still an issue.

[MeggyCal]

Looks like Google has changed the policy again (or am I wrong?):

 File "/home/abuild/rpmbuild/BUILD/zope.testbrowser-5.3.2/src/zope/testbrowser/over_the_wire.txt", line 21, in over_the_wire.txt
 Failed example:
     browser.open('http://google.com/ncr')
 Expected:
     Traceback (most recent call last):
     ...
     RobotExclusionError: HTTP Error 403: request disallowed by robots.txt
 Got:
     Traceback (most recent call last):
       File "/usr/lib64/python2.7/doctest.py", line 1315, in __run
         compileflags, 1) in test.globs
       File "<doctest over_the_wire.txt[3]>", line 1, in <module>
         browser.open('http://google.com/ncr')
       File "/home/abuild/rpmbuild/BUILD/zope.testbrowser-5.3.2/src/zope/testbrowser/browser.py", line 251, in open
         self._processRequest(url, make_request)
       File "/home/abuild/rpmbuild/BUILD/zope.testbrowser-5.3.2/src/zope/testbrowser/browser.py", line 275, in _processRequest
         resp = make_request(reqargs)
       File "/home/abuild/rpmbuild/BUILD/zope.testbrowser-5.3.2/src/zope/testbrowser/browser.py", line 249, in make_request
         return self.testapp.get(url, **args)
       File "/usr/lib/python2.7/site-packages/webtest/app.py", line 335, in get
         expect_errors=expect_errors)
       File "/home/abuild/rpmbuild/BUILD/zope.testbrowser-5.3.2/src/zope/testbrowser/browser.py", line 91, in do_request
         self._assertAllowed(req.url)
       File "/home/abuild/rpmbuild/BUILD/zope.testbrowser-5.3.2/src/zope/testbrowser/browser.py", line 85, in _assertAllowed
         rp.read()
       File "/usr/lib64/python2.7/robotparser.py", line 58, in read
         f = opener.open(self.url)
       File "/usr/lib64/python2.7/urllib.py", line 213, in open
         return getattr(self, name)(url)
       File "/usr/lib64/python2.7/urllib.py", line 350, in open_http
         h.endheaders(data)
       File "/usr/lib64/python2.7/httplib.py", line 1038, in endheaders
         self._send_output(message_body)
       File "/usr/lib64/python2.7/httplib.py", line 882, in _send_output
         self.send(msg)
       File "/usr/lib64/python2.7/httplib.py", line 844, in send
         self.connect()
       File "/usr/lib64/python2.7/httplib.py", line 821, in connect
         self.timeout, self.source_address)
       File "/usr/lib64/python2.7/socket.py", line 557, in create_connection
         for res in getaddrinfo(host, port, 0, SOCK_STREAM):
     IOError: [Errno socket error] [Errno -3] Temporary failure in name resolution

[mgedmin]

No, the error you see here indicates that the tests were unable to resolve google.com.

(I still think it's a mistake for zope.testbrowser's tests to do real HTTP requests to 3rd-party websites, but I've no energy to fix this.)