reworked cache directory with new framework

[RomanBelozerov]

I added start_all_services() for TempestaTest class and send_request() for DeproxyClient class because it repeats in all tests. I used parametrization by methods because it looks more clear for me.

[krizhanovsky]

Just a comment: in #265 we rename frang to t_frang to make the directories with the test cases more observable from the service directories (e.g. framework), so it makes sense to do this with other directories which you touch. @b3b @pale-emperor @nickzaev FYI

[RomanBelozerov]

Blinking tests. Is it introduced with changes or was here before?

FAIL: test (cache.test_cache_control.StoringResponsesToAuthenticatedRequestsMustRevalidateCache2)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/root/tempesta-test/cache/test_cache_control.py", line 184, in test
    self._test()
  File "/root/tempesta-test/cache/test_cache_control.py", line 171, in _test
    self.assertEqual(1, len(srv.requests), 'response not cached as expected')
AssertionError: 1 != 2 : response not cached as expected

----------------------------------------------------------------------

======================================================================
FAIL: test (cache.test_cache_control.ResponseSMaxageCached)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/root/tempesta-pr/cache/test_cache_control.py", line 184, in test
    self._test()
  File "/root/tempesta-pr/cache/test_cache_control.py", line 171, in _test
    self.assertEqual(1, len(srv.requests), 'response not cached as expected')
AssertionError: 1 != 2 : response not cached as expected

----------------------------------------------------------------------

======================================================================
FAIL: test (cache.test_cache_control.ResponseMaxAgeCached2)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/root/tempesta-test/cache/test_cache_control.py", line 184, in test
    self._test()
  File "/root/tempesta-test/cache/test_cache_control.py", line 171, in _test
    self.assertEqual(1, len(srv.requests), 'response not cached as expected')
AssertionError: 1 != 2 : response not cached as expected

----------------------------------------------------------------------

I didn't touch or check test_cache_control.py. StoringResponsesToAuthenticatedRequestsMustRevalidateCache2 has been disabled in #329.

When backend server running on separate host (X-Forwarded-For header != 127.0.0.1):

FAIL: test_0_purge_non_hch (cache.test_purge_hch.HeavyChunkedPurgeTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/b3bf/git/tempesta-pr/cache/test_purge_hch.py", line 96, in test_0_purge_non_hch
    self.common_check(
  File "/home/b3bf/git/tempesta-pr/cache/test_purge_hch.py", line 89, in common_check
    self.assertTrue(
AssertionError: False is not true : Request sent to backend differs from expected one

======================================================================
FAIL: test_1_purge_hch (cache.test_purge_hch.HeavyChunkedPurgeTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/b3bf/git/tempesta-pr/cache/test_purge_hch.py", line 118, in test_1_purge_hch
    self.common_check(
  File "/home/b3bf/git/tempesta-pr/cache/test_purge_hch.py", line 89, in common_check
    self.assertTrue(
AssertionError: False is not true : Request sent to backend differs from expected one

----------------------------------------------------------------------

Fixed.

[b3b]

@RomanBelozerov, about expected headers mismatch, I will check my setup why headers go in different order. It is OK on other test hosts, and does not seems like related to this PR. But some cache tests still has X-Forwarded-For: 127.0.0.1 hardcoded as expected values.

[const-t]

Still has errors when Server is separate from Tempesta:

======================================================================
FAIL: test_0_purge_non_hch (cache.test_purge_hch.HeavyChunkedPurgeTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/b3bf/git/tempesta-pr/cache/test_purge_hch.py", line 98, in test_0_purge_non_hch
    self.common_check(
  File "/home/b3bf/git/tempesta-pr/cache/test_purge_hch.py", line 90, in common_check
    self.assertTrue(
AssertionError: False is not true : Request sent to backend differs from expected one

======================================================================
FAIL: test_1_purge_hch (cache.test_purge_hch.HeavyChunkedPurgeTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/b3bf/git/tempesta-pr/cache/test_purge_hch.py", line 115, in test_1_purge_hch
    self.common_check(
  File "/home/b3bf/git/tempesta-pr/cache/test_purge_hch.py", line 90, in common_check
    self.assertTrue(
AssertionError: False is not true : Request sent to backend differs from expected one

======================================================================
FAIL: test_0_purge_resp_non_hch (cache.test_purge_resp_hch.HeavyChunkedPurgeRespTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/b3bf/git/tempesta-pr/cache/test_purge_resp_hch.py", line 138, in test_0_purge_resp_non_hch
    self.common_check(
  File "/home/b3bf/git/tempesta-pr/cache/test_purge_resp_hch.py", line 119, in common_check
    self.assertTrue(
AssertionError: False is not true : Request sent to backend differs from expected one with chunksize = 0

----------------------------------------------------------------------

X-Forwarded-For has Tempesta IP instead of the Server. And seems like headers can go in different order:

expect:

'GET / HTTP/1.1\r\nHost: localhost\r\nX-Tempesta-Cache: GET\r\nX-Forwarded-For: 127.0.0.1\r\nvia: 1.1 tempesta_fw (Tempesta FW pre-0.7.0)\r\nConnection: keep-alive\r\n\r\n'

frequest.original_data:

'GET / HTTP/1.1\r\nHost: localhost\r\nX-Tempesta-Cache: GET\r\nConnection: keep-alive\r\nvia: 1.1 tempesta_fw (Tempesta FW pre-0.7.0)\r\nX-Forwarded-For: 192.168.122.1\r\n\r\n'

i guess you get different order of X-Forwarded-For because for each request tempesta removes existing XFF header, and adds new, with IP of tempesta.