search

torproject / stem

Python controller library for Tor
https://stem.torproject.org/
GNU Lesser General Public License v3.0
257 stars 75 forks source link

Adds ClientAuthV3 support to the controller, for setting Client Auth with ADD_ONION on v3 onions. - 1.8.x version #99

Closed mig5 closed 3 years ago

mig5 commented 3 years ago

Also sets some v2 tests to require older Tor versions so that tests for Client Auth on v3 onions pass on newer Tor without failing tests where v2 support has been dropped.

@atagar this has been branched directly off of the 1.8 tag in order to maybe get a 1.8.1 point release as per discussion in https://github.com/torproject/stem/pull/98 . I can't choose anything but master branch to merge this into, which will obviously conflict.. maybe needs a 1.8.0 branch made and then set as the target.

Please also note that on this branch I seem to get some test failures that look entirely unrelated to this work, presumably it's either a) an issue with my setup locally (noting the expected 'listener address' in the output) or b) existing issues in the 1.8.x code. Nonetheless the new tests pass with the nightly tor 0.4.6 alpha installed, and I have also brought across the changes needed to skip the v2 tests on that newer tor.

Running tests...

  util.conf...                                         success (0.00s)
  util.connection...                                   success (0.00s)
  util.proc...                                         success (0.00s)
  util.system...                                       success (0.22s)
  interpreter...                                       success (0.15s)
  version...                                           success (0.01s)
  manual...                                            success (0.00s)
  directory.authority...                               success (0.00s)
  directory.fallback...                                success (0.00s)
  client.connection...                                 success (0.04s)
  response.protocolinfo...                             success (0.01s)
  socket.control_socket...                             success (0.02s)
  socket.control_message...                            success (0.01s)
  connection.authentication...                         success (0.02s)
  connection.connect...                                success (0.00s)
  control.base_controller...                           success (0.54s)
  control.controller...                                failed (0.65s)
test_attachstream (requires online target)                   [SKIPPED]
test_authenticate                                      5 ms  [SUCCESS]
test_close_circuit (requires online target)                  [SKIPPED]
test_close_stream (requires online target)                   [SKIPPED]
test_enable_feature (requires online target)                 [SKIPPED]
test_ephemeral_hidden_services_v2 (requires 0.4.6.1-alpha)   [SKIPPED]
test_ephemeral_hidden_services_v3 (requires 0.4.6.1-alpha)   [SKIPPED]
test_event_handling                                   20 ms  [SUCCESS]
test_extendcircuit (requires online target)                  [SKIPPED]
test_from_port                                        12 ms  [SUCCESS]
test_from_socket_file                                  0 ms  [SUCCESS]
test_get_circuits (requires online target)                   [SKIPPED]
test_get_exit_policy                                   7 ms  [SUCCESS]
test_get_hidden_service_descriptor (requires online target)  [SKIPPED]
test_get_listeners                                           [FAILURE]
test_get_microdescriptor (requires online target)            [SKIPPED]
test_get_microdescriptors (requires online target)           [SKIPPED]
test_get_network_status (requires online target)             [SKIPPED]
test_get_network_statuses (requires online target)           [SKIPPED]
test_get_ports                                               [FAILURE]
test_get_server_descriptor (requires server descriptors)     [SKIPPED]
test_get_server_descriptors (requires online target)         [SKIPPED]
test_get_socks_listeners                               4 ms  [SUCCESS]
test_get_streams (requires online target)                    [SKIPPED]
test_get_version                                       6 ms  [SUCCESS]
test_getconf                                          11 ms  [SUCCESS]
test_getinfo                                           7 ms  [SUCCESS]
test_getinfo_dir_status (requires online target)             [SKIPPED]
test_getinfo_freshrelaydescs                                 [FAILURE]
test_hidden_services_conf (requires 0.4.6.1-alpha)           [SKIPPED]
test_is_set                                           11 ms  [SUCCESS]
test_loadconf                                         12 ms  [SUCCESS]
test_mapaddress (requires online target)                     [SKIPPED]
test_mapaddress_offline                                6 ms  [SUCCESS]
test_missing_capabilities                              3 ms  [SUCCESS]
test_newnym_availability                               3 ms  [SUCCESS]
test_protocolinfo                                      2 ms  [SUCCESS]
test_reattaching_listeners                            11 ms  [SUCCESS]
test_rejecting_unanonymous_hidden_services_creation    5 ms  [SUCCESS]
test_repurpose_circuit (requires online target)              [SKIPPED]
test_reset_notification                                9 ms  [SUCCESS]
test_saveconf                                         11 ms  [SUCCESS]
test_set_conf                                         15 ms  [SUCCESS]
test_set_conf_for_usebridges                         244 ms  [SUCCESS]
test_set_conf_when_immutable                           4 ms  [SUCCESS]
test_signal                                            3 ms  [SUCCESS]
test_transition_to_relay                             170 ms  [SUCCESS]
test_with_detached_ephemeral_hidden_services          20 ms  [SUCCESS]
test_with_ephemeral_hidden_services_basic_auth (requires 0.4.6.1-alpha)[SKIPPED]
test_with_ephemeral_hidden_services_basic_auth_no_credentials (requires 0.4.6.1-alpha)[SKIPPED]
test_with_ephemeral_hidden_services_v3_client_auth     6 ms  [SUCCESS]
test_with_ephemeral_hidden_services_v3_client_auth_invalid  5 ms  [SUCCESS]
test_with_invalid_ephemeral_hidden_service_port (requires 0.4.6.1-alpha)[SKIPPED]
test_without_ephemeral_hidden_services                 5 ms  [SUCCESS]

======================================================================
ERROR: test_getinfo_freshrelaydescs
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/user/git/stem/test/require.py", line 58, in wrapped
    return func(self, *args, **kwargs)
  File "/home/user/git/stem/test/integ/control/controller.py", line 278, in test_getinfo_freshrelaydescs
    self.assertEqual(controller.get_info('address'), server_desc.address)
  File "/home/user/git/stem/stem/control.py", line 489, in wrapped
    return func(self, *args, **kwargs)
  File "/home/user/git/stem/stem/control.py", line 1209, in get_info
    stem.response.convert('GETINFO', response)
  File "/home/user/git/stem/stem/response/__init__.py", line 124, in convert
    message._parse_message(**kwargs)
  File "/home/user/git/stem/stem/response/getinfo.py", line 46, in _parse_message
    raise stem.OperationFailed(error_code, error_msg)
stem.OperationFailed: Address unknown

======================================================================
FAIL: test_get_listeners
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/user/git/stem/test/require.py", line 58, in wrapped
    return func(self, *args, **kwargs)
  File "/home/user/git/stem/test/integ/control/controller.py", line 986, in test_get_listeners
    self.assertEqual([('0.0.0.0', test.runner.ORPORT)], controller.get_listeners(Listener.OR))
AssertionError: Lists differ: [('0.0.0.0', 1113)] != [('0.0.0.0', 1113), ('::', 1113)]

Second list contains 1 additional elements.
First extra element 1:
('::', 1113)

- [('0.0.0.0', 1113)]
+ [('0.0.0.0', 1113), ('::', 1113)]

======================================================================
FAIL: test_get_ports
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/user/git/stem/test/require.py", line 58, in wrapped
    return func(self, *args, **kwargs)
  File "/home/user/git/stem/test/integ/control/controller.py", line 965, in test_get_ports
    self.assertEqual([test.runner.ORPORT], controller.get_ports(Listener.OR))
AssertionError: Lists differ: [1113] != [1113, 1113]

Second list contains 1 additional elements.
First extra element 1:
1113

- [1113]
+ [1113, 1113]

----------------------------------------------------------------------
Ran 54 tests in 0.653s

FAILED (failures=2, errors=1, skipped=23)
  descriptor.collector...                              success (0.00s)
  descriptor.remote...                                 success (0.00s)
  descriptor.server_descriptor...                      success (0.00s)
  descriptor.extrainfo_descriptor...                   success (0.00s)
  descriptor.microdescriptor...                        success (0.00s)
  descriptor.networkstatus...                          success (0.00s)
  installation...                                      success (0.75s)
  process...                                           failed (12.66s)
test_can_run_multithreaded                             0 ms  [SUCCESS]
test_dump_config_argument                              0 ms  [SUCCESS]
test_hash_password                                     0 ms  [SUCCESS]
test_hash_password_requires_argument                   0 ms  [SUCCESS]
test_help_argument                                     0 ms  [SUCCESS]
test_hush_argument                                     0 ms  [SUCCESS]
test_launch_tor_with_config_via_file                   0 ms  [SUCCESS]
test_launch_tor_with_config_via_stdin                  0 ms  [SUCCESS]
test_launch_tor_with_timeout                          94 ms  [SUCCESS]
test_list_fingerprint_argument                         0 ms  [SUCCESS]
test_list_torrc_options_argument                       0 ms  [SUCCESS]
test_no_orphaned_process                               0 ms  [SUCCESS]
test_quiet_argument                                    0 ms  [SUCCESS]
test_take_ownership_via_controller                     0 ms  [SUCCESS]
test_take_ownership_via_pid                          12.56s  [SUCCESS]
test_torrc_arguments                                   0 ms  [SUCCESS]
test_torrc_arguments_via_stdin                         0 ms  [SUCCESS]
test_unanonymous_hidden_service_config_must_match      0 ms  [SUCCESS]
test_validate_config_argument                          0 ms  [SUCCESS]
test_version_argument                                        [FAILURE]
test_with_invalid_config                               0 ms  [SUCCESS]
test_with_missing_torrc                                0 ms  [SUCCESS]

======================================================================
FAIL: test_version_argument
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/user/git/stem/stem/util/test_tools.py", line 152, in <lambda>
    self.method = lambda test: self.result(test)  # method that can be mixed into TestCases
  File "/home/user/git/stem/stem/util/test_tools.py", line 225, in result
    test.fail(self._result.msg)
AssertionError: Expected 'Tor version 0.4.7.0-alpha-dev.
' but was 'Tor version 0.4.7.0-alpha-dev.
Tor is running on Linux with Libevent 2.1.8-stable, OpenSSL 1.1.1d, Zlib 1.2.11, Liblzma 5.2.4, Libzstd 1.3.8 and Glibc 2.28 as libc.
Tor compiled with GCC version 8.3.0
'

----------------------------------------------------------------------
Ran 22 tests in 12.655s

FAILED (failures=1)

Shutting down tor... done

STATIC CHECKS
* /home/user/git/stem/stem/util/ed25519.py
  line 59   - undefined name 'xrange'                  | range = xrange

* /home/user/git/stem/stem/descriptor/hidden_service.py
  line 289  - E741 ambiguous variable name 'l'         | link_specifiers = link_count + b''.join([l.pack() for l in self.link_specifiers])

* /home/user/git/stem/stem/interpreter/commands.py
  line 274  - E741 ambiguous variable name 'l'         | lines += [format(l, *STANDARD_OUTPUT) for l in str(desc).splitlines()]

* /home/user/git/stem/test/integ/control/controller.py
  line 39   - E302 expected 2 blank lines, found 1     | class TestController(unittest.TestCase):

TESTING FAILED (15 seconds)
  [RUN_OPEN] test_get_listeners (test.integ.control.controller.TestController) ... FAIL
  [RUN_OPEN] test_get_ports (test.integ.control.controller.TestController) ... FAIL
  [RUN_OPEN] test_getinfo_freshrelaydescs (test.integ.control.controller.TestController) ... ERROR
  [RUN_OPEN] test_version_argument (test.integ.process.TestProcess) ... FAIL

You can re-run just these tests with:

  run_tests.py --integ --test process
  run_tests.py --integ --test control.controller

Note also the Tor version test failure because the nightly version has a very long version string with supplementary information. I modified the test to use your assert_in() as a slightly more relaxed check, I hope that's ok.

mig5 commented 3 years ago

Yay, the core Tor bug that prevented ADD_ONION working with ClientAuthV3 properly, has been fixed and merged. https://gitlab.torproject.org/tpo/core/tor/-/merge_requests/374

I tested this (Stem 1.8.x) patch with the nightly 0.4.6.x alpha release on Tor's debian repository and it really does work now as expected :) :) Exciting!

atagar commented 3 years ago

Thanks mig5! Merged as a new 'maint' branch for 1.8 work.

https://github.com/torproject/stem/tree/maint