Refactor out global settings usage in exclusions

[gfyoung]

Title is self-explanatory.

Follow-up to #356.

[gfyoung]

@StevenBlack : friendly ping

[StevenBlack]

Hi @gfyoung, sorry for the delay.

Testing here, I get this:

$ python testUpdateHostsFile.py
Traceback (most recent call last):
  File "testUpdateHostsFile.py", line 35, in <module>
    import mock
ImportError: No module named mock

$ python --version
Python 2.7.10

[StevenBlack]

Testing with Python3, here's what I get.

$ python3 testUpdateHostsFile.py
......EE...................................EE.E.EE.EEE..E....................................
======================================================================
ERROR: test_gather_exclusions (__main__.TestDisplayExclusionsOptions)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/unittest/mock.py", line 1157, in patched
    return func(*args, **keywargs)
  File "testUpdateHostsFile.py", line 447, in test_gather_exclusions
    mock_gather.assert_called_once()
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/unittest/mock.py", line 583, in __getattr__
    raise AttributeError(name)
AttributeError: assert_called_once

======================================================================
ERROR: test_mixture_gather_exclusions (__main__.TestDisplayExclusionsOptions)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/unittest/mock.py", line 1157, in patched
    return func(*args, **keywargs)
  File "testUpdateHostsFile.py", line 457, in test_mixture_gather_exclusions
    mock_gather.assert_called_once()
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/unittest/mock.py", line 583, in __getattr__
    raise AttributeError(name)
AttributeError: assert_called_once

======================================================================
ERROR: testNoSkipPromptDisplay (__main__.TestPromptForExclusions)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/unittest/mock.py", line 1157, in patched
    return func(*args, **keywargs)
  File "testUpdateHostsFile.py", line 287, in testNoSkipPromptDisplay
    mock_query.assert_called_once()
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/unittest/mock.py", line 583, in __getattr__
    raise AttributeError(name)
AttributeError: assert_called_once

======================================================================
ERROR: testNoSkipPromptNoDisplay (__main__.TestPromptForExclusions)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/unittest/mock.py", line 1157, in patched
    return func(*args, **keywargs)
  File "testUpdateHostsFile.py", line 277, in testNoSkipPromptNoDisplay
    mock_query.assert_called_once()
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/unittest/mock.py", line 583, in __getattr__
    raise AttributeError(name)
AttributeError: assert_called_once

======================================================================
ERROR: testFlushCache (__main__.TestPromptForFlushDnsCache)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/unittest/mock.py", line 1157, in patched
    return func(*args, **keywargs)
  File "testUpdateHostsFile.py", line 300, in testFlushCache
    mock_flush.assert_called_once()
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/unittest/mock.py", line 583, in __getattr__
    raise AttributeError(name)
AttributeError: assert_called_once

======================================================================
ERROR: testNoFlushCachePromptFlush (__main__.TestPromptForFlushDnsCache)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/unittest/mock.py", line 1157, in patched
    return func(*args, **keywargs)
  File "testUpdateHostsFile.py", line 329, in testNoFlushCachePromptFlush
    mock_query.assert_called_once()
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/unittest/mock.py", line 583, in __getattr__
    raise AttributeError(name)
AttributeError: assert_called_once

======================================================================
ERROR: testNoFlushCachePromptNoFlush (__main__.TestPromptForFlushDnsCache)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/unittest/mock.py", line 1157, in patched
    return func(*args, **keywargs)
  File "testUpdateHostsFile.py", line 320, in testNoFlushCachePromptNoFlush
    mock_query.assert_called_once()
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/unittest/mock.py", line 583, in __getattr__
    raise AttributeError(name)
AttributeError: assert_called_once

======================================================================
ERROR: testPromptMove (__main__.TestPromptForMove)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/unittest/mock.py", line 1157, in patched
    return func(*args, **keywargs)
  File "testUpdateHostsFile.py", line 405, in testPromptMove
    mock_query.assert_called_once()
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/unittest/mock.py", line 583, in __getattr__
    raise AttributeError(name)
AttributeError: assert_called_once

======================================================================
ERROR: testPromptNoMove (__main__.TestPromptForMove)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/unittest/mock.py", line 1157, in patched
    return func(*args, **keywargs)
  File "testUpdateHostsFile.py", line 392, in testPromptNoMove
    mock_query.assert_called_once()
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/unittest/mock.py", line 583, in __getattr__
    raise AttributeError(name)
AttributeError: assert_called_once

======================================================================
ERROR: testReplaceNoSkipStaticHosts (__main__.TestPromptForMove)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/unittest/mock.py", line 1157, in patched
    return func(*args, **keywargs)
  File "testUpdateHostsFile.py", line 366, in testReplaceNoSkipStaticHosts
    mock_move.assert_called_once()
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/unittest/mock.py", line 583, in __getattr__
    raise AttributeError(name)
AttributeError: assert_called_once

======================================================================
ERROR: test_freshen_update (__main__.TestPromptForUpdate)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/unittest/mock.py", line 1157, in patched
    return func(*args, **keywargs)
  File "testUpdateHostsFile.py", line 236, in test_freshen_update
    mock_update.assert_called_once()
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/unittest/mock.py", line 583, in __getattr__
    raise AttributeError(name)
AttributeError: assert_called_once

----------------------------------------------------------------------
Ran 93 tests in 0.084s

FAILED (errors=11)

[StevenBlack]

$ python3 --version
Python 3.5.1

[gfyoung]

@StevenBlack : For Python 2.x, did you install the mock library (I've been using this library for testing for quite some time for these unit tests)?

For Python 3.5, sigh...it appears that assert_called_once only showed up in Python 3.6 (we test only with Python 3.6 on Travis). Perhaps it would be worthwhile subsequently to add a Python 3.5 build on Travis given this incompatibility.

[StevenBlack]

@gfyoung

For Python 2.x, did you install the mock library (I've been using this library for testing for quite some time for these unit tests)?

I tried :-) The weekend machine I was testing on, an iMAc running OSX latest, wouldn't install it.

Today I get the same error on my MacBook Pro running OSX latest. Mock is installed on this machine:

$ pip install mock
Requirement already satisfied: mock in /usr/local/lib/python2.7/site-packages
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /usr/local/lib/python2.7/site-packages (from mock)
Requirement already satisfied: six>=1.9 in /usr/local/lib/python2.7/site-packages (from mock)
Requirement already satisfied: pbr>=0.11 in /usr/local/lib/python2.7/site-packages (from mock)

But still...

$ python testUpdateHostsFile.py
Traceback (most recent call last):
  File "testUpdateHostsFile.py", line 35, in <module>
    import mock
ImportError: No module named mock

on this machine

$ python --version
Python 2.7.10

[rautamiekka]

Since import mock keeps on failing, shut down Python, set PYTHONVERBOSE environment variable to 1, then try importing. The repeating failure tells me Python is trying to get mock from elsewhere than you've got it saved to, which in turn likely means Python is preconffed to not look where mock is.

[gfyoung]

@StevenBlack : Just how did you confirm my other test PRs then? Surely you must have had a working Python environment for those. 😄

[gfyoung]

I would suggest try creating a virtualenv or a conda environment and then running tests in that environment. That should (hopefully) ensure that Python will find the library without any issues OR you can try uninstalling and re-installing mock on your root environment.

[gfyoung]

The (slightly) more pressing question for me is what to do with Python 3.5. Give some of the incompatibilities, I'm perfectly happy to adjust tests (as a follow-up) to account for Python 3.5's mock library. What do you think?

[StevenBlack]

I think I'm going to rewrite all this in something that isn't Python.

[StevenBlack]

@gfyoung uninstalling mock doesn't work.

$ pip uninstall mock
Uninstalling mock-2.0.0:
  /usr/local/lib/python2.7/site-packages/mock-2.0.0.dist-info/DESCRIPTION.rst
  /usr/local/lib/python2.7/site-packages/mock-2.0.0.dist-info/INSTALLER
  /usr/local/lib/python2.7/site-packages/mock-2.0.0.dist-info/METADATA
  /usr/local/lib/python2.7/site-packages/mock-2.0.0.dist-info/RECORD
  /usr/local/lib/python2.7/site-packages/mock-2.0.0.dist-info/WHEEL
  /usr/local/lib/python2.7/site-packages/mock-2.0.0.dist-info/metadata.json
  /usr/local/lib/python2.7/site-packages/mock-2.0.0.dist-info/pbr.json
  /usr/local/lib/python2.7/site-packages/mock-2.0.0.dist-info/top_level.txt
  /usr/local/lib/python2.7/site-packages/mock/__init__.py
  /usr/local/lib/python2.7/site-packages/mock/__init__.pyc
  /usr/local/lib/python2.7/site-packages/mock/mock.py
  /usr/local/lib/python2.7/site-packages/mock/mock.pyc
  /usr/local/lib/python2.7/site-packages/mock/tests/__init__.py
  /usr/local/lib/python2.7/site-packages/mock/tests/__init__.pyc
  /usr/local/lib/python2.7/site-packages/mock/tests/__main__.py
  /usr/local/lib/python2.7/site-packages/mock/tests/__main__.pyc
  /usr/local/lib/python2.7/site-packages/mock/tests/support.py
  /usr/local/lib/python2.7/site-packages/mock/tests/support.pyc
  /usr/local/lib/python2.7/site-packages/mock/tests/testcallable.py
  /usr/local/lib/python2.7/site-packages/mock/tests/testcallable.pyc
  /usr/local/lib/python2.7/site-packages/mock/tests/testhelpers.py
  /usr/local/lib/python2.7/site-packages/mock/tests/testhelpers.pyc
  /usr/local/lib/python2.7/site-packages/mock/tests/testmagicmethods.py
  /usr/local/lib/python2.7/site-packages/mock/tests/testmagicmethods.pyc
  /usr/local/lib/python2.7/site-packages/mock/tests/testmock.py
  /usr/local/lib/python2.7/site-packages/mock/tests/testmock.pyc
  /usr/local/lib/python2.7/site-packages/mock/tests/testpatch.py
  /usr/local/lib/python2.7/site-packages/mock/tests/testpatch.pyc
  /usr/local/lib/python2.7/site-packages/mock/tests/testsentinel.py
  /usr/local/lib/python2.7/site-packages/mock/tests/testsentinel.pyc
  /usr/local/lib/python2.7/site-packages/mock/tests/testwith.py
  /usr/local/lib/python2.7/site-packages/mock/tests/testwith.pyc
Proceed (y/n)? y
  Successfully uninstalled mock-2.0.0

Steve at 0xf  ~/Dropbox/dev/hosts on pr/366
$ pip install mock
Collecting mock
  Using cached mock-2.0.0-py2.py3-none-any.whl
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /usr/local/lib/python2.7/site-packages (from mock)
Requirement already satisfied: six>=1.9 in /usr/local/lib/python2.7/site-packages (from mock)
Requirement already satisfied: pbr>=0.11 in /usr/local/lib/python2.7/site-packages (from mock)
Installing collected packages: mock
Successfully installed mock-2.0.0

$ python testUpdateHostsFile.py
Traceback (most recent call last):
  File "testUpdateHostsFile.py", line 35, in <module>
    import mock
ImportError: No module named mock

[gfyoung]

I think I'm going to rewrite all this in something that isn't Python.

Want to give a shot at Windows Batch Script? 😄

[StevenBlack]

At the moment I would entertain a PR that completely removes mock-based testing.

[gfyoung]

At the moment I would entertain a PR that completely removes mock-based testing.

To be fair, I'm a little confused why this issue with mock has resurfaced given that you were able to run this file beforehand with your Python 2.x environment.

That being said, the reason why I chose to use mock was because it would allow us to make our tests (almost) OS-agnostic.

[StevenBlack]

It's not working now on two different Macs. The only thing I can think-of is I recently updaded MacOS on both.

Can we add mock to the repo in a subfolder, and import it explicitly?

[gfyoung]

Can we add mock to the repo in a subfolder, and import it explicitly?

I'm hesitant to do this because that's just extra "stuff" a user has to download when doing a git clone

The reason why I'm trying to figure out what's going on with your Mac environment is because the tests run without problems on Travis OSX, and those are largely vanilla OSX environments with a bunch of packages installed for developer convenience and testing.

FWIW, I can run these tests successfully on a personal desktop Mac from git clone to testing as well using Python 2.x (just did it now).

[StevenBlack]

OK @gfyoung on this other iMac python testUpdateHostsFile.py executes cleanly under Python 2.7.11. So that's great.

There's still the issue of it failing under Python 3.5.1.

[gfyoung]

Yes, that problem I realize is a compatibility issue. Python 3.5 builtin mock library is missing several important functions that are available in Python 3.6 and in the port to Python 2.

I'll first make a PR to add Python 3.5 to Travis and then fix up this PR.

[gfyoung]

@StevenBlack : With #373 merged in, I've updated this PR to be Python 3.5.x-friendly.

[StevenBlack]

@gfyoung Beauty! Everything works great.

Just one quibble: why do we have such a huge diff with updateHostsFile.py in this PR? That's the only blocker before merging this, because I can't evaluate this large, unspecific diff.

[gfyoung]

@StevenBlack : Gosh darn it. I blame Git for this. I was working on a different machine, and I think line-endings got the best of me there. Good catch. 😄

[StevenBlack]

Happens to me all the time :-)

[gfyoung]

I'll go back and fix that and push again.

[gfyoung]

@StevenBlack : I think that diff looks a lot better now, right? 😄

[StevenBlack]

Perfect. Thanks @gfyoung !