add functional tests

[ydlr]

Add tests for deploying nixos to a droplet, enabling ipv6, and rolling back changes.

Add nose and coverage as dev dependencies. Update coverage-tests.py to use python3.

[ixxie]

@ydlr sorry for responding slowly... I am not exactly sure I have the time to be a full-time maintainer here.

The test is failing when I try running your branch:

[nix-shell:~/sparklet/repos/nixops-digitalocean]$ nosetests
EEEdeployment ‘51f2ce08-08d5-11eb-ad1c-b88a606a8655’ destroyed
deployment ‘5197c9ca-08d5-11eb-a33c-b88a606a8655’ destroyed
deployment ‘524cf8a6-08d5-11eb-bce6-b88a606a8655’ destroyed

======================================================================
ERROR: tests.functional.test_deploys_nixos.TestDeploysNixos.test_do_droplet
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/nix/store/3bn7qlh80p91kgd8q3myy3f56ai2v1fs-python3-3.7.9-env/lib/python3.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/home/ixxie/sparklet/repos/nixops-digitalocean/tests/functional/single_droplet_test.py", line 21, in test_do_droplet
    self.run_check()
  File "/home/ixxie/sparklet/repos/nixops-digitalocean/tests/functional/test_deploys_nixos.py", line 8, in run_check
    self.depl.deploy()
  File "/nix/store/3bn7qlh80p91kgd8q3myy3f56ai2v1fs-python3-3.7.9-env/lib/python3.7/site-packages/nixops/deployment.py", line 1479, in deploy
    self.run_with_notify("deploy", lambda: self._deploy(**kwargs))
  File "/nix/store/3bn7qlh80p91kgd8q3myy3f56ai2v1fs-python3-3.7.9-env/lib/python3.7/site-packages/nixops/deployment.py", line 1468, in run_with_notify
    f()
  File "/nix/store/3bn7qlh80p91kgd8q3myy3f56ai2v1fs-python3-3.7.9-env/lib/python3.7/site-packages/nixops/deployment.py", line 1479, in <lambda>
    self.run_with_notify("deploy", lambda: self._deploy(**kwargs))
  File "/nix/store/3bn7qlh80p91kgd8q3myy3f56ai2v1fs-python3-3.7.9-env/lib/python3.7/site-packages/nixops/deployment.py", line 1266, in _deploy
    self.evaluate_active(include, exclude, kill_obsolete)
  File "/nix/store/3bn7qlh80p91kgd8q3myy3f56ai2v1fs-python3-3.7.9-env/lib/python3.7/site-packages/nixops/deployment.py", line 1207, in evaluate_active
    self.evaluate()
  File "/nix/store/3bn7qlh80p91kgd8q3myy3f56ai2v1fs-python3-3.7.9-env/lib/python3.7/site-packages/nixops/deployment.py", line 565, in evaluate
    defn = _create_definition(name, cfg, cfg["targetEnv"])
  File "/nix/store/3bn7qlh80p91kgd8q3myy3f56ai2v1fs-python3-3.7.9-env/lib/python3.7/site-packages/nixops/deployment.py", line 1800, in _create_definition
    "unknown resource type ‘{0}’".format(type_name)
nixops.deployment.UnknownBackend: unknown resource type ‘droplet’

======================================================================
ERROR: tests.functional.test_ipv6.TestIpv6.test_do_droplet
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/nix/store/3bn7qlh80p91kgd8q3myy3f56ai2v1fs-python3-3.7.9-env/lib/python3.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/home/ixxie/sparklet/repos/nixops-digitalocean/tests/functional/single_droplet_test.py", line 21, in test_do_droplet
    self.run_check()
  File "/home/ixxie/sparklet/repos/nixops-digitalocean/tests/functional/test_ipv6.py", line 18, in run_check
    self.depl.deploy()
  File "/nix/store/3bn7qlh80p91kgd8q3myy3f56ai2v1fs-python3-3.7.9-env/lib/python3.7/site-packages/nixops/deployment.py", line 1479, in deploy
    self.run_with_notify("deploy", lambda: self._deploy(**kwargs))
  File "/nix/store/3bn7qlh80p91kgd8q3myy3f56ai2v1fs-python3-3.7.9-env/lib/python3.7/site-packages/nixops/deployment.py", line 1468, in run_with_notify
    f()
  File "/nix/store/3bn7qlh80p91kgd8q3myy3f56ai2v1fs-python3-3.7.9-env/lib/python3.7/site-packages/nixops/deployment.py", line 1479, in <lambda>
    self.run_with_notify("deploy", lambda: self._deploy(**kwargs))
  File "/nix/store/3bn7qlh80p91kgd8q3myy3f56ai2v1fs-python3-3.7.9-env/lib/python3.7/site-packages/nixops/deployment.py", line 1266, in _deploy
    self.evaluate_active(include, exclude, kill_obsolete)
  File "/nix/store/3bn7qlh80p91kgd8q3myy3f56ai2v1fs-python3-3.7.9-env/lib/python3.7/site-packages/nixops/deployment.py", line 1207, in evaluate_active
    self.evaluate()
  File "/nix/store/3bn7qlh80p91kgd8q3myy3f56ai2v1fs-python3-3.7.9-env/lib/python3.7/site-packages/nixops/deployment.py", line 565, in evaluate
    defn = _create_definition(name, cfg, cfg["targetEnv"])
  File "/nix/store/3bn7qlh80p91kgd8q3myy3f56ai2v1fs-python3-3.7.9-env/lib/python3.7/site-packages/nixops/deployment.py", line 1800, in _create_definition
    "unknown resource type ‘{0}’".format(type_name)
nixops.deployment.UnknownBackend: unknown resource type ‘droplet’

======================================================================
ERROR: tests.functional.test_rollback_rollsback.TestRollbackRollsback.test_do_droplet
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/nix/store/3bn7qlh80p91kgd8q3myy3f56ai2v1fs-python3-3.7.9-env/lib/python3.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/home/ixxie/sparklet/repos/nixops-digitalocean/tests/functional/single_droplet_test.py", line 21, in test_do_droplet
    self.run_check()
  File "/home/ixxie/sparklet/repos/nixops-digitalocean/tests/functional/test_rollback_rollsback.py", line 23, in run_check
    self.depl.deploy()
  File "/nix/store/3bn7qlh80p91kgd8q3myy3f56ai2v1fs-python3-3.7.9-env/lib/python3.7/site-packages/nixops/deployment.py", line 1479, in deploy
    self.run_with_notify("deploy", lambda: self._deploy(**kwargs))
  File "/nix/store/3bn7qlh80p91kgd8q3myy3f56ai2v1fs-python3-3.7.9-env/lib/python3.7/site-packages/nixops/deployment.py", line 1468, in run_with_notify
    f()
  File "/nix/store/3bn7qlh80p91kgd8q3myy3f56ai2v1fs-python3-3.7.9-env/lib/python3.7/site-packages/nixops/deployment.py", line 1479, in <lambda>
    self.run_with_notify("deploy", lambda: self._deploy(**kwargs))
  File "/nix/store/3bn7qlh80p91kgd8q3myy3f56ai2v1fs-python3-3.7.9-env/lib/python3.7/site-packages/nixops/deployment.py", line 1266, in _deploy
    self.evaluate_active(include, exclude, kill_obsolete)
  File "/nix/store/3bn7qlh80p91kgd8q3myy3f56ai2v1fs-python3-3.7.9-env/lib/python3.7/site-packages/nixops/deployment.py", line 1207, in evaluate_active
    self.evaluate()
  File "/nix/store/3bn7qlh80p91kgd8q3myy3f56ai2v1fs-python3-3.7.9-env/lib/python3.7/site-packages/nixops/deployment.py", line 565, in evaluate
    defn = _create_definition(name, cfg, cfg["targetEnv"])
  File "/nix/store/3bn7qlh80p91kgd8q3myy3f56ai2v1fs-python3-3.7.9-env/lib/python3.7/site-packages/nixops/deployment.py", line 1800, in _create_definition
    "unknown resource type ‘{0}’".format(type_name)
nixops.deployment.UnknownBackend: unknown resource type ‘droplet’

----------------------------------------------------------------------
Ran 3 tests in 2.365s

FAILED (errors=3)

[ydlr]

Well, dang. I am now having the same issue. Since it can't find the resource type "droplet," I thought the problem could be with installing and loading the poetry environment. Sure enough, I now get an error trying to install nixops from poetry install or poetry update.

[nix-shell:~/Projects/nixops-digitalocean]$ poetry update
Updating dependencies
Resolving dependencies... (1.2s)

Writing lock file

Package operations: 3 installs, 0 updates, 0 removals

  - Installing nixops (2.0.0 c9fdc93)

[EnvCommandError]
Command ['/home/ydlr/.cache/pypoetry/virtualenvs/nixops-digitalocean-onX3IehA-py3.7/bin/pip', 'install', '--no-deps', '-U', '-e', '/home/ydlr/.cache/pypoetry/virtualenvs/nixops-digitalocean-onX3IehA-py3.7/src/nixops'] errored with the following return code 1, and output: 
Obtaining file:///home/ydlr/.cache/pypoetry/virtualenvs/nixops-digitalocean-onX3IehA-py3.7/src/nixops
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
    Preparing wheel metadata: started
    Preparing wheel metadata: finished with status 'done'
Installing collected packages: nixops
  Running setup.py develop for nixops
    ERROR: Command errored out with exit status 1:
     command: /home/ydlr/.cache/pypoetry/virtualenvs/nixops-digitalocean-onX3IehA-py3.7/bin/python3.7 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/home/ydlr/.cache/pypoetry/virtualenvs/nixops-digitalocean-onX3IehA-py3.7/src/nixops/setup.py'"'"'; __file__='"'"'/home/ydlr/.cache/pypoetry/virtualenvs/nixops-digitalocean-onX3IehA-py3.7/src/nixops/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps
         cwd: /home/ydlr/.cache/pypoetry/virtualenvs/nixops-digitalocean-onX3IehA-py3.7/src/nixops/
    Complete output (3 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
    ModuleNotFoundError: No module named 'setuptools'
    ----------------------------------------
ERROR: Command errored out with exit status 1: /home/ydlr/.cache/pypoetry/virtualenvs/nixops-digitalocean-onX3IehA-py3.7/bin/python3.7 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/home/ydlr/.cache/pypoetry/virtualenvs/nixops-digitalocean-onX3IehA-py3.7/src/nixops/setup.py'"'"'; __file__='"'"'/home/ydlr/.cache/pypoetry/virtualenvs/nixops-digitalocean-onX3IehA-py3.7/src/nixops/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps Check the logs for full command output.
WARNING: You are using pip version 19.2.3, however version 20.2.3 is available.
You should consider upgrading via the 'pip install --upgrade pip' command

I tried to pin the nixops version to an earlier commit where I had previously been able to install and successfully run the tests. Same error.

I also tried adding setuptools to both shell.nix and pyproject.toml. No luck.

[tewfik-ghariani]

@ydlr I've seen this error as well, I believe the solution is to spawn the nix-shell using the nixos-20.09 version

[ydlr]

@tewfik-ghariani thanks for the tip.

Tests are all passing for me when I run the following:

nix-shell -I nixpkgs=channel:nixos-20.09
poetry install
poetry shell
./coverage-tests.py

If I run nosetests or python ./tests.py instead of ./coverage-tests.py I get the same error that @ixxie is getting. I am not sure why. I am not very familiar with nose (or python, or tbh).

[ydlr]

I did a little digging and found that the droplet resource type was not found when running tests because nose was not loading the plugin module. This is fixed by running nosetests --all-modules. I have added that flag to ./tests.py, as well.

./coverage-tests.py was working because the --cover-inclusive flag has roughly the same effect as --all-modules.

Tests are now failing because #10

[Kiwi]

merged the other one so we can close this one right?