search

nsacyber / WALKOFF

A flexible, easy to use, automation framework allowing users to integrate their capabilities and devices to cut through the repetitive, tedious tasks slowing them down. #nsacyber
https://nsacyber.github.io/WALKOFF/
Other
1.2k stars 222 forks source link

setup_walkoff.py will not install WALKOFF natively, runs into errors #214

Closed astrohart closed 4 years ago

astrohart commented 5 years ago

OS: Ubuntu Desktop 18.04 LTS Cloned latest from master at 1322 hrs on 26 Feb 2019

Installation failed due to missing modules. Did I not catch all the necessary pre-reqs? This is with the latest setup_walkoff.py. 

walkoff@nsacyber:~/nsacyber/WALKOFF$ sudo python3 setup_walkoff.py

Checking if pip is installed:
pip 19.0.3 from /usr/local/lib/python3.6/dist-packages/pip (python 3.6)

Installing Python Dependencies...
The directory '/home/walkoff/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/walkoff/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Ignoring futures: markers 'python_version < "3"' don't match your environment
Collecting Flask>=0.10.0 (from -r requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/7f/e7/08578774ed4536d3242b14dacb4696386634607af824ea997202cd0edb4b/Flask-1.0.2-py2.py3-none-any.whl (91kB)
    100% |████████████████████████████████| 92kB 415kB/s 
Collecting Flask_SQLAlchemy>=2.1 (from -r requirements.txt (line 2))
  Downloading https://files.pythonhosted.org/packages/a1/44/294fb7f6bf49cc7224417cd0637018db9fee0729b4fe166e43e2bbb1f1c8/Flask_SQLAlchemy-2.3.2-py2.py3-none-any.whl
Collecting flask_jwt_extended>=3.4.0 (from -r requirements.txt (line 3))
  Downloading https://files.pythonhosted.org/packages/56/88/2db993849102501be954cebeef3f7842b208c74ab96a09fbd601e27e0939/Flask-JWT-Extended-3.17.0.tar.gz
Collecting sqlalchemy>=1.1.0 (from -r requirements.txt (line 4))
  Downloading https://files.pythonhosted.org/packages/21/ed/7eb53305b43ca51774a458d7c292f3bc7664d7a9bbb5bac4149fa34756b9/SQLAlchemy-1.2.18.tar.gz (5.7MB)
    100% |████████████████████████████████| 5.7MB 2.6MB/s 
Collecting sqlalchemy-utils>=0.32.0 (from -r requirements.txt (line 5))
  Downloading https://files.pythonhosted.org/packages/fe/8b/a68f81076e9a2729675253228f43986914f0510078c86f14f6cd51dd3f01/SQLAlchemy-Utils-0.33.11.tar.gz (128kB)
    100% |████████████████████████████████| 133kB 1.1MB/s 
Collecting APscheduler>=3.0.0 (from -r requirements.txt (line 6))
  Downloading https://files.pythonhosted.org/packages/97/3a/fa3213cc325091b7729616594611fff31d72c2d4d590418c3efdf7424ae2/APScheduler-3.5.3-py2.py3-none-any.whl (57kB)
    100% |████████████████████████████████| 61kB 16.0MB/s 
Collecting gevent>=1.2 (from -r requirements.txt (line 7))
  Downloading https://files.pythonhosted.org/packages/f2/ca/5b5962361ed832847b6b2f9a2d0452c8c2f29a93baef850bb8ad067c7bf9/gevent-1.4.0-cp36-cp36m-manylinux1_x86_64.whl (5.5MB)
    100% |████████████████████████████████| 5.5MB 2.3MB/s 
Collecting connexion>=2.0 (from -r requirements.txt (line 8))
  Downloading https://files.pythonhosted.org/packages/13/07/6a67c7944e6a9aa904ea06fdab615784a2c8527c3e677adf7bd8eb3fe80c/connexion-2.2.0-py2.py3-none-any.whl (1.0MB)
    100% |████████████████████████████████| 1.0MB 2.0MB/s 
Collecting pyyaml>=3.0 (from -r requirements.txt (line 9))
  Downloading https://files.pythonhosted.org/packages/9e/a3/1d13970c3f36777c583f136c136f804d70f500168edc1edea6daa7200769/PyYAML-3.13.tar.gz (270kB)
    100% |████████████████████████████████| 276kB 2.1MB/s 
Collecting pyzmq>14.1.0 (from -r requirements.txt (line 10))
  Downloading https://files.pythonhosted.org/packages/27/5a/8ce70c20efaf78fac2524b2f8d98b7360c997bc11a748dd233f3add7507e/pyzmq-18.0.0-cp36-cp36m-manylinux1_x86_64.whl (1.1MB)
    100% |████████████████████████████████| 1.1MB 490kB/s 
Collecting passlib>=1.7.0 (from -r requirements.txt (line 11))
  Downloading https://files.pythonhosted.org/packages/ee/a7/d6d238d927df355d4e4e000670342ca4705a72f0bf694027cf67d9bcf5af/passlib-1.7.1-py2.py3-none-any.whl (498kB)
    100% |████████████████████████████████| 501kB 4.9MB/s 
Collecting blinker>=1.4 (from -r requirements.txt (line 12))
  Downloading https://files.pythonhosted.org/packages/1b/51/e2a9f3b757eb802f61dc1f2b09c8c99f6eb01cf06416c0671253536517b6/blinker-1.4.tar.gz (111kB)
    100% |████████████████████████████████| 112kB 3.2MB/s 
Collecting protobuf<3.5.2,>=3.4.0 (from -r requirements.txt (line 13))
  Downloading https://files.pythonhosted.org/packages/40/99/471fa05dab1cf69419c91bcd7b5a7f3a0e251c76025bbdb40e08c367b728/protobuf-3.5.1-cp36-cp36m-manylinux1_x86_64.whl (6.4MB)
    100% |████████████████████████████████| 6.4MB 2.3MB/s 
Collecting enum34 (from -r requirements.txt (line 14))
  Downloading https://files.pythonhosted.org/packages/af/42/cb9355df32c69b553e72a2e28daee25d1611d2c0d9c272aa1d34204205b2/enum34-1.1.6-py3-none-any.whl
Collecting semver (from -r requirements.txt (line 16))
  Downloading https://files.pythonhosted.org/packages/21/18/a0de8cda637ba3efee1b3617ded00601507ce15bd70a39399740e0fd415f/semver-2.8.1-py2.py3-none-any.whl
Collecting jsonschema==2.6.0 (from -r requirements.txt (line 17))
  Downloading https://files.pythonhosted.org/packages/77/de/47e35a97b2b05c2fadbec67d44cfcdcd09b8086951b331d82de90d2912da/jsonschema-2.6.0-py2.py3-none-any.whl
Collecting six>=1.10.0 (from -r requirements.txt (line 18))
  Downloading https://files.pythonhosted.org/packages/73/fb/00a976f728d0d1fecfe898238ce23f502a721c0ac0ecfedb80e0d88c64e9/six-1.12.0-py2.py3-none-any.whl
Collecting marshmallow>=3.0.0rc2 (from -r requirements.txt (line 19))
  Downloading https://files.pythonhosted.org/packages/59/ef/887fa252964e8960237edaf12877419fd78b8e4513ae5ea86034f2ae95c4/marshmallow-3.0.0rc4-py2.py3-none-any.whl (44kB)
    100% |████████████████████████████████| 51kB 4.3MB/s 
Collecting marshmallow-sqlalchemy>=0.13.0 (from -r requirements.txt (line 20))
  Downloading https://files.pythonhosted.org/packages/7d/4d/5d6f7d01904e57c1131e06433ef013f1bd0640d47605b4dc3f93f73424fc/marshmallow_sqlalchemy-0.16.0-py2.py3-none-any.whl
Collecting pynacl (from -r requirements.txt (line 21))
  Downloading https://files.pythonhosted.org/packages/27/15/2cd0a203f318c2240b42cd9dd13c931ddd61067809fee3479f44f086103e/PyNaCl-1.3.0-cp34-abi3-manylinux1_x86_64.whl (759kB)
    100% |████████████████████████████████| 768kB 4.0MB/s 
Collecting alembic (from -r requirements.txt (line 22))
  Downloading https://files.pythonhosted.org/packages/a4/06/f1ae8393463c26f3dafa21eebac611088da02a26e1f1e23bd75fee2dbffe/alembic-1.0.7.tar.gz (1.0MB)
    100% |████████████████████████████████| 1.0MB 864kB/s 
Collecting healthcheck (from -r requirements.txt (line 23))
  Downloading https://files.pythonhosted.org/packages/b1/49/962f329b1d9ed2ded6c667f26b9a09eaf08a0545c33a17668dbe2c68a9a5/healthcheck-1.3.3.tar.gz
Collecting psycopg2-binary (from -r requirements.txt (line 24))
  Downloading https://files.pythonhosted.org/packages/2c/85/97db05dd8f6adff57cd1cb8acceeffdaf4724efec18b9af23f3cd75ad089/psycopg2_binary-2.7.7-cp36-cp36m-manylinux1_x86_64.whl (2.7MB)
    100% |████████████████████████████████| 2.7MB 6.7MB/s 
Collecting click (from -r requirements.txt (line 25))
  Downloading https://files.pythonhosted.org/packages/fa/37/45185cb5abbc30d7257104c434fe0b07e5a195a6847506c074527aa599ec/Click-7.0-py2.py3-none-any.whl (81kB)
    100% |████████████████████████████████| 81kB 693kB/s 
Collecting kubernetes (from -r requirements.txt (line 26))
  Downloading https://files.pythonhosted.org/packages/40/eb/4d6a80db84ac24c867c94fbf16e6f26db9780f5232f46ddd2e5539b42205/kubernetes-8.0.1-py2.py3-none-any.whl (1.3MB)
    100% |████████████████████████████████| 1.4MB 1.8MB/s 
Collecting pyopenssl (from -r requirements.txt (line 27))
  Downloading https://files.pythonhosted.org/packages/01/c8/ceb170d81bd3941cbeb9940fc6cc2ef2ca4288d0ca8929ea4db5905d904d/pyOpenSSL-19.0.0-py2.py3-none-any.whl (53kB)
    100% |████████████████████████████████| 61kB 1.0MB/s 
Collecting gitpython (from -r requirements.txt (line 28))
  Downloading https://files.pythonhosted.org/packages/fe/e5/fafe827507644c32d6dc553a1c435cdf882e0c28918a5bab29f7fbebfb70/GitPython-2.1.11-py2.py3-none-any.whl (448kB)
    100% |████████████████████████████████| 450kB 1.7MB/s 
Collecting dill (from -r requirements.txt (line 29))
  Downloading https://files.pythonhosted.org/packages/fe/42/bfe2e0857bc284cbe6a011d93f2a9ad58a22cb894461b199ae72cfef0f29/dill-0.2.9.tar.gz (150kB)
    100% |████████████████████████████████| 153kB 1.9MB/s 
Collecting redis>=3.0 (from -r requirements.txt (line 30))
  Downloading https://files.pythonhosted.org/packages/d0/8b/c43ef27d02382853b22c49bc41a8389e47d60811dd1d72b9a45bc905a5f8/redis-3.2.0-py2.py3-none-any.whl (65kB)
    100% |████████████████████████████████| 71kB 3.4MB/s 
Collecting confluent-kafka (from -r requirements.txt (line 31))
  Downloading https://files.pythonhosted.org/packages/dd/0a/c8bbf50a0b8e1d623521565e4c3211aebf7c9a7cf7eeb35b5360132a0ccc/confluent_kafka-0.11.6-cp36-cp36m-manylinux1_x86_64.whl (3.9MB)
    100% |████████████████████████████████| 3.9MB 5.1MB/s 
Collecting fluent-logger (from -r requirements.txt (line 32))
  Downloading https://files.pythonhosted.org/packages/7b/bd/ca035aa263f98d0d5ca3b6ffbaa8289eac5f97e1fd07f04baa7c3e088d5d/fluent_logger-0.9.3-py2.py3-none-any.whl
Collecting msgpack (from -r requirements.txt (line 33))
  Downloading https://files.pythonhosted.org/packages/92/7e/ae9e91c1bb8d846efafd1f353476e3fd7309778b582d2fb4cea4cc15b9a2/msgpack-0.6.1-cp36-cp36m-manylinux1_x86_64.whl (248kB)
    100% |████████████████████████████████| 256kB 1.3MB/s 
Collecting prometheus_client (from -r requirements.txt (line 34))
  Downloading https://files.pythonhosted.org/packages/4c/bd/b42db3ec90ffc6be805aad09c1cea4bb13a620d0cd4b21aaa44d13541d71/prometheus_client-0.6.0.tar.gz
Collecting werkzeug (from -r requirements.txt (line 35))
  Downloading https://files.pythonhosted.org/packages/20/c4/12e3e56473e52375aa29c4764e70d1b8f3efa6682bef8d0aae04fe335243/Werkzeug-0.14.1-py2.py3-none-any.whl (322kB)
    100% |████████████████████████████████| 327kB 1.5MB/s 
Collecting prometheus_flask_exporter (from -r requirements.txt (line 36))
  Downloading https://files.pythonhosted.org/packages/56/fc/e1d0a6a0021291913dcc11a05ab2725524ed10f0c7e1e775e5cf4b26a50f/prometheus_flask_exporter-0.5.1.tar.gz
Collecting requests (from -r requirements.txt (line 37))
  Downloading https://files.pythonhosted.org/packages/7d/e3/20f3d364d6c8e5d2353c72a67778eb189176f08e873c9900e10c0287b84b/requests-2.21.0-py2.py3-none-any.whl (57kB)
    100% |████████████████████████████████| 61kB 3.1MB/s 
Collecting swagger_spec_validator (from -r requirements.txt (line 38))
  Downloading https://files.pythonhosted.org/packages/3a/98/1845e3f23fc910f05461d2f981eb0f09ab1f6e75b4f20f9fedb2da3be4c5/swagger_spec_validator-2.4.3-py2.py3-none-any.whl
Collecting flask-swagger-ui (from -r requirements.txt (line 39))
  Downloading https://files.pythonhosted.org/packages/d9/ca/2bae07c7dd2cebc3f6cb3ddea7253fad7a86b2a47b6460eed380e2dbd8f9/flask-swagger-ui-3.18.0.tar.gz (542kB)
    100% |████████████████████████████████| 552kB 103kB/s 
Collecting itsdangerous>=0.24 (from Flask>=0.10.0->-r requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/76/ae/44b03b253d6fade317f32c24d100b3b35c2239807046a4c953c7b89fa49e/itsdangerous-1.1.0-py2.py3-none-any.whl
Collecting Jinja2>=2.10 (from Flask>=0.10.0->-r requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/7f/ff/ae64bacdfc95f27a016a7bed8e8686763ba4d277a78ca76f32659220a731/Jinja2-2.10-py2.py3-none-any.whl (126kB)
    100% |████████████████████████████████| 133kB 2.6MB/s 
Collecting PyJWT (from flask_jwt_extended>=3.4.0->-r requirements.txt (line 3))
  Downloading https://files.pythonhosted.org/packages/87/8b/6a9f14b5f781697e51259d81657e6048fd31a113229cf346880bb7545565/PyJWT-1.7.1-py2.py3-none-any.whl
Collecting tzlocal>=1.2 (from APscheduler>=3.0.0->-r requirements.txt (line 6))
  Downloading https://files.pythonhosted.org/packages/cb/89/e3687d3ed99bc882793f82634e9824e62499fdfdc4b1ae39e211c5b05017/tzlocal-1.5.1.tar.gz
Requirement already satisfied, skipping upgrade: pytz in /usr/lib/python3/dist-packages (from APscheduler>=3.0.0->-r requirements.txt (line 6)) (2018.3)
Requirement already satisfied, skipping upgrade: setuptools>=0.7 in /usr/lib/python3/dist-packages (from APscheduler>=3.0.0->-r requirements.txt (line 6)) (39.0.1)
Collecting greenlet>=0.4.14; platform_python_implementation == "CPython" (from gevent>=1.2->-r requirements.txt (line 7))
  Downloading https://files.pythonhosted.org/packages/bf/45/142141aa47e01a5779f0fa5a53b81f8379ce8f2b1cd13df7d2f1d751ae42/greenlet-0.4.15-cp36-cp36m-manylinux1_x86_64.whl (41kB)
    100% |████████████████████████████████| 51kB 5.0MB/s 
Collecting openapi-spec-validator>=0.2.4 (from connexion>=2.0->-r requirements.txt (line 8))
  Downloading https://files.pythonhosted.org/packages/3b/25/2578ef159bf090b28ca311cbbb14010a3d44dfe1bcf175e233bd2b2c995b/openapi_spec_validator-0.2.5-py3-none-any.whl
Collecting inflection>=0.3.1 (from connexion>=2.0->-r requirements.txt (line 8))
  Downloading https://files.pythonhosted.org/packages/d5/35/a6eb45b4e2356fe688b21570864d4aa0d0a880ce387defe9c589112077f8/inflection-0.3.1.tar.gz
Collecting clickclick>=1.2 (from connexion>=2.0->-r requirements.txt (line 8))
  Downloading https://files.pythonhosted.org/packages/b6/51/2b04f7a56dcbacc0e3a7cf726e1d88d28866bf488a7a0668582306e1e643/clickclick-1.2.2-py2.py3-none-any.whl
Collecting cffi>=1.4.1 (from pynacl->-r requirements.txt (line 21))
  Downloading https://files.pythonhosted.org/packages/20/f7/87b62a8895bf7c93e907b05b97bc4459c81a38a61151f03a6eae13d863aa/cffi-1.12.2-cp36-cp36m-manylinux1_x86_64.whl (428kB)
    100% |████████████████████████████████| 430kB 5.3MB/s 
Requirement already satisfied, skipping upgrade: Mako in /usr/lib/python3/dist-packages (from alembic->-r requirements.txt (line 22)) (1.0.7)
Collecting python-dateutil (from alembic->-r requirements.txt (line 22))
  Downloading https://files.pythonhosted.org/packages/41/17/c62faccbfbd163c7f57f3844689e3a78bae1f403648a6afb1d0866d87fbb/python_dateutil-2.8.0-py2.py3-none-any.whl (226kB)
    100% |████████████████████████████████| 235kB 1.7MB/s 
Collecting python-editor>=0.3 (from alembic->-r requirements.txt (line 22))
  Downloading https://files.pythonhosted.org/packages/c6/d3/201fc3abe391bbae6606e6f1d598c15d367033332bd54352b12f35513717/python_editor-1.0.4-py3-none-any.whl
Collecting adal>=1.0.2 (from kubernetes->-r requirements.txt (line 26))
  Downloading https://files.pythonhosted.org/packages/00/72/53dce9e4f5d6c1aa57b8d408cb34dff1969ecbf10ab7e678f32c5e0e2397/adal-1.2.1-py2.py3-none-any.whl (52kB)
    100% |████████████████████████████████| 61kB 3.0MB/s 
Requirement already satisfied, skipping upgrade: certifi>=14.05.14 in /usr/lib/python3/dist-packages (from kubernetes->-r requirements.txt (line 26)) (2018.1.18)
Collecting urllib3>=1.23 (from kubernetes->-r requirements.txt (line 26))
  Downloading https://files.pythonhosted.org/packages/62/00/ee1d7de624db8ba7090d1226aebefab96a2c71cd5cfa7629d6ad3f61b79e/urllib3-1.24.1-py2.py3-none-any.whl (118kB)
    100% |████████████████████████████████| 122kB 2.1MB/s 
Collecting websocket-client!=0.40.0,!=0.41.*,!=0.42.*,>=0.32.0 (from kubernetes->-r requirements.txt (line 26))
  Downloading https://files.pythonhosted.org/packages/38/54/684db2ba1b7a203602808446b8686ee786f93b4a7e080cdc440cc7e06e56/websocket_client-0.55.0-py2.py3-none-any.whl (200kB)
    100% |████████████████████████████████| 204kB 956kB/s 
Collecting google-auth>=1.0.1 (from kubernetes->-r requirements.txt (line 26))
  Downloading https://files.pythonhosted.org/packages/c5/9b/ed0516cc1f7609fb0217e3057ff4f0f9f3e3ce79a369c6af4a6c5ca25664/google_auth-1.6.3-py2.py3-none-any.whl (73kB)
    100% |████████████████████████████████| 81kB 3.1MB/s 
Collecting requests-oauthlib (from kubernetes->-r requirements.txt (line 26))
  Downloading https://files.pythonhosted.org/packages/c2/e2/9fd03d55ffb70fe51f587f20bcf407a6927eb121de86928b34d162f0b1ac/requests_oauthlib-1.2.0-py2.py3-none-any.whl
Collecting cryptography>=2.3 (from pyopenssl->-r requirements.txt (line 27))
  Downloading https://files.pythonhosted.org/packages/98/71/e632e222f34632e0527dd41799f7847305e701f38f512d81bdf96009bca4/cryptography-2.5-cp34-abi3-manylinux1_x86_64.whl (2.4MB)
    100% |████████████████████████████████| 2.4MB 5.0MB/s 
Collecting gitdb2>=2.0.0 (from gitpython->-r requirements.txt (line 28))
  Downloading https://files.pythonhosted.org/packages/da/30/a407568aa8d8f25db817cf50121a958722f3fc5f87e3a6fba1f40c0633e3/gitdb2-2.0.5-py2.py3-none-any.whl (62kB)
    100% |████████████████████████████████| 71kB 1.8MB/s 
Requirement already satisfied, skipping upgrade: idna<2.9,>=2.5 in /usr/lib/python3/dist-packages (from requests->-r requirements.txt (line 37)) (2.6)
Requirement already satisfied, skipping upgrade: chardet<3.1.0,>=3.0.2 in /usr/lib/python3/dist-packages (from requests->-r requirements.txt (line 37)) (3.0.4)
Requirement already satisfied, skipping upgrade: MarkupSafe>=0.23 in /usr/lib/python3/dist-packages (from Jinja2>=2.10->Flask>=0.10.0->-r requirements.txt (line 1)) (1.0)
Collecting pycparser (from cffi>=1.4.1->pynacl->-r requirements.txt (line 21))
  Downloading https://files.pythonhosted.org/packages/68/9e/49196946aee219aead1290e00d1e7fdeab8567783e83e1b9ab5585e6206a/pycparser-2.19.tar.gz (158kB)
    100% |████████████████████████████████| 163kB 1.5MB/s 
Collecting cachetools>=2.0.0 (from google-auth>=1.0.1->kubernetes->-r requirements.txt (line 26))
  Downloading https://files.pythonhosted.org/packages/39/2b/d87fc2369242bd743883232c463f28205902b8579cb68dcf5b11eee1652f/cachetools-3.1.0-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from google-auth>=1.0.1->kubernetes->-r requirements.txt (line 26))
  Downloading https://files.pythonhosted.org/packages/02/e5/38518af393f7c214357079ce67a317307936896e961e35450b70fad2a9cf/rsa-4.0-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.2.1 (from google-auth>=1.0.1->kubernetes->-r requirements.txt (line 26))
  Downloading https://files.pythonhosted.org/packages/da/98/8ddd9fa4d84065926832bcf2255a2b69f1d03330aa4d1c49cc7317ac888e/pyasn1_modules-0.2.4-py2.py3-none-any.whl (66kB)
    100% |████████████████████████████████| 71kB 1.8MB/s 
Collecting oauthlib>=3.0.0 (from requests-oauthlib->kubernetes->-r requirements.txt (line 26))
  Downloading https://files.pythonhosted.org/packages/16/95/699466b05b72b94a41f662dc9edf87fda4289e3602ecd42d27fcaddf7b56/oauthlib-3.0.1-py2.py3-none-any.whl (142kB)
    100% |████████████████████████████████| 143kB 2.3MB/s 
Requirement already satisfied, skipping upgrade: asn1crypto>=0.21.0 in /usr/lib/python3/dist-packages (from cryptography>=2.3->pyopenssl->-r requirements.txt (line 27)) (0.24.0)
Collecting smmap2>=2.0.0 (from gitdb2>=2.0.0->gitpython->-r requirements.txt (line 28))
  Downloading https://files.pythonhosted.org/packages/55/d2/866d45e3a121ee15a1dc013824d58072fd5c7799c9c34d01378eb262ca8f/smmap2-2.0.5-py2.py3-none-any.whl
Collecting pyasn1>=0.1.3 (from rsa>=3.1.4->google-auth>=1.0.1->kubernetes->-r requirements.txt (line 26))
  Downloading https://files.pythonhosted.org/packages/7b/7c/c9386b82a25115cccf1903441bba3cbadcfae7b678a20167347fa8ded34c/pyasn1-0.4.5-py2.py3-none-any.whl (73kB)
    100% |████████████████████████████████| 81kB 3.2MB/s 
openapi-spec-validator 0.2.5 has requirement PyYAML>=4.2b1, but you'll have pyyaml 3.13 which is incompatible.
Installing collected packages: click, itsdangerous, Jinja2, werkzeug, Flask, sqlalchemy, Flask-SQLAlchemy, PyJWT, flask-jwt-extended, six, sqlalchemy-utils, tzlocal, APscheduler, greenlet, gevent, pyyaml, jsonschema, openapi-spec-validator, inflection, clickclick, urllib3, requests, connexion, pyzmq, passlib, blinker, protobuf, enum34, semver, marshmallow, marshmallow-sqlalchemy, pycparser, cffi, pynacl, python-dateutil, python-editor, alembic, healthcheck, psycopg2-binary, cryptography, adal, websocket-client, cachetools, pyasn1, rsa, pyasn1-modules, google-auth, oauthlib, requests-oauthlib, kubernetes, pyopenssl, smmap2, gitdb2, gitpython, dill, redis, confluent-kafka, msgpack, fluent-logger, prometheus-client, prometheus-flask-exporter, swagger-spec-validator, flask-swagger-ui
  Running setup.py install for sqlalchemy ... done
  Running setup.py install for flask-jwt-extended ... done
  Found existing installation: six 1.11.0
    Uninstalling six-1.11.0:
      Successfully uninstalled six-1.11.0
  Running setup.py install for sqlalchemy-utils ... done
  Running setup.py install for tzlocal ... done
  Found existing installation: PyYAML 3.12
Cannot uninstall 'PyYAML'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.
Traceback (most recent call last):
  File "scripts/install_dependencies.py", line 27, in <module>
    from walkoff.config import Config
  File "/home/walkoff/nsacyber/WALKOFF/walkoff/config.py", line 10, in <module>
    import msgpack
ModuleNotFoundError: No module named 'msgpack'

Generating Certificates...
Traceback (most recent call last):
  File "scripts/generate_certificates.py", line 5, in <module>
    import zmq.auth
ModuleNotFoundError: No module named 'zmq'
Traceback (most recent call last):
  File "setup_walkoff.py", line 63, in <module>
    check_redis(args.unattended)
  File "setup_walkoff.py", line 35, in check_redis
    import redis
ModuleNotFoundError: No module named 'redis'
astrohart commented 5 years ago

Am not sure what it is saying about `ModuleNotFoundError: No module named 'redis'.

I have redis installed, FYI:

walkoff@nsacyber:~/nsacyber/WALKOFF$ redis-cli
127.0.0.1:6379> ping
PONG
127.0.0.1:6379> exit
astrohart commented 5 years ago

I did

$ sudo python3 -m pip install msgpack
$ sudo python3 -m pip install zmq
$ sudo python3 -m pip install redis
$ sudo python3 setup_walkoff.py

And then it installed. It has become my understanding that, on Ubuntu Desktop 18.04 LTS, there is such a heavy-handed preference in the OS for the python command to call the python2 executable, and pips for both versions of Python are on the system, that I have to be explicit about which Python's pip I mean. Still, should the msgpack and zmq modules be added to the requirements.txt file?

adpham95 commented 5 years ago

This is because pip was unable to update PyYAML on your system:

Cannot uninstall 'PyYAML'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.

Changes to be made to setup_walkoff.py to address the issues raised:

My only disagreement is using sudo with pip. A Python environment manager (e.g. venv, pyenv, conda, etc.) should be used instead as recommended in the readme. The --user flag for pip can also be used to direct pip to install packages to the user's home directory instead of a system directory, which doesn't require sudo (I can add this to the readme).

Using sudo with pip causes root to own the installed packages, and can also present a security issue.

astrohart commented 5 years ago

Shouldn't the required packages just be added to requirements.txt?

adpham95 commented 5 years ago

They are in requirements.txt. However if the pip installation fails, the setup continues anyways, and fails when it checks if redis exists. That's what I was referring to with "Better error handling to stop the setup script from proceeding if there are errors in pip install."

astrohart commented 5 years ago

i guess I was confused because I was not following the readme, rather, following the Overview video's instructions on installing natively, found at: https://nsacyber.github.io/WALKOFF/vid/overview.mp4

astrohart commented 5 years ago

Is there a script somewhere to properly configure a fresh, e.g., Ubuntu VM properly with the pre-reqs, so that setup_walkoff.py will work all the way through?