pystorm / streamparse

Run Python in Apache Storm topologies. Pythonic API, CLI tooling, and a topology DSL.
http://streamparse.readthedocs.io/
Apache License 2.0
1.5k stars 218 forks source link

Updating to 3.14.0: submit throws errors #446

Open mbande opened 6 years ago

mbande commented 6 years ago

Using previous version, the following config works fine:

{
    "serializer": "json",
    "topology_specs": "topologies/",
    "envs": {
        "prod": {
            "user": "",
            "nimbus": "localhost",
            "workers": ["localhost"],
            "use_virtualenv": true,
            "install_virtualenv": false,
            "use_ssh_for_nimbus": false,
            "virtualenv_root": "/opt/",
            "virtualenv_name": "venv"
        }
    }
}

as you can see, my app uses an existing virtual env and does not need virtualenv_specs, also my app use local nimbus updating to 3.14.0, submit throws:

Traceback (most recent call last):
  File "/opt/venv/bin/sparse", line 11, in <module>
    sys.exit(main())
  File "/opt/venvlib/python3.4/site-packages/streamparse/cli/sparse.py", line 79, in main
    args.func(args)
  File "/opt/venv/lib/python3.4/site-packages/streamparse/cli/submit.py", line 299, in main
    user=args.user)
  File "/opt/venv/lib/python3.4/site-packages/streamparse/cli/submit.py", line 185, in submit_topology
    user=user)
  File "/opt/venv/lib/python3.4/site-packages/streamparse/cli/update_virtualenv.py", line 91, in create_or_update_virtualenvs
    config["virtualenv_specs"] = config["virtualenv_specs"].rstrip("/")
KeyError: 'virtualenv_specs'

i'v added a dummy virtualenv_specs to config, and a new error appears:

[localhost] Executing task '_create_or_update_virtualenv'
Warning: Unable to load SSH config file '/home/user/.ssh/config'
Fatal error: Needed to prompt for a connection or sudo password (host: localhost), but input would be ambiguous in parallel mode
Aborting.
Fatal error: One or more hosts failed while executing task '_create_or_update_virtualenv'
Aborting.
amoyiki commented 6 years ago

you can see this How to ssh to localhost without password?

mbande commented 6 years ago

@amoyiki "use_ssh_for_nimbus": false is supposed to disable ssh at all!