Closed mhkeller closed 9 years ago
easier to read the screenshot
Did you see any messages about pulling / installing repos? I'm guessing you still have the old version of newslynx/dot_newslynx in your path. Try pip uninstall newslynx and then reinstall.
If that doesnt work, you should be able to add a list of git urls for sous chef modules to your config.yaml under the key "default_sous_chefs"
Sorry for the formatting, on a phone.
I installed from a fresh virtual environment. i did have the sous chefs in ~/.newslynx
i'll remove that folder and try again from a fresh venv again.
You can also manually install sous chefs by running newslynx sc-install
This also happens on automation into an aws box.
TASK: [initialize newslynx with application defaults.] ************************
<54.204.39.239> ESTABLISH CONNECTION FOR USER: ubuntu
<54.204.39.239> REMOTE_MODULE command chdir=/opt/newslynx/newslynx-core/ /usr/local/bin/newslynx init
<54.204.39.239> EXEC ssh -C -tt -vvv -o UserKnownHostsFile=/dev/null -o IdentitiesOnly=yes -o ControlMaster=auto -o ControlPersist=60s -o ControlPath="/Users/mike/.ansible/cp/ansible-ssh-%h-%p-%r" -o StrictHostKeyChecking=no -o Port=22 -o IdentityFile="/users/mike/.ssh/mhkeller-ec2.pem" -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=ubuntu -o ConnectTimeout=30 54.204.39.239 /bin/sh -c 'mkdir -p $HOME/.ansible/tmp/ansible-tmp-1440384417.75-201224494040102 && chmod a+rx $HOME/.ansible/tmp/ansible-tmp-1440384417.75-201224494040102 && echo $HOME/.ansible/tmp/ansible-tmp-1440384417.75-201224494040102'
<54.204.39.239> PUT /var/folders/gp/__h0yj4s4lsc84w79k4b8mnh0000gn/T/tmpmxjbF8 TO /home/ubuntu/.ansible/tmp/ansible-tmp-1440384417.75-201224494040102/command
<54.204.39.239> EXEC ssh -C -tt -vvv -o UserKnownHostsFile=/dev/null -o IdentitiesOnly=yes -o ControlMaster=auto -o ControlPersist=60s -o ControlPath="/Users/mike/.ansible/cp/ansible-ssh-%h-%p-%r" -o StrictHostKeyChecking=no -o Port=22 -o IdentityFile="/users/mike/.ssh/mhkeller-ec2.pem" -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=ubuntu -o ConnectTimeout=30 54.204.39.239 /bin/sh -c 'sudo -k && sudo -H -S -p "[sudo via ansible, key=rfrjaltsqkorcfetcntkmigwryyqptui] password: " -u root /bin/sh -c '"'"'echo BECOME-SUCCESS-rfrjaltsqkorcfetcntkmigwryyqptui; LANG=en_US.UTF-8 LC_CTYPE=en_US.UTF-8 /usr/bin/python /home/ubuntu/.ansible/tmp/ansible-tmp-1440384417.75-201224494040102/command; rm -rf /home/ubuntu/.ansible/tmp/ansible-tmp-1440384417.75-201224494040102/ >/dev/null 2>&1'"'"''
failed: [newslynx] => {"changed": true, "cmd": ["/usr/local/bin/newslynx", "init"], "delta": "0:00:01.827630", "end": "2015-08-24 02:46:08.861944", "rc": 1, "start": "2015-08-24 02:46:07.034314", "warnings": []}
stderr: Traceback (most recent call last):
File "/usr/local/bin/newslynx", line 9, in <module>
load_entry_point('newslynx==1.1.1', 'console_scripts', 'newslynx')()
File "/usr/local/lib/python2.7/dist-packages/newslynx/cli/__init__.py", line 79, in run
subcommands = setup(parser)
File "/usr/local/lib/python2.7/dist-packages/newslynx/cli/__init__.py", line 29, in setup
from newslynx.cli import (
File "/usr/local/lib/python2.7/dist-packages/newslynx/cli/init.py", line 15, in <module>
from newslynx.sc import sc_module
File "/usr/local/lib/python2.7/dist-packages/newslynx/sc/sc_module.py", line 8, in <module>
import pip
File "/usr/lib/python2.7/dist-packages/pip/__init__.py", line 61, in <module>
from pip.vcs import git, mercurial, subversion, bazaar # noqa
File "/usr/lib/python2.7/dist-packages/pip/vcs/mercurial.py", line 9, in <module>
from pip.download import path_to_url
File "/usr/lib/python2.7/dist-packages/pip/download.py", line 25, in <module>
from requests.compat import IncompleteRead
ImportError: cannot import name IncompleteRead
I meant newslynx sc-install {git-url}
hmm that looks like some silent dependency for pip, will investiagate shortly.
I have only config.yaml
in ~/.newslynx
but now I'm getting
(abc)➜ newslynx-core git:(master) newslynx init
/ \ / \ / \ / \ / \ / \ / \ / \
( n | e | w | s | l | y | n | x )
\_/ \_/ \_/ \_/ \_/ \_/ \_/ \_/
23:09:26 | newslynx.cli-114 | No config file found.
23:09:26 | newslynx.cli-131 | Now, modify your configurations in /Users/mike/.newslynx/config.yaml.
23:09:26 | newslynx.cli-132 | Once you're done, you can run $ newslynx init
config file looks like this
# _ _ _ _ _ _ _ _
# / \ / \ / \ / \ / \ / \ / \ / \
# ( n | e | w | s | l | y | n | x )
# \_/ \_/ \_/ \_/ \_/ \_/ \_/ \_/
# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# Secret Stuff
secret_key: changeme
app_secret_key: changemetoo
super_user: admin
super_user_email: changeme@example.com
super_user_password: admin
super_user_apikey: mj
super_user_org: 'admin'
super_user_timezone: UTC
# The Database URI (This is mandatory!)
sqlalchemy_database_uri: 'postgresql://localhost:5432/newslynx'
# API information for the app to know
api_url: http://localhost:5000
api_version: v1
https: false # Change to `true` for production.
# Directory which contains Sous Chefs
sous_chefs_dir: '~/.newslynx/sous-chefs/'
# Filepaths to default recipes + tags + reports to add for each
# new organization.
default_recipes: '~/.newslynx/defaults/recipes.yaml'
default_tags: '~/.newslynx/defaults/tags.yaml'
# List of sous chefs to install on initialization.
default_sous_chefs:
- 'https://github.com/newslynx/newslynx-sc-twitter.git'
- 'https://github.com/newslynx/newslynx-sc-shares.git'
- 'https://github.com/newslynx/newslynx-sc-rss.git'
- 'https://github.com/newslynx/newslynx-sc-reddit.git'
- 'https://github.com/newslynx/newslynx-sc-google-analytics.git'
- 'https://github.com/newslynx/newslynx-sc-facebook.git'
What happens when you run this in the root directory of newslynx-core
?
dropdb newslynx && createdb newslynx && pip uninstall newslynx && pip install --editable . && newslynx init
now i get the first error
23:24:34 | newslynx.tasks.default-95 | Creating sous chef: "internal-refresh-content-summary-metrics"
23:24:34 | newslynx.cli.init-117 | Traceback (most recent call last):
File "/Users/mike/wrk/projects/newslynx/newslynx-core/newslynx/cli/init.py", line 112, in run
default.org()
File "/Users/mike/wrk/projects/newslynx/newslynx-core/newslynx/tasks/default.py", line 57, in org
sous_chefs(org)
File "/Users/mike/wrk/projects/newslynx/newslynx-core/newslynx/tasks/default.py", line 89, in sous_chefs
for sc, fp in init.load_sous_chefs():
File "/Users/mike/wrk/projects/newslynx/newslynx-core/newslynx/init.py", line 91, in load_sous_chefs
for sc, fp in findsc(sous_chef_dir):
File "/Users/mike/wrk/projects/newslynx/newslynx-core/newslynx/init.py", line 87, in findsc
sc = sous_chef_schema.load(fp)
File "/Users/mike/wrk/projects/newslynx/newslynx-core/newslynx/models/sous_chef_schema.py", line 50, in load
raise SousChefSchemaError(msg)
SousChefSchemaError: Traceback (most recent call last):
File "/Users/mike/wrk/projects/newslynx/newslynx-core/newslynx/models/sous_chef_schema.py", line 47, in load
return validate(sc, fp)
File "/Users/mike/wrk/projects/newslynx/newslynx-core/newslynx/models/sous_chef_schema.py", line 99, in validate
raise SousChefSchemaError(msg)
SousChefSchemaError: Traceback (most recent call last):
File "/Users/mike/wrk/projects/newslynx/newslynx-core/newslynx/models/sous_chef_schema.py", line 74, in validate
_validate_python_sous_chef(sc)
File "/Users/mike/wrk/projects/newslynx/newslynx-core/newslynx/models/sous_chef_schema.py", line 248, in _validate_python_sous_chef
_raise_sous_chef_schema_error(sc, msg)
File "/Users/mike/wrk/projects/newslynx/newslynx-core/newslynx/models/sous_chef_schema.py", line 279, in _raise_sous_chef_schema_error
raise SousChefSchemaError(msg)
SousChefSchemaError: There were problems validating SousChef 'facebook-page-to-event': newslynx_sc_facebook.events is not importable. Here's the error: No module named newslynx_sc_facebook.events
failed on file /Users/mike/.newslynx/sous-chefs/newslynx-sc-facebook/newslynx_sc_facebook/facebook_page_to_event.yaml
failed on file /Users/mike/.newslynx/sous-chefs/newslynx-sc-facebook/newslynx_sc_facebook/facebook_page_to_event.yaml
full output
leaning up...
/ \ / \ / \ / \ / \ / \ / \ / \
( n | e | w | s | l | y | n | x )
\_/ \_/ \_/ \_/ \_/ \_/ \_/ \_/
23:23:42 | newslynx.cli.init-59 | (Re)Creating database "postgresql://localhost:5432/newslynx"
23:23:43 | newslynx.cli.init-63 | (Re)Loading SQL Extensions
23:23:43 | newslynx.cli.init-71 | (Re)Initializing App Defaults
23:23:43 | newslynx.sc.sc_module-134 | Creating directory: /Users/mike/.newslynx/sous-chefs/
23:23:43 | newslynx.sc.sc_module-160 | Cloning "https://github.com/newslynx/newslynx-sc-twitter.git" ==> "/Users/mike/.newslynx/sous-chefs/newslynx-sc-twitter"
23:23:44 | newslynx.sc.sc_module-167 | Installing: newslynx-sc-twitter
23:23:46 | pip._vendor.requests.packages.urllib3.connectionpool-657 | Starting new HTTPS connection (1): pypi.python.org
23:23:58 | pip._vendor.distlib.util-404 | changing mode of /Users/mike/.virtualenvs/abc/bin/nosetests to 755
23:23:58 | pip._vendor.distlib.util-404 | changing mode of /Users/mike/.virtualenvs/abc/bin/nosetests-2.7 to 755
23:23:59 | pip._vendor.distlib.util-404 | changing mode of /Users/mike/.virtualenvs/abc/bin/sphinx-apidoc to 755
23:23:59 | pip._vendor.distlib.util-404 | changing mode of /Users/mike/.virtualenvs/abc/bin/sphinx-build to 755
23:23:59 | pip._vendor.distlib.util-404 | changing mode of /Users/mike/.virtualenvs/abc/bin/sphinx-quickstart to 755
23:23:59 | pip._vendor.distlib.util-404 | changing mode of /Users/mike/.virtualenvs/abc/bin/sphinx-autogen to 755
23:24:05 | pip._vendor.distlib.util-404 | changing mode of /Users/mike/.virtualenvs/abc/bin/pygmentize to 755
23:24:09 | newslynx.sc.sc_module-172 | Successfully installed: newslynx-sc-twitter
23:24:09 | newslynx.sc.sc_module-160 | Cloning "https://github.com/newslynx/newslynx-sc-shares.git" ==> "/Users/mike/.newslynx/sous-chefs/newslynx-sc-shares"
23:24:10 | newslynx.sc.sc_module-167 | Installing: newslynx-sc-shares
23:24:13 | newslynx.sc.sc_module-172 | Successfully installed: newslynx-sc-shares
23:24:13 | newslynx.sc.sc_module-160 | Cloning "https://github.com/newslynx/newslynx-sc-rss.git" ==> "/Users/mike/.newslynx/sous-chefs/newslynx-sc-rss"
23:24:14 | newslynx.sc.sc_module-167 | Installing: newslynx-sc-rss
23:24:18 | newslynx.sc.sc_module-172 | Successfully installed: newslynx-sc-rss
23:24:18 | newslynx.sc.sc_module-160 | Cloning "https://github.com/newslynx/newslynx-sc-reddit.git" ==> "/Users/mike/.newslynx/sous-chefs/newslynx-sc-reddit"
23:24:18 | newslynx.sc.sc_module-167 | Installing: newslynx-sc-reddit
23:24:21 | pip._vendor.requests.packages.urllib3.connectionpool-657 | Starting new HTTPS connection (1): pypi.python.org
23:24:22 | pip._vendor.distlib.util-404 | changing mode of /Users/mike/.virtualenvs/abc/bin/praw-multiprocess to 755
23:24:24 | newslynx.sc.sc_module-172 | Successfully installed: newslynx-sc-reddit
23:24:24 | newslynx.sc.sc_module-160 | Cloning "https://github.com/newslynx/newslynx-sc-google-analytics.git" ==> "/Users/mike/.newslynx/sous-chefs/newslynx-sc-google-analytics"
23:24:24 | newslynx.sc.sc_module-167 | Installing: newslynx-sc-google-analytics
23:24:28 | newslynx.sc.sc_module-172 | Successfully installed: newslynx-sc-google-analytics
23:24:28 | newslynx.sc.sc_module-160 | Cloning "https://github.com/newslynx/newslynx-sc-facebook.git" ==> "/Users/mike/.newslynx/sous-chefs/newslynx-sc-facebook"
23:24:28 | newslynx.sc.sc_module-167 | Installing: newslynx-sc-facebook
23:24:30 | pip._vendor.requests.packages.urllib3.connectionpool-657 | Starting new HTTPS connection (1): pypi.python.org
23:24:34 | newslynx.sc.sc_module-172 | Successfully installed: newslynx-sc-facebook
23:24:34 | newslynx.cli.init-91 | Storing default tags in: /Users/mike/.newslynx/config.yaml
23:24:34 | newslynx.cli.init-91 | Storing default recipes in: /Users/mike/.newslynx/config.yaml
23:24:34 | newslynx.cli.init-106 | Storing new configurations to: /Users/mike/.newslynx/config.yaml
23:24:34 | newslynx.cli.init-111 | (Re)Initializing Super User Org admin
23:24:34 | newslynx.tasks.default-29 | Creating super user org: "admin"
23:24:34 | newslynx.tasks.default-40 | Creating super user: "changeme@example.com"
23:24:34 | newslynx.tasks.default-73 | Creating tag: "politics"
23:24:34 | newslynx.tasks.default-73 | Creating tag: "environment"
23:24:34 | newslynx.tasks.default-73 | Creating tag: "health"
23:24:34 | newslynx.tasks.default-73 | Creating tag: "education"
23:24:34 | newslynx.tasks.default-73 | Creating tag: "economy"
23:24:34 | newslynx.tasks.default-73 | Creating tag: "feature"
23:24:34 | newslynx.tasks.default-73 | Creating tag: "reprint-pickup"
23:24:34 | newslynx.tasks.default-73 | Creating tag: "localization"
23:24:34 | newslynx.tasks.default-73 | Creating tag: "editorial"
23:24:34 | newslynx.tasks.default-73 | Creating tag: "staff-appearance"
23:24:34 | newslynx.tasks.default-73 | Creating tag: "influencer-mention"
23:24:34 | newslynx.tasks.default-73 | Creating tag: "celebrity-commentary"
23:24:34 | newslynx.tasks.default-73 | Creating tag: "humorist-pop-culture"
23:24:34 | newslynx.tasks.default-73 | Creating tag: "internal-discussion"
23:24:34 | newslynx.tasks.default-73 | Creating tag: "govt-investigation"
23:24:34 | newslynx.tasks.default-73 | Creating tag: "policy-reg-change"
23:24:34 | newslynx.tasks.default-73 | Creating tag: "promotion-internal"
23:24:34 | newslynx.tasks.default-95 | Creating sous chef: "internal-delete-old-events"
23:24:34 | newslynx.tasks.default-95 | Creating sous chef: "internal-refresh-content-comparisons"
23:24:34 | newslynx.tasks.default-95 | Creating sous chef: "internal-refresh-content-summary-metrics"
23:24:34 | newslynx.cli.init-117 | Traceback (most recent call last):
File "/Users/mike/wrk/projects/newslynx/newslynx-core/newslynx/cli/init.py", line 112, in run
default.org()
File "/Users/mike/wrk/projects/newslynx/newslynx-core/newslynx/tasks/default.py", line 57, in org
sous_chefs(org)
File "/Users/mike/wrk/projects/newslynx/newslynx-core/newslynx/tasks/default.py", line 89, in sous_chefs
for sc, fp in init.load_sous_chefs():
File "/Users/mike/wrk/projects/newslynx/newslynx-core/newslynx/init.py", line 91, in load_sous_chefs
for sc, fp in findsc(sous_chef_dir):
File "/Users/mike/wrk/projects/newslynx/newslynx-core/newslynx/init.py", line 87, in findsc
sc = sous_chef_schema.load(fp)
File "/Users/mike/wrk/projects/newslynx/newslynx-core/newslynx/models/sous_chef_schema.py", line 50, in load
raise SousChefSchemaError(msg)
SousChefSchemaError: Traceback (most recent call last):
File "/Users/mike/wrk/projects/newslynx/newslynx-core/newslynx/models/sous_chef_schema.py", line 47, in load
return validate(sc, fp)
File "/Users/mike/wrk/projects/newslynx/newslynx-core/newslynx/models/sous_chef_schema.py", line 99, in validate
raise SousChefSchemaError(msg)
SousChefSchemaError: Traceback (most recent call last):
File "/Users/mike/wrk/projects/newslynx/newslynx-core/newslynx/models/sous_chef_schema.py", line 74, in validate
_validate_python_sous_chef(sc)
File "/Users/mike/wrk/projects/newslynx/newslynx-core/newslynx/models/sous_chef_schema.py", line 248, in _validate_python_sous_chef
_raise_sous_chef_schema_error(sc, msg)
File "/Users/mike/wrk/projects/newslynx/newslynx-core/newslynx/models/sous_chef_schema.py", line 279, in _raise_sous_chef_schema_error
raise SousChefSchemaError(msg)
SousChefSchemaError: There were problems validating SousChef 'facebook-page-to-event': newslynx_sc_facebook.events is not importable. Here's the error: No module named newslynx_sc_facebook.events
failed on file /Users/mike/.newslynx/sous-chefs/newslynx-sc-facebook/newslynx_sc_facebook/facebook_page_to_event.yaml
failed on file /Users/mike/.newslynx/sous-chefs/newslynx-sc-facebook/newslynx_sc_facebook/facebook_page_to_event.yaml
Deleting the defaults
and sous-chefs
folder in ~./newslynx
and running that again gave me the same error as previous
It seems like if you run it twice it works. It must have something to do with installing the packages and then trying to read them in the same process. I guess they won't exist in the path yet on the first run.
that seems to work for me as well. is there a way to trick it and run those off of a list of separate requirements files?
Any luck?
Le Aug 23, 2015 à 23:51, Brian Abelson notifications@github.com a écrit :
It seems like if you run it twice it works. It must have something to do with installing the packages and then trying to read them in the same process. I guess they won't exist in the path.
— Reply to this email directly or view it on GitHub.
It seems like there is no way to refresh the list of accessible modules in the midst of a executing a program since these are determined by python
at runtime. So we'll need to somehow separate the installation of Sous Chefs and the initialization of default recipes which require these Sous Chefs. The easiest way to deal with this would be to leave this to automation, and for development convenience, add a command to the Makefile
which takes care of installing these modules before you run newslynx init
. In practice, you will only have to run this extra command once, since, once these modules are installed, they will exist in the PATH
, and you should always be able to run newslynx init
without hassle. I'll wait to hear your opinion before proceeding.
That sounds good.
Le Aug 25, 2015 à 01:30, Brian Abelson notifications@github.com a écrit :
It seems like there is no way to refresh the list of accessible modules in the midst of a executing a program since these are determined by python at runtime. So we'll need to somehow separate the installation of Sous Chefs and the initialization of default recipes which require these Sous Chefs. The easiest way to deal with this would be to leave this to automation, and for development convenience, add a command to the Makefile which takes care of installing these modules before you run newslynx init. In practice, you will only have to run this extra command once, since, once these modules are installed, they will exist in the PATH, and you should always be able to run newslynx init without hassle. I'll wait to hear your opinion before proceeding.
— Reply to this email directly or view it on GitHub.
icym the notification, this sounds good.
Just got this using automation
this is superseded by https://github.com/newslynx/opportunities/issues/103 now
22:36:17 | newslynx.tasks.default-95 | Creating sous chef: "internal-refresh-content-comparisons" 22:36:17 | newslynx.tasks.default-95 | Creating sous chef: "internal-refresh-content-summary-metrics"```` 22:36:17 | newslynx.cli.init-117 | Traceback (most recent call last): File "/Users/mike/.virtualenvs/newslynx/lib/python2.7/site-packages/newslynx-1.1.1-py2.7.egg/newslynx/cli/init.py", line 112, in run default.org() File "/Users/mike/.virtualenvs/newslynx/lib/python2.7/site-packages/newslynx-1.1.1-py2.7.egg/newslynx/tasks/default.py", line 57, in org sous_chefs(org) File "/Users/mike/.virtualenvs/newslynx/lib/python2.7/site-packages/newslynx-1.1.1-py2.7.egg/newslynx/tasks/default.py", line 89, in sous_chefs for sc, fp in init.load_sous_chefs(): File "/Users/mike/.virtualenvs/newslynx/lib/python2.7/site-packages/newslynx-1.1.1-py2.7.egg/newslynx/init.py", line 91, in load_sous_chefs for sc, fp in findsc(sous_chef_dir): File "/Users/mike/.virtualenvs/newslynx/lib/python2.7/site-packages/newslynx-1.1.1-py2.7.egg/newslynx/init.py", line 87, in findsc sc = sous_chef_schema.load(fp) File "/Users/mike/.virtualenvs/newslynx/lib/python2.7/site-packages/newslynx-1.1.1-py2.7.egg/newslynx/models/sous_chef_schema.py", line 50, in load raise SousChefSchemaError(msg) SousChefSchemaError: Traceback (most recent call last): File "/Users/mike/.virtualenvs/newslynx/lib/python2.7/site-packages/newslynx-1.1.1-py2.7.egg/newslynx/models/sous_chef_schema.py", line 47, in load return validate(sc, fp) File "/Users/mike/.virtualenvs/newslynx/lib/python2.7/site-packages/newslynx-1.1.1-py2.7.egg/newslynx/models/sous_chef_schema.py", line 99, in validate raise SousChefSchemaError(msg) SousChefSchemaError: Traceback (most recent call last): File "/Users/mike/.virtualenvs/newslynx/lib/python2.7/site-packages/newslynx-1.1.1-py2.7.egg/newslynx/models/sous_chef_schema.py", line 74, in validate _validate_python_sous_chef(sc) File "/Users/mike/.virtualenvs/newslynx/lib/python2.7/site-packages/newslynx-1.1.1-py2.7.egg/newslynx/models/sous_chef_schema.py", line 248, in _validate_python_sous_chef _raise_sous_chef_schema_error(sc, msg) File "/Users/mike/.virtualenvs/newslynx/lib/python2.7/site-packages/newslynx-1.1.1-py2.7.egg/newslynx/models/sous_chef_schema.py", line 279, in _raise_sous_chef_schema_error raise SousChefSchemaError(msg) SousChefSchemaError: There were problems validating SousChef 'facebook-page-to-event': newslynx_sc_facebook.events is not importable. Here's the error: No module named newslynx_sc_facebook.events
failed on file /Users/mike/.newslynx/sous-chefs/newslynx-sc-facebook/newslynx_sc_facebook/facebook_page_to_event.yaml
failed on file /Users/mike/.newslynx/sous-chefs/newslynx-sc-facebook/newslynx_sc_facebook/facebook_page_to_event.yaml