Dev env setup: "source: not found" error with "npm start"

[heatherbooker]

Hi! I am trying to set up the dev environment and got as far as [x] git clone webcompat.com [x] npm run module

Then npm run setup didn't work so I ran the commands individually:
[x] virtualenv env [x] . env/bin/activate [x] pip install -r config/requirements.txt [x] sudo npm install -g grunt-cli [x] npm install edit: I forgot to run grunt here! That's why the server couldn't find the css. -_- Oops. [x] cp config/secrets.py.example config/secrets.py

Then made a test repo, filled in OAUTH_TOKEN and github client secret & id.

Then I ran python run.py and while it serves the site to localhost:5000, the CSS didn't seem to be applying - I found in the browser console that it couldn't locate some files in css and api dirs:

http://localhost:5000/css/webcompat.dev.css
http://localhost:5000/api/issues/labels?per_page=100
http://localhost:5000/api/issues/category/needstriage

edit: running grunt fixed the css, but the api dir is still a 404.

I tried leaving the virtualenv and running npm start, but it had the same problem as when I tried to npm run setup before -

sh: 1: source: not found

I guess it is using sh shell instead of bash (or zsh, which I normally use). I modified package.json to not use source:

"start": ". env/bin/activate && python run.py",

and then running npm start worked equally as well as manually entering the virtualenv and doing python run.py.

Running the server also tells me

home/webcompat/env/local/lib/python2.7/site-packages/flask_limiter/extension.py:124: UserWarning: Use of the default `get_ipaddr` function is discouraged. Please refer to https://flask-limiter.readthedocs.org/#rate-limit-domain for the recommended configuration

but if I am reading this comment in #1304 correctly, that is a non-issue right now??

Anyway. I'm running Linux Mint 18 Sarah and running the commands from zshell, if that makes a difference (edit: apparently not, I tried running it from bash and same outcome.). I would love to know how to tackle this! :)

[miketaylr]

Hi! I am trying to set up the dev environment and got as far as

Hello! Thanks for the detailed bug report. :)

Just so I understand -- the problem you're having is not being able to get the virtualenv to activate via the source command? I'll confess to not knowing much about shell differences -- I'm on a mac.

It sounds like you were able to get the local server running though, is that correct?

but if I am reading this comment in #1304 correctly, that is a non-issue right now??

Yeah, that's just console spam for now. OK to ignore.

[heatherbooker]

Cool. Yes, the problem seems to be with activating the virtualenv via the source command.

Straight from my shell (tried with both zsh and bash), I can run source with no problems. I can also change the npm start command to use . instead of source and that runs with no problems.

But yes, the local server is running!

[karlcow]

fwiw, I never start the server outside of the virtualenv.

→ nvm ls
         v4.0.0
         v4.2.0
->       system
node -> stable (-> v4.2.0) (default)
stable -> 4.2 (-> v4.2.0) (default)
iojs -> N/A (default)

Here I try to start the server.

→ npm start

> webcompat@ start /Users/karl/code/webcompat.com
> source env/bin/activate && python run.py

('secrets', '/Users/karl/code/webcompat.com')
Traceback (most recent call last):
  File "run.py", line 43, in <module>
    raise ImportError('{0}\n\n{1}'.format(e, IMPORT_ERROR))
ImportError: No module named flask_wtf

==============================================
It seems like you don't have all dependencies.
Please re-run:
    pip install -r config/requirements.txt
==============================================

npm ERR! webcompat@ start: `source env/bin/activate && python run.py`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the webcompat@ start script.
npm ERR! This is most likely a problem with the webcompat package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     source env/bin/activate && python run.py
npm ERR! You can get their info via:
npm ERR!     npm owner ls webcompat
npm ERR! There is likely additional logging output above.
npm ERR! System Darwin 16.4.0
npm ERR! command "node" "/usr/local/bin/npm" "start"
npm ERR! cwd /Users/karl/code/webcompat.com
npm ERR! node -v v0.10.30
npm ERR! npm -v 1.4.21
npm ERR! code ELIFECYCLE
npm ERR! 
npm ERR! Additional logging details can be found in:
npm ERR!     /Users/karl/code/webcompat.com/npm-debug.log
npm ERR! not ok code 0

Then

→ nvm use 4.2
Now using node v4.2.0 (npm v2.14.7)

and trying again

→ npm start

> webcompat@ start /Users/karl/code/webcompat.com
> source env/bin/activate && python run.py

('secrets', '/Users/karl/code/webcompat.com')
Traceback (most recent call last):
  File "run.py", line 43, in <module>
    raise ImportError('{0}\n\n{1}'.format(e, IMPORT_ERROR))
ImportError: No module named flask_wtf

==============================================
It seems like you don't have all dependencies.
Please re-run:
    pip install -r config/requirements.txt
==============================================

npm ERR! Darwin 16.4.0
npm ERR! argv "/Users/karl/.nvm/versions/node/v4.2.0/bin/node" "/Users/karl/.nvm/versions/node/v4.2.0/bin/npm" "start"
npm ERR! node v4.2.0
npm ERR! npm  v2.14.7
npm ERR! code ELIFECYCLE
npm ERR! webcompat@ start: `source env/bin/activate && python run.py`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the webcompat@ start script 'source env/bin/activate && python run.py'.
npm ERR! This is most likely a problem with the webcompat package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     source env/bin/activate && python run.py
npm ERR! You can get their info via:
npm ERR!     npm owner ls webcompat
npm ERR! There is likely additional logging output above.

npm ERR! Please include the following file with any support request:
npm ERR!     /Users/karl/code/webcompat.com/npm-debug.log

It never worked for me.

I usually do:

# my alias for the project using virtualenvwrapper. Same than the activate.
workon webcompatcom
# Then I go on to python run.py
python run.py

[heatherbooker]

@karlcow Interesting.. two questions.

1) Do you always enter the venv when you are doing anything with the project? Running unit tests? Functional tests? npm installing? 2) Jk I guess this is not actually a question. I guess I'll try setting up virtualenvwrapper. My only concern is that it wouldn't have helped in the first place, as a newbie trying to set up a dev environment. I'd like to smooth this bump for other people who might run into the same difficulty I have. :)

[karlcow]

@heatherbooker Always. I do everything in the virtualenv. never outside.

yes what you are doing is good. We probably need to improve the first steps. It's still bumpy.

[magsout]

except the wanrning it works for me:

MacBook-Air:webcompat.com GuillaumeDEMESY$ npm run start

> webcompat@ start /Users/GuillaumeDEMESY/Github/webcompat/webcompat.com
> source env/bin/activate && python run.py

/Users/GuillaumeDEMESY/Github/webcompat/webcompat.com/env/lib/python2.7/site-packages/flask_limiter/extension.py:124: UserWarning: Use of the default `get_ipaddr` function is discouraged. Please refer to https://flask-limiter.readthedocs.org/#rate-limit-domain for the recommended configuration
  UserWarning
 * Running on http://localhost:5000/ (Press CTRL+C to quit)
 * Restarting with stat
/Users/GuillaumeDEMESY/Github/webcompat/webcompat.com/env/lib/python2.7/site-packages/flask_limiter/extension.py:124: UserWarning: Use of the default `get_ipaddr` function is discouraged. Please refer to https://flask-limiter.readthedocs.org/#rate-limit-domain for the recommended configuration
  UserWarning

[magsout]

Ok, I will try to make a fresh install and understand why...

[zoepage]

@heatherbooker This sounds familiar. Did you log into the app? (Despite having the correct credentials, you need to authorize the app.) The login is in the upper right corner.

[miketaylr]

Maybe we should change instructions to only use virtualenvwrapper

(but also, unless you globally installed all your pip packages, which is not recommended, the server shouldn't work outside of a venv. it's not bad, you just lose all the benefits of a sandboxed virtual env)

[heatherbooker]

So regarding the sh: source: not found error I was getting: Changing my package.json start script to include an option to use . instead of source:

// Current:
"start": "source env/bin/activate && python run.py",

// Proposed change:
"start": "source env/bin/activate || . env/bin/activate && python run.py",

means npm run start goes smoothly - starts the virtualenv and then the server. This also seems to work equally well for the npm run virtualenv command. Specifically, it enters the virtualenv to pip install stuff, but once the command completes, I'm not in the virtualenv. Is that the expected behaviour?

1) Should I send a PR with those changes to npm run start and npm run virtualenv? 2) Or simply send a PR adding a note to docs that if you get this error, try manually running the command and/or modifying source -> . ?

It seems to me that the virtualenv / wrapper is a separate issue - I am neither trying to nor succeeding in running the server without the virtualenv being activated. I wouldn't object to adding emphasis about needing to be in the virtualenv to the docs?

also @zoepage : Ah, I now see that instruction to log in on the contributions docs! Cool, thank you. I'm debugging rn but not currently having luck accessing issues, despite being logged in. update: Oh I see, there are no issues because it references my test-repo, which of course has no issues. All good on that front then. ^^

[miketaylr]

Ah https://github.com/heatherbooker/webcompat.com/commit/273025419c3c80d283a7e776e7e0ce7ceb09b295 is a nice solution.

[zoepage]

@heatherbooker glad it worked out :) LMK if you have more questions!

[heatherbooker]

@miketaylr do you want to see it as a PR?

[miketaylr]

@miketaylr do you want to see it as a PR?

yes please.