search

p-v-o-s / infrapix-flask

flask app for developing infragram photos
9 stars 3 forks source link

install infrapix-flask on public lab machine #15

Closed dwblair closed 11 years ago

dwblair commented 11 years ago

Following tutorial: http://blog.kramerapps.com/post/22551999777/flask-uwsgi-nginx-ubuntu

dwblair commented 11 years ago

Another nice, related set of docs: https://library.linode.com/web-servers/nginx/python-uwsgi/ubuntu-12.04-precise-pangolin

dwblair commented 11 years ago

Following the tutorial here: http://blog.kramerapps.com/post/22551999777/flask-uwsgi-nginx-ubuntu

Everything works until I get here:

sudo service uwsgi restart

which returns

uwsgi: unrecognized service

I believe that this means that there is no uwsgi startup script in /etc/init.d/ ... and checking, there seems to be none.

There's a guide to setting up an appropriate script, here:

http://www.mrj0.com/2011/05/06/outage/

I'm thinking of following this next ...

jywarren commented 11 years ago

I believe it means uwsgi is not installed -- i'm going to try running "aptitude install uwsgi" and try restarting it again.

jywarren commented 11 years ago

OK, great: 

[ ok ] Starting app server(s): uwsgi (omitted; missing conffile(s) in /etc/uwsgi/apps-enabled).

However, i think that means we need to create an entry in /etc/uwsgi/apps-available, and symlink it into /etc/uwsgi/apps-enabled. Reading more about how to do that now.

dwblair commented 11 years ago

Ah, wow -- great progress!

Also, forgot to mention: Dogi had the great idea of keeping track of installation steps we're following in this document, here:

https://docs.google.com/document/d/1DPS9isCaOmln4o_TE8k-9lzjWTg8fecu6FTES_s3it4/edit

When you get a chance you might add the steps you've followed, there ...

On Mon, Aug 5, 2013 at 1:52 PM, Jeffrey Warren notifications@github.comwrote:

OK, great:

[ ok ] Starting app server(s): uwsgi (omitted; missing conffile(s) in /etc/uwsgi/apps-enabled).

However, i think that means we need to create an entry in /etc/uwsgi/apps-available, and symlink it into /etc/uwsgi/apps-enabled. Reading more about how to do that now.

— Reply to this email directly or view it on GitHubhttps://github.com/Pioneer-Valley-Open-Science/infrapix-flask/issues/15#issuecomment-22123743 .

web: http://dwblair.github.io <---- info, projects, docs, etc skype: dwingateb voice & SMS: +1-651-252-4765 twitter: @donwblair

jywarren commented 11 years ago

OK, a few things:

dwblair commented 11 years ago

Wow ... more progress!

Question (not sure if this makes sense): from which directory will http://beta.infragram.org/ (or infragram.org, for that matter) content be served? I can start building up the app in that directory, but I'm such a n00b that I can't figure out where it ought to live ...

On Mon, Aug 5, 2013 at 2:05 PM, Jeffrey Warren notifications@github.comwrote:

OK, a few things:

  • I pointed http://beta.infragram.org/ at the server, lets use that until we're ready to push to infragram.org
  • I moved the nginx config to the more standard /etc/nginx/sites-available/default and commented out the conflicting default localhost:80 declaration so ours is now default
  • I'm still getting 404, so checking permissions...

— Reply to this email directly or view it on GitHubhttps://github.com/Pioneer-Valley-Open-Science/infrapix-flask/issues/15#issuecomment-22124667 .

web: http://dwblair.github.io <---- info, projects, docs, etc skype: dwingateb voice & SMS: +1-651-252-4765 twitter: @donwblair

jywarren commented 11 years ago

I believe it should be /srv/www/infragram/ or /srv/www/infrapix/, your choice. We'll then move the pointers to the helloworld app over to match your new app. Thanks!

I'm currently stuck on getting uwsgi to recognize a conf file in /etc/uwsgi/apps-enabled/ ... i get the error message:

[ ok ] Restarting app server(s): uwsgi (omitted; missing conffile(s) in

/etc/uwsgi/apps-enabled).

although i tried moving your conf file from /etc/init/uwsgi.conf into /etc/uwsgi/apps-available/ and symlinking it into apps-enabled, as is normal for such conf files... not sure why it's not picking it up.

jywarren commented 11 years ago

Darn, i got past that last error, by renaming the conf file /etc/uwsgi/apps-available/infragram.ini

... (.ini suffix important i guess?) but although uwsgi now restarts with no error, nginx is still showing a 404 not found error: http://beta.infragram.org/ even after rebooting nginx too...

dwblair commented 11 years ago

Sounds like you're getting really close!

(oops -- just realized -- duh ... users won't see the /srv/www/infrapix directory anyway, so no chance of confusing them -- still and all, let's go for infragram ...)

jywarren commented 11 years ago

OK, now getting: 

2013/08/05 18:42:22 [error] 20585#0: *19 open() "/usr/share/nginx/html/50x.html" failed (2: No such file or directory), client: 18.248.6.62, server: localhost, request: "GET /favicon.ico HTTP/1.1", upstream: "uwsgi://unix:/tmp/uwsgi.sock:", host: "beta.infragram.org"

Sorry to spam up this thread with my troubleshooting, but i think it's helpful to document how we get there...

I'm assuming uwsgi is somehow not creating the socket file. Will see if we can read the error on that end too. Oooh, there's an "infragram" log in /var/log/uwsgi/app/infragram.log:

Mon Aug  5 18:37:18 2013 - *** Starting uWSGI 1.2.3-debian (64bit) on [Mon Aug  5 18:37:18 2013] ***
Mon Aug  5 18:37:18 2013 - compiled with version: 4.7.1 on 13 September 2012 19:07:14
Mon Aug  5 18:37:18 2013 - detected number of CPU cores: 1
Mon Aug  5 18:37:18 2013 - current working directory: /
Mon Aug  5 18:37:18 2013 - writing pidfile to /run/uwsgi/app/infragram/pid
Mon Aug  5 18:37:18 2013 - detected binary path: /usr/bin/uwsgi-core
Mon Aug  5 18:37:18 2013 - setgid() to 33
Mon Aug  5 18:37:18 2013 - setuid() to 33
Mon Aug  5 18:37:18 2013 - your memory page size is 4096 bytes
Mon Aug  5 18:37:18 2013 - detected max file descriptor number: 1024
Mon Aug  5 18:37:18 2013 - lock engine: pthread robust mutexes
Mon Aug  5 18:37:18 2013 - uwsgi socket 0 bound to UNIX address /run/uwsgi/app/infragram/socket fd 3
Mon Aug  5 18:37:18 2013 - your server socket listen backlog is limited to 100 connections
Mon Aug  5 18:37:18 2013 - *** Operational MODE: preforking ***
Mon Aug  5 18:37:18 2013 - *** no app loaded. going in full dynamic mode ***
Mon Aug  5 18:37:18 2013 - *** uWSGI is running in multiple interpreter mode ***
Mon Aug  5 18:37:18 2013 - spawned uWSGI master process (pid: 20545)
Mon Aug  5 18:37:18 2013 - spawned uWSGI worker 1 (pid: 20547, cores: 1)
Mon Aug  5 18:37:18 2013 - spawned uWSGI worker 2 (pid: 20548, cores: 1)
jywarren commented 11 years ago

Why would it be running anything called "infragram"? Is that you installing infrapix, don?

dwblair commented 11 years ago

Hmm ... I haven't touched the server since yesterday (when I installed r-base-dev, on a whim) ... but perhaps now it's running what I'd set up previously (I forget if I set up a version of the Flask app already) ... lemme check ...

On Mon, Aug 5, 2013 at 2:47 PM, Jeffrey Warren notifications@github.comwrote:

Why would it be running anything called "infragram"? Is that you installing infrapix, don?

— Reply to this email directly or view it on GitHubhttps://github.com/Pioneer-Valley-Open-Science/infrapix-flask/issues/15#issuecomment-22128010 .

web: http://dwblair.github.io <---- info, projects, docs, etc skype: dwingateb voice & SMS: +1-651-252-4765 twitter: @donwblair

dwblair commented 11 years ago

Nope, I hadn't installed anything re: infrapix / infragram, yet ... so I dunno what's going on ...

Aside: spam away! This is useful (and dramatic)!

jywarren commented 11 years ago

ok, well, in any case the problem is that /tmp/uwsgi.sock does not exist... presumably because uwsgi is not actually booting the "helloworld" app, or in fact any app at all, on a socket. Maybe my moving the config from /etc/init/uwsgi.conf to /etc/uwsgi/apps-available/ and /etc/uwsgi/apps-enabled/ was misguided.

I do see this guy: https://fclef.wordpress.com/2013/01/12/bottle-virtualenv-uwsgi-nginx-installation-on-ubuntu-12-04-1-lts/ recommending a config file looking like this:

[uwsgi]
plugins=python
socket=/tmp/uwsgi.myapp.socket
pythonpath=/var/www/myapp

and putting it here: /etc/uwsgi/apps-enabled/uwsgi.ini

... we need better examples...

jywarren commented 11 years ago

OK, reading the above-linked docs: https://library.linode.com/web-servers/nginx/python-uwsgi/ubuntu-12.04-precise-pangolin

says we need "uwsgi-plugin-python" package installed, and it's not! so I ran: 

sudo aptitude install uwsgi-plugin-python

But no change. I went back to the nginx logs at /var/log/nginx/error.log, and found that we are actually reading from the right socket, it seems, but there's an odd message: "upstream prematurely closed connection while reading response header from upstream", and mentions: 'upstream: "uwsgi://unix:/run/uwsgi/app/infragram/socket" which is what I'd just set the socket to in /etc/nginx/sites-available/default since /tmp/uwsgi.sock was not working.

This seems to me like there is actually a file at /run/uwsgi/app/infragram/socket -- but that it's not responding appropriately to requests from nginx.

jywarren commented 11 years ago

Hooray!!!! It's working -- the missing steps were:

Let's just remember that to get the infrapix app working, we need to change all instances of "helloworld" to "infrapix-flask"

Adding this all to Dogi's config doc.

jywarren commented 11 years ago

Ah, i don't have write-access to that google doc. Compiling here:

jywarren commented 11 years ago

I git cloned infrapix-flask into /srv/www/infrapix-flask, and chmodded it to be owned by the nginx group, and started converting all instances of "helloworld" to "infrapix-flask" and where necessary "infrapix-flask/app". But i'm not on as sure ground here, maybe Don should take over for a bit?

jywarren commented 11 years ago

OK, actually we are almost there. It needs us to point at the main launch script of the infrapix app, which I gather is "passenger_wsgi.py" -- and to do so correctly in both config files, /etc/uwsgi/apps-available/infragram.org.xml and /etc/nginx/sites-available/default -- which I've tried to do. But I think i don't know enough about how a python app is supposed to be set up to fill out those config files correctly.

dwblair commented 11 years ago

WOOHOOO!

Okay, I'll look back through what you've done and see if I be helpful ... there's material on Dreamhost re: the way they set up passenger_wsgi.py, and I'll see if it's relevant ...

jywarren commented 11 years ago

I think if instead of passenger_wsgi, if there were a default sort of application.py, that would be more what we're looking for.

On Mon, Aug 5, 2013 at 3:55 PM, dwblair notifications@github.com wrote:

WOOHOOO!

Okay, I'll look back through what you've done and see if I be helpful ... there's material on Dreamhost re: the way they set up passenger_wsgi.py, and I'll see if it's relevant ...

— Reply to this email directly or view it on GitHubhttps://github.com/Pioneer-Valley-Open-Science/infrapix-flask/issues/15#issuecomment-22136403 .

dwblair commented 11 years ago

Oh ... that should be "views.py" in the "app" folder ... lemme double check ...

On Mon, Aug 5, 2013 at 3:57 PM, Jeffrey Warren notifications@github.comwrote:

I think if instead of passenger_wsgi, if there were a default sort of application.py, that would be more what we're looking for.

On Mon, Aug 5, 2013 at 3:55 PM, dwblair notifications@github.com wrote:

WOOHOOO!

Okay, I'll look back through what you've done and see if I be helpful ... there's material on Dreamhost re: the way they set up passenger_wsgi.py, and I'll see if it's relevant ...

— Reply to this email directly or view it on GitHub< https://github.com/Pioneer-Valley-Open-Science/infrapix-flask/issues/15#issuecomment-22136403>

.

— Reply to this email directly or view it on GitHubhttps://github.com/Pioneer-Valley-Open-Science/infrapix-flask/issues/15#issuecomment-22136552 .

web: http://dwblair.github.io <---- info, projects, docs, etc skype: dwingateb voice & SMS: +1-651-252-4765 twitter: @donwblair

dwblair commented 11 years ago

Okay, installing some prereq's ...

and whilst in the virtualenv, installing Flask via pip ...

On Mon, Aug 5, 2013 at 4:03 PM, Don Blair donblair@pvos.org wrote:

Oh ... that should be "views.py" in the "app" folder ... lemme double check ...

On Mon, Aug 5, 2013 at 3:57 PM, Jeffrey Warren notifications@github.comwrote:

I think if instead of passenger_wsgi, if there were a default sort of application.py, that would be more what we're looking for.

On Mon, Aug 5, 2013 at 3:55 PM, dwblair notifications@github.com wrote:

WOOHOOO!

Okay, I'll look back through what you've done and see if I be helpful ... there's material on Dreamhost re: the way they set up passenger_wsgi.py, and I'll see if it's relevant ...

— Reply to this email directly or view it on GitHub< https://github.com/Pioneer-Valley-Open-Science/infrapix-flask/issues/15#issuecomment-22136403>

.

— Reply to this email directly or view it on GitHubhttps://github.com/Pioneer-Valley-Open-Science/infrapix-flask/issues/15#issuecomment-22136552 .

web: http://dwblair.github.io <---- info, projects, docs, etc skype: dwingateb voice & SMS: +1-651-252-4765 twitter: @donwblair

web: http://dwblair.github.io <---- info, projects, docs, etc skype: dwingateb voice & SMS: +1-651-252-4765 twitter: @donwblair

dwblair commented 11 years ago

pip install numpy pip install matplotlib pip install Pillow

... and edited the passenger_wgsi.py file to refer to the virtualenv that I set up in /srv/www/venv-infrapix-flask ...

on Dreamhost, one then touches tmp/restart.txt in the application's directory -- which I just did ...

... and now it might be worth testing. I'll navigate to the site and see what happens ... (maybe something needs to be restarted, tho?)

On Mon, Aug 5, 2013 at 4:21 PM, Don Blair donblair@pvos.org wrote:

Okay, installing some prereq's ...

  • python-virtualenv
  • python-matplotlib

and whilst in the virtualenv, installing Flask via pip ...

On Mon, Aug 5, 2013 at 4:03 PM, Don Blair donblair@pvos.org wrote:

Oh ... that should be "views.py" in the "app" folder ... lemme double check ...

On Mon, Aug 5, 2013 at 3:57 PM, Jeffrey Warren notifications@github.comwrote:

I think if instead of passenger_wsgi, if there were a default sort of application.py, that would be more what we're looking for.

On Mon, Aug 5, 2013 at 3:55 PM, dwblair notifications@github.com wrote:

WOOHOOO!

Okay, I'll look back through what you've done and see if I be helpful ... there's material on Dreamhost re: the way they set up passenger_wsgi.py, and I'll see if it's relevant ...

— Reply to this email directly or view it on GitHub< https://github.com/Pioneer-Valley-Open-Science/infrapix-flask/issues/15#issuecomment-22136403>

.

— Reply to this email directly or view it on GitHubhttps://github.com/Pioneer-Valley-Open-Science/infrapix-flask/issues/15#issuecomment-22136552 .

web: http://dwblair.github.io <---- info, projects, docs, etc skype: dwingateb voice & SMS: +1-651-252-4765 twitter: @donwblair

web: http://dwblair.github.io <---- info, projects, docs, etc skype: dwingateb voice & SMS: +1-651-252-4765 twitter: @donwblair

web: http://dwblair.github.io <---- info, projects, docs, etc skype: dwingateb voice & SMS: +1-651-252-4765 twitter: @donwblair

dwblair commented 11 years ago

(forgot to add: sudo apt-get build-dep python-matplotlib)

On Mon, Aug 5, 2013 at 4:45 PM, Don Blair donblair@pvos.org wrote:

pip install numpy pip install matplotlib pip install Pillow

... and edited the passenger_wgsi.py file to refer to the virtualenv that I set up in /srv/www/venv-infrapix-flask ...

on Dreamhost, one then touches tmp/restart.txt in the application's directory -- which I just did ...

... and now it might be worth testing. I'll navigate to the site and see what happens ... (maybe something needs to be restarted, tho?)

On Mon, Aug 5, 2013 at 4:21 PM, Don Blair donblair@pvos.org wrote:

Okay, installing some prereq's ...

  • python-virtualenv
  • python-matplotlib

and whilst in the virtualenv, installing Flask via pip ...

On Mon, Aug 5, 2013 at 4:03 PM, Don Blair donblair@pvos.org wrote:

Oh ... that should be "views.py" in the "app" folder ... lemme double check ...

On Mon, Aug 5, 2013 at 3:57 PM, Jeffrey Warren <notifications@github.com

wrote:

I think if instead of passenger_wsgi, if there were a default sort of application.py, that would be more what we're looking for.

On Mon, Aug 5, 2013 at 3:55 PM, dwblair notifications@github.com wrote:

WOOHOOO!

Okay, I'll look back through what you've done and see if I be helpful ... there's material on Dreamhost re: the way they set up passenger_wsgi.py, and I'll see if it's relevant ...

— Reply to this email directly or view it on GitHub< https://github.com/Pioneer-Valley-Open-Science/infrapix-flask/issues/15#issuecomment-22136403>

.

— Reply to this email directly or view it on GitHubhttps://github.com/Pioneer-Valley-Open-Science/infrapix-flask/issues/15#issuecomment-22136552 .

web: http://dwblair.github.io <---- info, projects, docs, etc skype: dwingateb voice & SMS: +1-651-252-4765 twitter: @donwblair

web: http://dwblair.github.io <---- info, projects, docs, etc skype: dwingateb voice & SMS: +1-651-252-4765 twitter: @donwblair

web: http://dwblair.github.io <---- info, projects, docs, etc skype: dwingateb voice & SMS: +1-651-252-4765 twitter: @donwblair

web: http://dwblair.github.io <---- info, projects, docs, etc skype: dwingateb voice & SMS: +1-651-252-4765 twitter: @donwblair

dwblair commented 11 years ago

Okay, so in sum:

Then one needs to create a the virtualenv, which I'm doing in the /srv/www directory:

Then one needs to edit the passenger_wgsi.py file to refer to the virtualenv: INTERP = os.path.expanduser("/srv/www/venv-infrapix-flask/bin/python")

... then I'm restarting uwsgi and nginx, via sudo service uwsgi restart and sudo service nginx restart

... no dice yet. I'm still getting: "uWSGI Error Python application not found"

hmmm ...

dwblair commented 11 years ago

Looking into following these instructions: http://serverfault.com/questions/343050/uwsgi-and-nginx-for-python-apps-on-ubuntu-11-10

dwblair commented 11 years ago

Doesn't seem like it's a problem with passenger_wsgi.py; instead, seems like it's an nginx configuration file issue ... also looking at these instructions here: http://paperairoplane.net/?p=457

jywarren commented 11 years ago

I reset everything to use passenger_wsgi.py as the base script -- part of the problem was also that the root directory for python needs to be the root directory for the infrapix repo -- so we should save the nginx and uwsgi config scripts as I think I have them correctly now.

I installed virtualenv in /srv/www/infrapix-flask/ as I think that's more standard? And then pointed the entry in passenger_wsgi.py at that version of python, i.e. /srv/www/infrapix-flask/env/bin/python

then that version of python didn't have uwsgi installed, so from the app root dir, source env/bin/activate and pip install uwsgi

Still getting /srv/www/infrapix-flask/env/bin/python: can't open file 'uwsgi': [Errno 2] No such file or directory - in /var/log/uwsgi/app/infragram.org.log but at least it's looking!

dwblair commented 11 years ago

Oh, great!

On Tue, Aug 6, 2013 at 9:27 AM, Jeffrey Warren notifications@github.comwrote:

I reset everything to use passenger_wsgi.py as the base script -- part of the problem was also that the root directory for python needs to be the root directory for the infrapix repo -- so we should save the nginx and uwsgi config scripts as I think I have them correctly now.

I installed virtualenv in /srv/www/infrapix-flask/ as I think that's more standard? And then pointed the entry in passenger_wsgi.py at that version of python, i.e. /srv/www/infrapix-flask/env/bin/python

then that version of python didn't have uwsgi installed, so from the app root dir, source env/bin/activate and pip install uwsgi

Still getting /srv/www/infrapix-flask/env/bin/python: can't open file 'uwsgi': [Errno 2] No such file or directory - in /var/log/uwsgi/app/infragram.org.log but at least it's looking!

— Reply to this email directly or view it on GitHubhttps://github.com/Pioneer-Valley-Open-Science/infrapix-flask/issues/15#issuecomment-22178307 .

web: http://dwblair.github.io <---- info, projects, docs, etc skype: dwingateb voice & SMS: +1-651-252-4765 twitter: @donwblair

jywarren commented 11 years ago

OK -- one final tweak seems to have gotten the app running: uwsgi_param UWSGI_CALLABLE application; in the nginx config file, which refers to the line in passenger_wsgi.py - from app import app as application

Now, image uploading doesn't work, but that's not surprising. Where do I look for the python log?

jywarren commented 11 years ago

I set "app.debug = True" and now see that it doesn't have write permissions for the /uploads/ dir. fixing.

done! it works!

jywarren commented 11 years ago

large image uploads now work: http://beta.infragram.org/show/wash-sq-park.png

im increasing nginx upload limit to 20mb, uwsgi memory limit to 2048 (from 512)

Even larger: http://beta.infragram.org/show/IMG_0876_copy.jpg

2.5mb: http://beta.infragram.org/show/IMG_0876.JPG

jywarren commented 11 years ago

moving to infragram.org now

jywarren commented 11 years ago

As a followup, I think we probably have redundant installations of uwsgi at this point -- one from apt-get and one from pip... although I believe we are running the apt-get version, installed as a system-level service. I also am not convinced we are actually using virtualenv at all -- It only began working when I removed the lines from passenger_wsgi.py which pointed at that version of python.

When using virtualenv, it was not able to find uwsgi, despite the fact that I had installed it with pip. Maybe I just don't know how to install dependencies in a virtualenv... i'm new to it. But as we're currently running on the system-level apt-get version, I'm OK with that :-P

dwblair commented 11 years ago

Ah ... that's interesting. I suppose I could test this in a crude way e.g. removing one of the packages that was installed via virtualenv, and seeing whether things break. Maybe I can do that in the development version, in order to nail down the dependencies. I am sort of taken with concept of virtualenv, but it seems that it only covers some of the needed dependencies. Anyway, I guess we'll learn more soon about how all of these pieces fit together. Well done!

On Tue, Aug 6, 2013 at 11:32 AM, Jeffrey Warren notifications@github.comwrote:

As a followup, I think we probably have redundant installations of uwsgi at this point -- one from apt-get and one from pip... although I believe we are running the apt-get version, installed as a system-level service. I also am not convinced we are actually using virtualenv at all -- It only began working when I removed the lines from passenger_wsgi.py which pointed at that version of python.

When using virtualenv, it was not able to find uwsgi, despite the fact that I had installed it with pip. Maybe I just don't know how to install dependencies in a virtualenv... i'm new to it. But as we're currently running on the system-level apt-get version, I'm OK with that :-P

— Reply to this email directly or view it on GitHubhttps://github.com/Pioneer-Valley-Open-Science/infrapix-flask/issues/15#issuecomment-22188001 .

web: http://dwblair.github.io <---- info, projects, docs, etc skype: dwingateb voice & SMS: +1-651-252-4765 twitter: @donwblair

jywarren commented 11 years ago

Starting a local-environment setup guide here, but it's very unfinished: https://github.com/Pioneer-Valley-Open-Science/infrapix-flask/wiki