Canvas app isn't on production infrastructure

[seanh]

See also

• Canvas is an LMS (Learning Management System) used by schools to deliver courses. (Similar to Blackboard, Moodle, Sakai...)
• Canvas source code on GitHub
• The Hypothesis Canvas app:
  • Production deployment
  • Source code
  • Canvas app Google Docs from @judell:
  • Canvas Architecture Notes
  • Installing the App
  • Using the App
  • Student Guide
  • Screencasts from @jeremydean:
  • Student walkthrough screencast
  • Teacher walkthrough screencast
• Feb 2017 Canvas spike by Nick
• May 2017 Canvas Feature Spec Sheet (google doc)
• Canvas GitHub Issues label in the product-backlog repo, including:
  • Canvas app isn't production code quality https://github.com/hypothesis/product-backlog/issues/340
  • Canvas users have to manually create and activate Hypothesis accounts https://github.com/hypothesis/product-backlog/issues/342
  • Canvas users have to manually create and join Hypothesis groups https://github.com/hypothesis/product-backlog/issues/341
• #feat-canvas Slack channel
• Canvas folder on Google Drive (we should move all Canvas-related Google Docs into this folder)
• Our Instructure-hosted Canvas instance, we can use this to try out Canvas
• This issue is also tightly coupled with getting the Canvas app to production code quality.

Problem

There is a Canvas app for Hypothesis currently deployed at https://lti.hypothesislabs.com (git repo). I believe the server behind this is a Digital Ocean droplet run by @judell, it's not on the main Hypothesis infrastructure. This app is currently being used by more than 20 schools for classroom annotation assignments.

The Canvas app needs to be migrated onto our production infrastructure so that it can be maintained by the dev team (bringing the app back up if it goes down, deploying new changes to the app).

There's a standard way that we deploy Python apps at Hypothesis.

Solution

Deploy the Canvas app (by which I mean the standalone web service that currently runs at https://lti.hypothesislabs.com/) to our production infrastructure in the same way that h and bouncer are deployed.

Issues with this solution

• Some changes to the Canvas app will probably be needed in order for it to work on our production infrastructure. We'll know more once we've looked into the code some more, but for example:
  • The app needs to read all of its configuration from environment variables
  • The app needs to store state in Postgres
  • The app will need a Dockerfile (plus any changes necessary to get it running inside Docker)
  • The app will need integration with our Jenkins server (including a Jenkinsfile) and Docker Hub so that a new Docker image is built for it every time new changes are pushed to GitHub
• Schools and courses that are currently using the https://lti.hypothesislabs.com/ instance of the Canvas app have entered the https://lti.hypothesislabs.com/ URL into their Canvas instances when they added the app. How can we migrate them to the Canvas app's new URL? Does this require us to ask all the teachers and school administrators still using it to update their URLs?

Tasks

• [x] Don't log messages to an lti.log file in the filesystem
• [x] Read all config from environment variables
• [x] Store all state in Postgres (for example: the credentials.txt file and the canvas-auth.json file need to be change to SQL data)
• [ ] Can it continue to cache HTML and PDF files to the filesystem? We're going to remove this caching but it can be deployed to production with the caching still present at first
• [x] Adding a new institution or course to the app seems to require ssh into the server and manually editing the credentials.txt or canvas-auth.json files currently. This needs to work automatically, or at least without sshing in and editing files. (It's still necessary to ssh in and manually edit the DB to do this, docs are in the LTI app's README, it should be improved at some point)
• [x] Get the app working in Docker, create a Dockerfile that can be used to create a Docker image for the app
• [x] Get Jenkins building the Docker image and pushing it to Docker Hub on each commit to master, as it does for h and bouncer
• [x] Decide on a domain name for the app and get that setup, including SSL
• [x] Get the app added to the playbook and deployed to our production infra in the same way as h and bouncer
• [x] Start telling users to use the production instance of the app, instead of the lti.hypothesislabs instance (e.g. update any install docs that are shared with users)
• [x] Monitoring: Grafana dashboard based on statsd/graphite data from at least gunicorn and collectd
• [x] Monitoring: Sentry
• [x] Monitoring: New Relic

[judell]

Thanks Sean, I look forward to having this happen. Regarding migration to the new URL, perhaps we can just let the old one age out naturally as courses end.

[judell]

Re: splitting up lti.py, note that it also contains all the JS and CSS and templates, it was always my assumption you'd arrange those assets according to your own conventions.

[seanh]

Thanks Sean, I look forward to having this happen. Regarding migration to the new URL, perhaps we can just let the old one age out naturally as courses end.

Yeah, that sounds good @judell

[chdorner]

Like with every new service: we need monitoring :)

• Grafana dashboard (based on statsd/graphite data from at least gunicorn and collectd)
• Sentry integration
• Maybe new relic?

[seanh]

@chdorner Can you comment on this caching of HTML pages to the filesystem, with regards to getting the app onto production infrastructure?

[seanh]

I've added Grafana, Sentry and New Relic to the task list above

[chdorner]

The app is now running on qa and production, has a grafana dashboard, and has new relic set up.

According to @seanh this isn't ready to test yet because the code doesn't play well with our infrastructure, but everything is set up so that as soon as the code is fixed it'll all work.

• https://lti.hypothes.is
• https://qa-lti.hypothes.is
• https://mon.hypothes.is/grafana/dashboard/db/lti
• New Relic QA: https://rpm.newrelic.com/accounts/1385283/applications/55905799
• New Relic Prod: https://rpm.newrelic.com/accounts/1385283/applications/55906438
• Jenkins integration with auto QA deployments and manual confirmation for production deployments: https://jenkins.hypothes.is/job/lti
• Jenkins integration for running database migrations: https://jenkins.hypothes.is/job/lti-tasks/

I will leave the Sentry integration to you @seanh, because that requires a deeper understanding of the code.

[seanh]

Thanks @chdorner ! Leaving this issue open until I get Sentry going, and also fix the app to use the DB properly, update the public docs, but we're close to closing this

[chdorner]

Sentry is now done, see hypothesis/lti#73.

[seanh]

The app's using the DB properly as well. Needs some testing, and the public google docs need updated. Almost there

[seanh]

Update: I need to test that the app is basically working in production but I can't do so because I don't have an admin account on https://hypothesis.instructure.com/. Once I've done that then I can ask others (hopefully Jeremy and Jon) to end-to-end test it. Assuming no problems we'll then just need to update the Hypothes.is Canvas Alpha: Installing the App google doc and this card will be done.

[judell]

Excellent. You have the credentials in PassPack.

On Fri, Aug 18, 2017 at 6:30 AM, Sean Hammond notifications@github.com wrote:

Update: I need to test that the app is basically working but I can't do so because I don't have an admin account on https://hypothesis. instructure.com/. Once I've done that then I can ask others (hopefully Jeremy and Jon) to end-to-end test it. Assuming no problems we'll then just need to update the Hypothes.is Canvas Alpha: Installing the App https://docs.google.com/document/d/13FFtk2qRogtU3qxR_oa3kq2ak-S_p7HHVnNM12eZGy8/edit# google doc and this card will be done.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/hypothesis/product-backlog/issues/339#issuecomment-323353971, or mute the thread https://github.com/notifications/unsubscribe-auth/AAC1rWdcUPvKAr_jDAy9jBlGL1H1MC9Rks5sZZHtgaJpZM4OZsIf .