The application is primarily written in TypeScript and makes use of the React framework for the client and the Nest framework for the NodeJS server.
Configuration for the project is currently managed using environment variables.
Docker handles the setting of key-value pairs in docker-compose.yml
,
through the environment
block and by consuming a .env
file.
Docker will expect to find a .env
file at the project's root directory.
This repository includes in the config
directory a .env.sample
template file for .env
,
with the expected keys provided. Comments above each key describe what the value is used for,
and in some cases, indicate that the key-value pair is optional.
The config.ts
module in the ccm_web/server
directory validates this file,
using fallbacks when available and throwing an error if required values are not present.
The application will exit if any of the required configuration values are not set properly.
Currently, the application can only be used with a Learning Tool Interoperability (LTI)
integration with a Canvas LMS instance. To set up the application in Canvas,
you must create an LTI Developer Key at the account level and then install the application
in "External Apps" at the course or account level through the appropriate "Settings" page.
Canvas has documentation related to
creating an LTI Developer Key
and installing an external app
(latter link is for course-level installation).
The client ID associated with the LTI Developer Key you create will be used when installing the application.
You will also need to specify the client ID -- as well as the Canvas instance URL -- in the .env
configuration file
(see Configuration - Application above).
This project requires some specific values and settings to be included in the LTI Developer Key.
To simplify the creation of these keys, a JSON template called lti_dev_key_sample.json
has
been provided in the config
directory. Once replacing all instances of {ccm_app_url}
with
the URL (without the protocol) for the instance you are configuring,
you can simply paste the JSON in the Canvas form.
(You will want to enter the Key Name and Owner Email separately.)
Authorization for making changes in Canvas using the application is managed using
the Canvas OAuth workflow
and scoped tokens. As such, you will also need to create an API Developer Key (see the Canvas documentation
here).
The "Redirect URI" must be the protocol and domain of the application, plus /canvas/returnFromOAuth
.
The list of API scopes to specify are in flux. Refer to ccm_web/server/src/canvas/canvas.scopes.ts
for details
on the latest expected scopes. Once the API Developer Key is created, you will need to collect
the associated client ID and secret and specify them in the .env
configuration file.
Once you have fully configured the application (see the Configuration section above), you can use the following steps to build and run the application using Docker.
Build a development image for the web application.
docker-compose build
Start up the server and webpack for development.
docker-compose up
Access the client by launching the tool from a Canvas course in your browser of choice.
Use ^C
to stop the container and docker-compose down
to remove the last used image from staging.
Many TypeScript and static file changes can be made without re-building the image, but as necessary, rebuild the image using Step 1.
Explicit steps for setting up CCM in a development environment.
ngrok http 4000
jq
is installed, use the command:
curl --silent http://127.0.0.1:4040/api/tunnels | jq -r '.tunnels[0].public_url'
Note: Get only the hostname, not the http
/https
scheme prefix.cp config/lti_dev_key_sample.json lti_dev_key.json
lti_dev_key.json
file to add or replace settings from the sample. Replace all occurrences of {ccm_app_hostname}
with the ngrok hostname copied in the earlier step.config/lti_dev_key.json
and paste it into the "LTI 1.3 Configuration" field.Optional: LTI Key, use "Manual Entry" under the "Method" menu to access "LTI Advantage Services" under the "Configuration" section. Enable all options, then click the "Save" button.
scopes
key of the JSON.17700000000000nnn
".💡 The name of this key is used as the title and in the body of the authorization dialog box shown to the user. Be sure to pick a meaningful name for this key. For example, name it "Canvas Course Manager", NOT "CCM API key".
https://{ccm_app_hostname}/canvas/returnFromOAuth
Where {ccm_app_hostname}
is the ngrok hostname copied earlier (in step 2).
Note: Do NOT use the "Redirect URI (Legacy)" field.ccm_web/server/src/canvas/canvas.scopes.ts
).17700000000000nnn
"..env
file for the project, based on the sample provided.
cp config/.env.sample .env
Edit the .env
file. The keys in the following list must/should be updated.
DOMAIN
– Hostname of the server hosting the CCM application.
For local development purposes with ngrok, use the hostname
copied during step 2.LTI_PLATFORM_URL
variable value is correct for the instance of Canvas used.LTI_CLIENT_ID
variable value the LTI key ID number copied earlier (in step 10).CANVAS_INSTANCE_URL
– For U-M development/testing, this should
be https://canvas-test.it.umich.edu
. Note that this is the URL
that would be used by the user to access Canvas. It may be
different from the value of the LTI_PLATFORM_URL
key.CANVAS_API_CLIENT_ID
variable value the API key copied earlier (in step 17).CANVAS_API_SECRET
variable value the secret copied earlier (in step 18).CANVAS_ADMIN_API_TOKEN
key the API token copied earlierINVITATION_API_KEY
INVITATION_API_SECRET
INVITATION_API_ENTITY_ID
INVITATION_API_SPONSOR_NAME
— Note that this is an email address,
which Cirrus uses to identify the sponsor's name.docker-compose build
docker-compose up
LTI_CLIENT_ID
in the .env
file above.Database migrations are managed using umzug
.
Umzug is a sister library to sequelize
for migration tasks.
After creating/modifying a model using the Sequelize ORM,
you create a migration file and run the migration using umzug
.
Developers have to write up
and down
migration steps manually.
Running migrations locally
docker exec -it ccm_web node --loader ts-node/esm server/src/migrator.ts up
docker exec -it ccm_web node --loader ts-node/esm server/src/migrator.ts down
.Create a migration file: docker exec -it ccm_web node --loader ts-node/esm server/src/migrator.ts create --name my-migration.ts
.
This generates a migration file called <timestamp>.my-migration.ts
.
The timestamp prefix can be customized to be date-only or omitted,
but be aware that it's strongly recommended to ensure your migrations are lexicographically sortable
so it's easy for humans and tools to determine what order they should run in
so the default prefix is recommended.
Running the migration are usually done when server is starting up, but in addition if you want to run migrations or revert use above commands
Running migrations docker-compose-prod.yml
docker exec -it ccm_web_prod node --loader ts-node/esm server/src/migrator.js up
and docker exec -it ccm_web_prod node --loader ts-node/esm server/src/migrator.js down
.ts-node/esm
module for running in node
environment.Error:
ccm_web | (node:64) UnhandledPromiseRejectionWarning: Error: An error occurred while setting up ltijs: Error: MISSING_PLATFORM_URL_OR_CLIENTID
ccm_web | at /base/server/src/lti/lti.middleware.ts:22:15
ccm_web | at processTicksAndRejections (internal/process/task_queues.js:93:5)
This happens when the LTI key is not configured or configured improperly. Solution: Follow the steps documented above to ensure the key is configured correctly, then start the application again.
Error:
ccm_db | 2021-05-05T19:46:18.445529Z 3 [Warning] InnoDB: Cannot open table ccm/platformStatuses from the internal data dictionary of InnoDB though the .frm file for the table exists. Please refer to http://dev.mysql.com/doc/refman/5.7/en/innodb-troubleshooting.html for how to resolve the issue.
ccm_web | Error during deployment: DatabaseError [SequelizeDatabaseError]: Table 'ccm.platformStatuses' doesn't exist
This may occur if the application has been run without being fully configured before. The DB may have been left in an invalid state.
When investigated, some DB clients show that table platformStatuses
is known, but when it is queried, an error like SQL Error [1146] [42S02]: Table 'ccm.platformStatuses' doesn't exist
is given.
Solution: Stop the Docker containers, delete the .mysql_data
directory, optionally delete the Docker images, then rebuild all again. Be sure the LTI key is properly configured before running the application again.
GitHub Pages is used for hosting the CCM feature documentation,
and files are placed and served from the docs
directory.
In order to test documentation changes locally, follow the steps below:
docs
directory.
cd docs
docker compose -f docker-compose-gh-pages.yml up`
http://locahost:4020
in your browser.
Recent changes to files will be automatically deployed; the changes will be displayed after a browser refresh.GitHub follows Jekyll structure for deploying. See here for more info.
To deploy the latest changes to the GitHub Pages site, follow the steps below:
docs
folder.https://{your_account}.github.io/canvas-course-manager-next
,
replacing {your_account}
with your GitHub user or organization name, to view the content.See here for more info about this process.
The backend application includes some unit and end-to-end tests built using Nest tooling as well as Jest and Supertest. To run the test suites, do the following:
Start up the application using the steps above, and then in a separate terminal window,
enter the container using docker exec
.
docker exec -it ccm_web /bin/bash
Run one of a handful of test commands defined in package.json
under scripts
.
# To run the unit tests and generate a coverage report
npm run test:cov
# To run the end-to-end test(s)
npm run test:e2e
The Nest framework comes with a command-line interface tool
for generating module and other code stubs and performing other tasks.
As the necessary libraries are not included in package.json
, follow the documentation's suggestion
and install @nestjs/cli
globally on your machine if you want to use the tool.
When in development mode (i.e. the NODE_ENV
environment variable's value is development
),
the application generates OpenAPI documentation and an explorer for
the API endpoints using a Nest package and
Swagger.
To access and use the Swagger UI, launch the application from a course in Canvas, then click on the "Swagger UI" link in the application interface's footer. This will take you to the Swagger page where you can view the documented API endpoints. Requests can also be made against the API using the "Try it out" functionality.
However, to execute PUT
and POST
requests using Swagger, you will need to provide a CSRF token
(GET
requests do not need this token).
The current CSRF token is currently made available as a URL parameter called csrfToken
,
which you can see and copy by using a browser tool to view the frame's source.
Once the token is obtained, in the Swagger UI, click the "Authorize" button, paste the token in the popup field,
and then click "Authorize" to complete the process.
Subsequent requests made using the Swagger UI to routes requiring the CSRF token
will include the value you provided in the proper header.
Taken together, all the stages in ccm_web/Dockerfile
will build an optimized image for production.
You can test the Docker production image and other production code branches by using docker-compose-prod.yml
.
To do so, issue the same commands as above under Development
with the -f
flag specifying docker-compose-prod.yml
.
The file uses the same .env
configuration file, so adjust any values there as needed
(see Configuration above for more info).
Note that, at minimum, the database host needs to be changed to ccm_db_prod
(since that is the name of the container).
The repository also includes a ccm_web/Dockerfile.openshift
file that is used
for OpenShift deployments. It combines the "base" and "build" stages of the other
Dockerfile
and pulls base images from the OpenShift namespace, which are not subject
to Docker Hub pull limits.
This repository contains modified portions of the npm package
@types/ltijs, which carries an MIT license.
See ccm_web/server/localTypes/ltijs/index.d.ts
for more information and the modified code.
This code will hopefully only remain in this repository temporarily.
main
branch. E.g., when a pull request is merged to main
.OpenShift projects can periodically pull this image from GHCR. Configure only NON-PRODUCTION CCM projects to pull the image…
oc tag ghcr.io/tl-its-umich-edu/canvas-course-manager-next:main canvas-course-manager-next:main --scheduled --reference-policy=local
See the OpenShift documentation "Managing image streams: Configuring periodic importing of image stream tags" for details.
reference-policy=local
: If you want to instruct OpenShift Container Platform to always fetch the tagged image from the integrated registry