Document the minimum viable configuration for running Bonsai in local development mode

calebhailey commented 4 years ago

In the .env.example config file there are quite a few external dependencies (e.g. GitHub secrets, SendGrid, AirBrake, Google Tag Manager, AWS S3, an Oath2 provider, PubSubHubbub?, and Fieri?). Which of these are required for local development?

pzupan commented 4 years ago

The environmental variables are contained in the .env file, and that file is annotated with which variables are required to run the application. Generally they are grouped under a comment, which identifies what the variables are used for, and that comment will have Required next to it to indicate that those variables are the minimum required .

calebhailey commented 4 years ago

Are Postgres and Redis not required for local development environments? The readme cites them as requirements, but the .env.example doesn't indicate that redis/postgres configuration is required.

Also, there are three configuration settings which appear to have no heading:

FEATURES=tools,hosted_extensions – is this one somehow related to the PubSub variables?
ROBOTS_ALLOW=/
ENFORCE_PRIVACY=true

Finally, should the non-required configuration settings be commented out in the .env.example file? Will the app run with invalid settings uncommented?

calebhailey commented 4 years ago

I'm also curious what the minimum viable IAM policy is for the AWS S3 configuration settings. We should document this with example IAM policy JSON in the README.

calebhailey commented 4 years ago

And what about the the GitHub credentials?

I presume GITHUB_CLIENT_ID and GITHUB_CLIENT_SECRET are GitHub OAuth Application credentials (so contributors will need to create a GitHub OAuth App, which requires a callback URL... which is presumably http://localhost:3000/ - but what's the route?).

But what is the GITHUB_ACCESS_TOKEN? Is it a GitHub Personal Access Token? If so, what permissions should that token have?

mbiang commented 4 years ago

[x] Group environment variables by type, starting with top level app configs (example to be provided by @calebhailey)
[x] Comment out non required environment variables.
[x] Trace GITHUB_ACCESS_TOKEN and determine if it is being used.
[x] Comment out POSTGRES_PASSWORD env var
[x] Document Github Oauth app setup and required config values (Homepage URL, Callback URL)
[x] Document and provide an example of the s3 bucket IAM permissions
[x] Document the different s3 buckets that are required

calebhailey commented 4 years ago

A reordering of the example config might be helpful as well. Here's how I have arranged mine, as discussed on our call this afternoon:

# application variables ** REQUIRED **
PROTOCOL=http
HOST=localhost
APP_PORT=3000
HOST_ORGANIZATION=sensu
HOST_EMAIL=sensu@example.com
HOST_LOGO=sensu.png # logo file should be in app/assets/images
HOST_PREFERRED_OWNER_NAMES="'sensu','sensu-plugins'" # ranked higher in search results
ROBOTS_ALLOW=/
ENFORCE_PRIVACY=true
FEATURES=tools,hosted_extensions
RAILS_MAX_THREADS=5
SECRET_KEY_BASE=YOUR_SECRET_KEY_BASE

# SendGrid specific ** REQUIRED **
SENDGRID_API_KEY=YOUR_SENDGRID_API_KEY
FROM_EMAIL=sensu@example.com
MAILER_DOMAIN=example.com

# Amazon S3 variables ** REQUIRED **
AWS_S3_KEY_ID=YOUR_AWS_S3_KEY_ID
AWS_S3_ACCESS_KEY=YOUR_AWS_S3_ACCESS_KEY
AWS_S3_REGION=YOUR_AWS_S3_REGION
AWS_S3_BUCKET=bonsai.example.com
AWS_S3_ASSETS_BUCKET=assets.bonsai.example.com

# Replaces [bucket].s3.amazonaws.com/ or [region].amazonaws.com/[bucket]
# Example: https://us.west.2.amazonaws.com/staging.assets.bonsai.sensu.io/73196.../...tar.gz
# Becomes: https://staging.assets.bonsai.sensu.io.s3.amazonaws.com/73196.../...tar.gz
# See https://docs.aws.amazon.com/AmazonS3/latest/dev/VirtualHosting.html#VirtualHostingCustomURLs
# AWS_S3_VANITY_HOST=YOUR_AWS_S3_VANITY_HOST

# github variables ** REQUIRED **
GITHUB_CLIENT_ID=YOUR_GITHUB_CLIENT_ID
GITHUB_CLIENT_SECRET=YOUR_GITHUB_CLIENT_SECRET
GITHUB_ACCESS_TOKEN=YOUR_GITHUB_ACCESS_TOKEN

# oauth variables ** REQUIRED **
OAUTH_ACCOUNT_PROVIDER=github
OAUTH2_VERIFY_SSL=false

# postgres specific variables
POSTGRES_MAX_THREADS=5
# POSTGRES_PASSWORD=YOUR_BONSAI_DATABASE_PASSWORD

# background jobs database
REDIS_URL=redis://redis:6379

# error reporting
# AIRBRAKE_PROJECT_ID=YOUR_AIRBRAKE_ID
# AIRBRAKE_API_KEY=YOUR_AIRBRAKE_KEY

# Google Tag variables
# GOOGLE_TAG_MANAGER_ID=YOUR_TAG_ID
# GOOGLE_ANALYTICS_ID=YOUR_ANALYTICS_ID

# optional CLA variables
# CLA_SIGNATURE_NOTIFICATION_EMAIL=some-email@example.com
# CLA_REPORT_EMAIL=some-email@example.com
# CCLA_VERSION=99999-2621/LEGAL14767024.1
# ICLA_VERSION=99999-2621/LEGAL14767024.1

# optional PubSub variables
# PUBSUBHUBBUB_SECRET=YOUR_PUBSUBHUBBUB_SECRET
# PUBSUBHUBBUB_CALLBACK_URL=http://example.com

# optional Fieri variables
# FIERI_URL=http://example.com
# FIERI_KEY=YOUR_FIERI_KEY

calebhailey commented 4 years ago

@mbiang @pzupan while you're at it, I can't seem to get the app to use the APP_PORT setting in my env file. If I shell into the container I can see that it's getting set, but the app still runs on port 3000 no matter what.

macbook $ docker-compose up -d
Creating network "bonsai_default" with the default driver
Creating bonsai_db_1    ... done
Creating bonsai_redis_1 ... done
Creating bonsai_web_1   ... done
Creating bonsai_sidekiq_1 ... done
macbook $ docker exec -it bonsai_web_1 /bin/bash
root@3737c6660ed8:/bonsai-asset-index# echo $APP_PORT
2674
root@3737c6660ed8:/bonsai-asset-index#

NOTE: I have added the appropriate port mapping to the docker-compose to support my custom port, and it's still not working; plus, I can verify in the app startup log that it's running on port 3000.

sensu / bonsai

Document the minimum viable configuration for running Bonsai in local development mode #267