Octopub is a Ruby on Rails application that provides a simple and frictionless way for users to publish data easily, quickly and correctly on GitHub.
More information is in the announcement blog post
The live instance of Octopub is running at http://octopub.io/
Follow the public feature roadmap for Octopub
These are the tools and services required to get Octopub fully working for development, testing and production environments. We'll explain how to set these up in the next section.
Sidekiq is used for managing the background proccessing of data uploads. To use Sidekiq just install Redis by following the instructions here, or if you are using Homebrew you can just do brew install redis
.
For development Octopub uses the dotenv gem to load environment variables. Create a file called .env
in your project root and paste in the variables below. We'll fill these in as we go along.
# GitHub App Client ID & secret
GITHUB_KEY=
GITHUB_SECRET=
# OAuth access token for GitHub API access
GITHUB_TOKEN=
S3_BUCKET=
AWS_ACCESS_KEY_ID=
AWS_SECRET_ACCESS_KEY=
PUSHER_APP_ID=
PUSHER_KEY=
PUSHER_SECRET=
PUSHER_CLUSTER=
BASE_URI=
ODC_API_KEY=
ODC_USERNAME=
# production only
SMTP_USERNAME=
SMTP_PASSWORD=
SMTP_SERVER=
Create a GitHub application:
Settings -> Developer settings -> OAuth applications
, create a new OAuth application with a unique name. You can use http://octopub.io
for the homepage and for the callback URL use your local server address, i.e. http://localhost:3000
. Click on your OAuth application to see your Client ID
and Client Secret
, and update your .env
file:GITHUB_KEY=<YOUR CLIENT ID>
GITHUB_SECRET=<YOUR CLIENT SECRET>
Settings -> Developer settings -> Personal access tokens
, generate a new token with a sensible description, e.g. octopub_dev_token, and update your .env
file:GITHUB_TOKEN=<Your token>
Create an S3 bucket:
In AWS go to the S3 service and create a bucket with a sensible name. Make sure the region is set to EU(Ireland) since Octopub uses this.
Click on your bucket and go to the Permissions
tab. Click on CORS Configuration
and paste in the configuration below. This will allow your local development version of Octopub to make requests to your S3 bucket.
<?xml version="1.0" encoding="UTF-8"?>
<CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<CORSRule>
<AllowedOrigin>http://localhost:3000</AllowedOrigin>
<AllowedMethod>GET</AllowedMethod>
<AllowedMethod>POST</AllowedMethod>
<AllowedMethod>PUT</AllowedMethod>
<AllowedHeader>*</AllowedHeader>
</CORSRule>
</CORSConfiguration>
Grant permissions to your bucket:
In AWS go to the IAM (Identity and Access Management page) service.
Click Users
.
Add a new user and give it a name, e.g. octopub-development, and for Access Type select Programmatic Access
.
For permissions, select Attach existing policies directly
- this will open a new tab in your browser.
Click create your own policy
and give it a name, e.g. octopub-dev-permissions. Then for the policy document, use the following template, but add your bucket name in place of <BUCKETNAME>
.
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AllowAdminAccessToBucketOnly",
"Action": [
"s3:*"
],
"Effect": "Allow",
"Resource": [
"arn:aws:s3:::<YOUR BUCKET NAME>",
"arn:aws:s3:::<YOUR BUCKET NAME>/*"
]
}
]
}
Click validate policy
just to be sure you've not made a typo, then confirm.
Back on the Set permissions page
, select the policy you've just created in the table by selecting the checkbox, then click Review
and then click Create user
.
Download the CSV file containing your Access key ID
and Secret access key
and update your .env
file:
AWS_ACCESS_KEY_ID=<YOUR ACCESS KEY ID>
AWS_SECRET_ACCESS_KEY=<YOUR SECRET ACCESS KEY>
S3_BUCKET=<YOUR BUCKET NAME>
App Keys
tab and use the relevant values there to update your .env
file:PUSHER_APP_ID=
PUSHER_KEY=
PUSHER_SECRET=
PUSHER_CLUSTER=
.env
file:ODC_API_KEY=<API TOKEN>
ODC_USERNAME=<YOUR USERNAME (email address you used when signing up)>
Assuming you have completed the setup instructions above...
redis-server
.bundle exec sidekiq
in the application directory.config/database.yml
and run rails db:migrate
.rails s
in the application directory. Congratulations, you should be signed in! Now try adding some data.
Start a rails console session and then...
require 'sidekiq/api'
Sidekiq::Queue.new.size
Sidekiq::Queue.new.first
Octopub uses the rspec
test framework and requires the presence of a .env
. See earlier section for details as you can (re)use your development variables*
The test suite can be run with bundle exec rspec
.
* Note - the tests use VCR or mocking to allow the tests to be run offline without interfacing with the services.
A commit to master will trigger a TravisCI run; If successful it will automatically deploy to Heroku.
The GitHub organisations are cached for the logged in user. They can be cleared from a console with Rails.cache.clear