A weekly GitHub newsletter tailored for you. Discover what your peers are doing on GitHub.
Discover monitors the people you follow on GitHub and sends you a weekly newsletter with their activity. Discover projects your friends are working on, new repositories they recently discovered, and issues or releases you might have missed.
It consists of three parts.
Crawler
The crawler is the part that monitors GitHub activity. It constantly crawls the activity of the people you follow and feeds our Graph DB. Based on that, then, we are able to export data based on your connections.
API
The API is serving our website.
Extraction (Better name pending?!)
Extraction is the part that queries our GraphDB, prepares the email template and sends it out as a weekly newsletter.
This project is using:
Consult the table below and export your preferences as environment variables.
export GITHUB_TOKEN=...
make run-crawler
make run-extraction
make run-api
Available env vars:
Variable | Description | Required | Default | |
---|---|---|---|---|
GITHUB_TOKEN |
GitHub token for the crawler | yes | ||
LOG_LEVEL |
Log level: error , info , debug , trace |
no | info |
|
QUEUE_STORE_DIR |
path for dqueue persistence; defaults to ~/go-discover |
|||
SUGGESTION_STORE_TYPE |
no | sqlite3 | ||
SUGGESTION_STORE_DSN |
no | ./local/suggestions.db | ||
NEO4J_HOST |
no | http://localhost:7474/db/data | ||
REDIS_HOST |
no | localhost:6379 | ||
API_BIND_ADDRESS |
no | 0.0.0.0:8080 | ||
GITHUB_CLIENT_SECRET |
GitHub OAuth secret | yes | ||
GITHUB_CLIENT_ID |
GitHub OAuth ID | yes | ||
GITHUB_CALLBACK_URL |
GitHub OAuth callback URL | no | http://localhost:8080/github/callback | |
MAILGUN_DOMAIN |
yes | |||
MAILGUN_APIKEY |
yes | |||
MAIL_SENDER_ADDRESS |
yes | |||
LOCK_USER_DURATION |
no | 12h | ||
LOCK_REPOSITORY_DURATION |
no | 24h |
make tools
make deps
make lint
make test
Available make targets:
make deps
- installs dependenciesmake tools
- installs required tools under ./bin
make lint
- lints package using ./bin/golangci-lint
make build-api
- builds cmd/api
as ./bin/api
make build-crawler
- builds cmd/crawler
as ./bin/crawler
make build-extraction
- builds cmd/extraction
as ./bin/extraction
make run-api
- builds and runs cmd/api
make run-crawler
- builds and runs cmd/crawler
make run-extraction
- builds and runs cmd/extraction
make test
- tests packagemake clean
- removes temp filesWant to contribute to this project?
We absolutely welcome contributions to this project. Take a look at our open issues and submit a PR if you see one you can help with.
Feel free to get in touch with us to discuss more about this project and how you can get involved. We love hearing from people!
Name | |
---|---|
Kostas Bariotis | @kbariotis |
George Antoniadis | @geoah |