Twisk - Golang RPC starter kit

Twisk is a starter kit for Twirp - a framework for service-to-service communication emphasizing simplicity and minimalism.

It contains things you need to kickstart your project - structured packaging, logging, autogenerated swagger docs, handlers for login/refresh and user CRUD.

I made this project as a preparation for Gophercon 2018 Lightning Talk - Building robust APIs with Twirp.

Read more about Twirp on the official repo, release blog post and documentation website.

Twisk currently has:

• Fully featured RESTful endpoints for authentication and CRUD operations on the user entity
• Session handling using JWT claims
• JWT Based authentication
• Application configuration via config file (yaml)
• RBAC (role-based access control)
• Structured logging
• Request marshaling and data validation
• Autogenerated API Docs using SwaggerUI
• Mocking using stdlib
• Basic test coverage

The following dependencies are used in this project (generated using Glice):

|---------------------------------------|-----------------------------------------------|-------------|
|             DEPENDENCY                |                    REPOURL                    |   LICENSE   |
|---------------------------------------|-----------------------------------------------|-------------|
| github.com/golang/protobuf            | https://github.com/golang/protobuf            | bsd-3-clause|
| gopkg.in/yaml.v2                      | https://github.com/go-yaml/yaml               | Apache-2.0  |
| github.com/mwitkow/go-proto-validators| https://github.com/mwitkow/go-proto-validators| Apache-2.0  |
| github.com/twitchtv/twirp             | https://github.com/twitchtv/twirp             | Other       |
| github.com/gorilla/mux                | https://github.com/gorilla/mux                | bsd-3-clause|
| github.com/justinas/alice             | https://github.com/justinas/alice             | MIT         |
| github.com/go-pg/pg                   | https://github.com/go-pg/pg                   | bsd-2-clause|
| github.com/rs/xid                     | https://github.com/rs/xid                     | MIT         |
| github.com/nbutton23/zxcvbn-go        | https://github.com/nbutton23/zxcvbn-go        | MIT         |
| github.com/rakyll/statik              | https://github.com/rakyll/statik              | Apache-2.0  |
| github.com/dgrijalva/jwt-go           | https://github.com/dgrijalva/jwt-go           | MIT         |
| github.com/rs/zerolog                 | https://github.com/rs/zerolog                 | MIT         |
|---------------------------------------|-----------------------------------------------|-------------|

1. Protobuf - Proto formats (timestamp)
2. Yaml - Unmarshalling YAML config file
3. Twirp - RPC Framework
4. Mux - Router
5. Alice - Chaining middlewares
6. PG - PostgreSQL ORM
7. Statik - Static files to binary generator
8. JWT-GO - JWT Authentication
9. Zerolog - Logging
10. XID - Refresh token generation
11. ZXCVBN-Go - Password strength checker

Most of these can easily be replaced with your own choices since their usage is abstracted and localized. In addition to these, the following tools were used:

• Retool - Go executables vendoring
• GoValidators - Proto (request/response) validation
• TwirpSwagger - Swagger docs generator from proto definitions

Project Structure

Twisk's project structure mostly follows THIS example repository.

Twisk doesn't follow all best practices advices by Twirp. For example, functions are named List, Create and Delete instead of ListUsers, CreateUser and DeleteUser (I prefer shorter function names, handlers and functions named user.List istead of user.ListUsers). You can read on Twirp's best practices here.

Getting started

Using Twisk requires having Go 1.7 or above. Once you downloaded Twisk (either using Git or go get) you need to configure the following:

1. To use Twisk as a starting point of a real project whose package name is something like github.com/author/project, move the directory $GOPATH/github.com/ribice/twisk to $GOPATH/github.com/author/project and do a global replacement of the string github.com/ribice/twisk with github.com/author/project.

2. Change the configuration file according to your needs. The local config file is found in cmd/api/conf.local.yaml. You'll need to configure database/psn at least.

3. In cmd/migration/main.go set up psn variable and then run it (go run main.go). It will create all tables, and necessary data, with a new account username/password admin/admin.

4. Run the app using:

go run cmd/api/main.go

The application runs as an HTTP server at port 8080. It exposes the following endpoints:

• GET /openapi/swaggerui: returns list of swagger specs in browser
• POST /twirp/twisk.iam.IAM/Auth: accepts username or email and password. Returns jwt token and refresh token
• POST /twirp/twisk.iam.IAM/Refresh: refreshes sessions and returns new jwt token
• POST /twirp/twisk.user.User/Create: creates a new user
• POST /twirp/twisk.user.User/List: returns list of users
• POST /twirp/twisk.user.User/View: returns single user
• POST /twirp/twisk.user.User/Delete: deletes a user
• POST /twirp/twisk.user.User/Update: updates user's contact info

You can log in as admin to the application by sending a post request to localhost:8080/twirp/twisk.iam.IAM/Auth with auth admin and password admin in JSON body.

Implementing new APIs

1. Under proto folder, create a new one named after your service. For example tenant. Inside it create service.proto

2. Define your proto file. If you are not familiar with Protobufs, you can read more about it here. You can use already existing proto files (proto/user/service.proto & proto/iam/service.proto) as a template.

3. Run make twirp -B.

4. Implement the Tenant interface from service.twirp.go in internal/tenant.

5. Tenant service integrations are located in internal/tenant/platform. For example cloud storage, database, message queue etc.

6. Implement the interface in logging.go, wrapping around the service implementation.

7. Add service link to openapi/swagger.html. Alternatively, use build-links in your CI/CD pipeline to auto-generate the swagger.html from swagger.html.template.

8. Wire up everything in cmd/api/main.go. If neeeded, you can easily skip auth checking on certain routes by passing the service name to hooks.WithJWTAuth(). For example hooks.WithJWTAuth(j, "View","List").

License

Twisk is licensed under the MIT license. Check the LICENSE file for details.

Honorable mention

Special thanks to @tonto for previously creating some parts of this project.

Author

Emir Ribic