My RESTful API Boilerplate

Image Copyright © SAWARATSUKI. All rights reserved.
Image used under permission from the copyright holder.

This project provides a boilerplate for building RESTful APIs using Go. It is designed to a quick start with best practices, easy configuration, and a clear project structure.

[!WARNING]
This boilerplate is specifically tailored for extensive codebases and scalable RESTful API applications, along with frontend websites that may span multiple sites. Its design ensures scalability and straightforward maintenance, making it highly suitable for complex projects. However, it may not be the ideal choice for smaller-scale RESTful API applications or single-website frontends where such robust architecture is not required.

Features

• Use Fiber Framework
• Pre-configured MySQL and Redis integration
• Middleware examples for logging and authentication
• Scalable project structure
• Dual-licensed under BSD 3-Clause and MIT (specific files)
• High Performance (Thanks to the Sonic JSON serializing & deserializing library)
• Cryptography Techniques
• High Quality Go Codes
• Rich Presence TUIs using charm.sh
• Boring TLS 1.3 Protocol (WIP)
• Certificate Transparency (CT) Log

[!NOTE]
Boring TLS 1.3 Protocol This project utilizes the industry-standard TLS 1.3 protocol, known for its exceptional security and reliability. We call it Boring TLS because its robust security makes it almost boring.

[!NOTE] Certificate Transparency (CT) Log This project includes built-in support for submitting certificates to Certificate Transparency logs. It supports certificates with both RSA and ECDSA keys. By submitting certificates to CT logs, it enhances the security and transparency of the TLS ecosystem, allowing domain owners to monitor and detect fraudulent certificates or other bad certificates. The CT log functionality is implemented using the SubmitToCTLog function, which takes the certificate, private key, and CT log details as input. It encodes the certificate, calculates the hash, creates a JSON payload, sends an HTTP request to the CT log server, and verifies the signed certificate timestamp (SCT) received in the response. The VerifySCT method is responsible for verifying the SCT based on the public key type (RSA or ECDSA) and ensures the integrity of the timestamp. With this feature, it is easy to integrate certificate transparency into the TLS setup and contribute to a more secure web.

[!WARNING] Some features might not work as expected when running this repo on certain cloud providers. For example, the Rich Presence TUIs feature requires a tty. The reason why some features might not work as expected after implementation is because this repo is designed to be top-level and follow the Unix philosophy. Plus, most of the Go code in this repo follows many best practices and idioms from Effective Go.

TODO Features

Move here

[!NOTE]
Additional features will be added later as my primary focus is on using this with the Fiber framework and Cryptography Techniques.

Supported Additional Features

[!NOTE]
The following list includes additional features that I've been using before for extremely scalable and high-performance applications:

• Protocol Buffers
• Templ

Motivation

The motivation for sharing this RESTful API Boilerplate on GitHub is to streamline my development process, ensuring I don't have to start from scratch each time. It's a robust, secure foundation that adheres to best practices, designed for quick and efficient project initiation.

Philosophy

This boilerplate is grounded in the Unix philosophy, emphasizing simplicity, modularity, and reusability. Each component is designed to do one thing and do it well, with the ability to combine components seamlessly to perform complex tasks. This approach ensures that the boilerplate remains lightweight, efficient, and easy to maintain.

Resource Memory Usage

• MySQL without Redis + 100K POST Request (Outdated - go1.22.2)

[!NOTE] The Resource Memory Usage section demonstrates how Go has stable and low memory overhead compared to other languages, especially Java (See this article for more information on Java memory management Lmao.) hahaha.

• Idle (Outdated - go1.22.2)

[!NOTE] The Idle section demonstrates the memory usage when there is no request. The maximum average memory usage is around 21.5MB. Go routines (100 goroutines) along with a semaphore are used to automatically handle high traffic situations (e.g., lots of requests).

Also Note that even with high incoming traffic (e.g., 1 million requests), the maximum average memory usage is still relatively low, around 100MB (e.g., 50MB), due to the use of the semaphore.

Latest

• Idle (Outdated - go1.22.3)

sample#memory_total=8.01MB
sample#memory_rss=7.92MB
sample#memory_cache=0.09MB
sample#memory_swap=0.00MB
sample#memory_pgpgin=0pages
sample#memory_pgpgout=0pages
sample#memory_quota=1024.00MB

[!NOTE] Go Memory Usage Improvements:

• The Idle memory usage in the latest version of Go has been optimized, reducing it to approximately 7.92MB compared to the previous version.

Average Memory Usage Breakdown:

• The average memory usage of 11.1MB includes:
  • Go routine scheduler for database operations (separate from the 100 goroutines used for handling high incoming traffic automatically with a semaphore)
  • Load for the front-end website
  • Load for the REST APIs (Full Stack)
• The application remains stable without any performance issues, such as bottlenecks or memory leaks, and has zero vulnerabilities.

Go Performance Comparison:

• Compared to other languages, particularly for full-stack development, Go demonstrates superior performance.

TLS 1.3 Connection Stable

• Standard TLS 1.3 (Latest Go Version - go1.22.5)

[!NOTE] The screenshots provided demonstrate a Standard TLS 1.3 connection using the Closed networks or intranets method, which involves Cloudflare (Frontend) and Heroku (Backend). Without Cloudflare, any browser or tools like curl won't be able to access the backend server.

Frontend Performance

• Error Page (Include Wildcard Handler)

[!NOTE] The screenshot provided demonstrates the performance of an Error Page, and it can easily be optimized to achieve all green metrics.

[!TIP] Also note that when optimized to achieve all green metrics which is easily, especially in the SEO category, it can be beneficial for business logic purposes. (e.g, Search engines like Google and Microsoft Bing tend to favor websites with good performance metrics, which can lead to improved search rankings and increased visibility.)

Prometheus

• Latest Go Version

[!NOTE] The grafana dashboard is not shareable due to it being bound to my security configurations for real-world monitoring in other production environments.

Architecture (Tree)

Below is the architecture of this boilerplate and how it looks. I created this for REST APIs about volcano 🌋 monitoring used by the government (has been done before), so it can easily monitor volcanoes in the real world.

• Backend (Pure Go)

backend/
|-- cmd/
|   `-- server/ (main application entry point)
|       `-- main.go
|-- pkg/
|   |-- restapis/ (API route handlers)
|   |-- any/ (any related code)
|-- internal/ (private application and library code)
|   |-- any/ (any related code)
|-- .env (optional environment variables since it can placed anywhere)
`-- go.mod (dependencies)

• Frontend (Any Framework TS e.g, React,NextJS or pure JS for HTMX)

Example:

frontend/
|-- pages/
|   |-- api/ (optional, for Next.js API routes if needed)
|   |-- _app.js (global page layouts and state)
|   |-- index.js (home page)
|   `-- [...other pages]
|-- public/ (static files like images, fonts)
|-- src/
|   |-- components/ (shared React components)
|   |   `-- [various components]
|   |-- styles/ (global styles, theme)
|   |-- hooks/ (custom React hooks)
|   |-- utils/ (utility functions)
|   |-- lib/ (libraries and configurations)
|   `-- context/ (React context files for state management)
|-- .env.local (environment variables)
|-- next.config.js (Next.js configuration)
`-- package.json (dependencies and scripts)

Current Boilerplate Tree:

├── Dockerfile
├── LICENSE
├── README.md
├── SECURITY.md
├── backend
│   ├── cmd
│   │   └── server
│   │       ├── run.go
│   │       └── run_heroku.go
│   ├── internal
│   │   ├── database
│   │   │   ├── auth.go
│   │   │   ├── cloudflare-kv
│   │   │   │   └── setup.go
│   │   │   ├── constant.go
│   │   │   ├── helper.go
│   │   │   ├── mysql_redis.go
│   │   │   ├── setup.go
│   │   │   └── tls.go
│   │   ├── logger
│   │   │   ├── constant.go
│   │   │   └── logger.go
│   │   ├── middleware
│   │   │   ├── authentication
│   │   │   │   ├── crypto
│   │   │   │   │   ├── bcrypt
│   │   │   │   │   │   ├── bcrypt.go
│   │   │   │   │   │   ├── bcrypt_test.go
│   │   │   │   │   │   ├── compare_password.go
│   │   │   │   │   │   ├── docs.go
│   │   │   │   │   │   └── hash_password.go
│   │   │   │   │   ├── cipher.go
│   │   │   │   │   ├── crypto.go
│   │   │   │   │   ├── crypto_test.go
│   │   │   │   │   ├── deadcode.go
│   │   │   │   │   ├── decrypt.go
│   │   │   │   │   ├── encrypt.go
│   │   │   │   │   ├── gopherpocket
│   │   │   │   │   │   └── keyrotation
│   │   │   │   │   │       ├── docs.go
│   │   │   │   │   │       ├── gopherkey.go
│   │   │   │   │   │       └── gopherkey_test.go
│   │   │   │   │   ├── helper.go
│   │   │   │   │   ├── hybrid
│   │   │   │   │   │   ├── decryptcookie.go
│   │   │   │   │   │   ├── encryptcookie.go
│   │   │   │   │   │   ├── hybrid.go
│   │   │   │   │   │   ├── hybrid_stream.go
│   │   │   │   │   │   ├── hybrid_test.go
│   │   │   │   │   │   └── stream
│   │   │   │   │   │       ├── benchmark_test.go
│   │   │   │   │   │       ├── chunk.go
│   │   │   │   │   │       ├── decrypt_stream.go
│   │   │   │   │   │       ├── digest.go
│   │   │   │   │   │       ├── docs.go
│   │   │   │   │   │       ├── encrypt_stream.go
│   │   │   │   │   │       ├── new_stream.go
│   │   │   │   │   │       ├── nonce.go
│   │   │   │   │   │       └── stream_test.go
│   │   │   │   │   ├── keyidentifier
│   │   │   │   │   │   ├── docs.go
│   │   │   │   │   │   ├── ecdsa_sign.go
│   │   │   │   │   │   ├── generate.go
│   │   │   │   │   │   ├── keyidentifier_test.go
│   │   │   │   │   │   └── new.go
│   │   │   │   │   ├── rand
│   │   │   │   │   │   ├── fixed_size.go
│   │   │   │   │   │   └── rand_test.go
│   │   │   │   │   ├── vault
│   │   │   │   │   │   ├── new.go
│   │   │   │   │   │   └── transit.go
│   │   │   │   │   └── web3
│   │   │   │   │       └── eth
│   │   │   │   │           ├── docs.go
│   │   │   │   │           └── new.go
│   │   │   │   ├── helper
│   │   │   │   │   ├── constant.go
│   │   │   │   │   └── keyauth.go
│   │   │   │   └── keyauth
│   │   │   │       ├── constant.go
│   │   │   │       ├── error.go
│   │   │   │       ├── success.go
│   │   │   │       └── validator.go
│   │   │   ├── constant.go
│   │   │   ├── csp
│   │   │   │   ├── config.go
│   │   │   │   ├── csp_test.go
│   │   │   │   └── new.go
│   │   │   ├── custom_next.go
│   │   │   ├── filesystem
│   │   │   │   └── crypto
│   │   │   │       └── signature
│   │   │   │           ├── hmac_sign.go
│   │   │   │           ├── hmac_test.go
│   │   │   │           └── hmac_verify.go
│   │   │   ├── frontend_routes.go
│   │   │   ├── helper.go
│   │   │   ├── monitor
│   │   │   │   ├── docs.go
│   │   │   │   └── prometheus.go
│   │   │   ├── restapis_routes.go
│   │   │   ├── restime
│   │   │   │   ├── config.go
│   │   │   │   └── new.go
│   │   │   ├── router
│   │   │   │   └── domain
│   │   │   │       ├── config.go
│   │   │   │       └── new.go
│   │   │   ├── routes.go
│   │   │   ├── storage.go
│   │   │   └── utils.go
│   │   ├── server
│   │   │   ├── boringtls.go
│   │   │   ├── boringtls_cert.go
│   │   │   ├── boringtls_cert_test.go
│   │   │   ├── boringtls_test.go
│   │   │   ├── constant.go
│   │   │   ├── ct_verifier.go
│   │   │   ├── ct_verifier_test.go
│   │   │   ├── helper.go
│   │   │   ├── helper_tls_test.go
│   │   │   ├── k8s
│   │   │   │   ├── docs.go
│   │   │   │   └── metrics
│   │   │   │       └── prometheus.go
│   │   │   ├── mount_routes.go
│   │   │   ├── register_routes.go
│   │   │   └── startup_async.go
│   │   └── translate
│   │       └── language.go
│   └── pkg
│       ├── gc
│       │   ├── docs.go
│       │   ├── reduce_http_client_overhead.go
│       │   └── reduce_overhead.go
│       ├── header
│       │   └── htmx
│       │       ├── constant.go
│       │       └── docs.go
│       ├── mime
│       │   ├── ascii.go
│       │   ├── docs.go
│       │   └── mime.go
│       └── restapis
│           ├── helper
│           │   ├── auth
│           │   │   ├── apikey.go
│           │   │   └── constant.go
│           │   ├── generate_apikey.go
│           │   ├── generate_apikey_test.go
│           │   ├── json
│           │   │   └── sonic
│           │   │       ├── config.go
│           │   │       └── docs.go
│           │   ├── numeric.go
│           │   ├── numeric_test.go
│           │   ├── restapis_error.go
│           │   └── restapis_error_test.go
│           └── server
│               └── health
│                   ├── cache.go
│                   ├── constant.go
│                   ├── db.go
│                   ├── helper.go
│                   ├── mysql.go
│                   └── redis.go
├── env
│   ├── docs.go
│   ├── env.go
│   └── getenv.go
├── frontend
│   ├── htmx
│   │   ├── error_page_handler
│   │   │   ├── 400.templ
│   │   │   ├── 400_templ.go
│   │   │   ├── 401.templ
│   │   │   ├── 401_templ.go
│   │   │   ├── 403.templ
│   │   │   ├── 403_templ.go
│   │   │   ├── 404.templ
│   │   │   ├── 404_templ.go
│   │   │   ├── 500.templ
│   │   │   ├── 500_templ.go
│   │   │   ├── 503.templ
│   │   │   ├── 503_templ.go
│   │   │   ├── base.templ
│   │   │   ├── base_templ.go
│   │   │   ├── page_handler.go
│   │   │   ├── render_frontend.go
│   │   │   └── static_handler.go
│   │   ├── public
│   │   │   └── assets
│   │   │       └── css
│   │   │           └── base-tailwind.css
│   │   └── site
│   │       ├── footer.templ
│   │       ├── footer_templ.go
│   │       ├── head.templ
│   │       ├── head_templ.go
│   │       ├── header.templ
│   │       ├── header_templ.go
│   │       ├── script.templ
│   │       └── script_templ.go
│   └── public
│       ├── assets
│       │   ├── css
│       │   │   ├── base-tailwind.css
│       │   │   └── raw.css
│       │   ├── images
│       │   │   ├── android-chrome-192x192.png
│       │   │   ├── android-chrome-512x512.png
│       │   │   ├── apple-touch-icon.png
│       │   │   ├── browserconfig.xml
│       │   │   ├── favicon-16x16.png
│       │   │   ├── favicon-32x32.png
│       │   │   ├── favicon.ico
│       │   │   ├── http_error_codes
│       │   │   │   ├── 403-Forbidden.png
│       │   │   │   ├── 404-NotFound.png
│       │   │   │   ├── 500-InternalServerError.png
│       │   │   │   └── 503-ServiceUnavailable.png
│       │   │   ├── logo
│       │   │   │   └── gopher-run.png
│       │   │   ├── mstile-150x150.png
│       │   │   ├── safari-pinned-tab.svg
│       │   │   └── site.webmanifest
│       │   └── js
│       │       ├── htmx.indicator.min.js
│       │       ├── htmx.min.js
│       │       └── tailwind.min.dark.js
│       └── magic_embedded.go
├── go.mod
├── go.sum
├── k8s-deployment
│   ├── MySQL.md
│   ├── README.md
│   ├── REDIS.md
│   ├── RESTAPIs.md
│   ├── create_k8s_secret.sh
│   ├── ingress-nginx-configmap.yaml
│   ├── mysql-deploy-cpu-boost.yaml
│   ├── mysql-deploy.yaml
│   ├── prometheus_portable.yaml
│   ├── prometheus_portable_rules_record.yaml
│   ├── redis-insight.yaml
│   ├── restapis-deploy.yaml
│   └── restapis-ingress.yaml
├── tailwind.config.js
├── translate.json
└── worker
    ├── config.go
    ├── do_work.go
    ├── docs.go
    ├── jobs.go
    └── worker_test.go

63 directories, 192 files

[!NOTE] The Current Boilerplate Tree is designed as a modular framework, and it is easily maintainable even if it reaches 1K files. Personally, I've been maintaining over 1k files as well from this boilerplate, and it runs smoothly on Kubernetes (K8s) ⛵ ☸.

Git Mirror (Auto Synced)

Server Location: Singapore (Stable and very low latency for Southeast Asian Regions, ranging from 10ms ~ 50ms)

• My REST APIs Boilerplate

💻 >_ Shell:

git clone https://git.b0zal.io/H0llyW00dzZ/My-RESTAPIs-Boilerplate.git

[!NOTE]

Server Management and Isolation by Kubernetes:

• The Storage is secure and fully encrypted (end-to-end), designed with flexibility in mind. It is suitable for automated attach/detach processes within a cluster.
• The Network utilizes a network load balancer controlled by Ingress Nginx, optimizing latency for the APAC region, ensuring smooth sailing ⛵ ☸.
• The Git SSH should work as it uses the TCP Service Nginx.

License

This project is dual-licensed under the BSD 3-Clause License and the MIT License - see the LICENSE file for details.