fabriziosalmi / blacklists

Hourly updated domains blacklist 🚫
https://github.com/fabriziosalmi/blacklists/releases/download/latest/blacklist.txt
GNU General Public License v3.0
117 stars 5 forks source link

Bouncer #41

Closed fabriziosalmi closed 11 months ago

fabriziosalmi commented 1 year ago

Alright, if you're looking to build a bouncer for CrowdSec in Go, here's a more detailed outline and some Go snippets to get you started.

  1. Initialization:
    • On startup, your Go application should fetch the initial blacklist.
    • You should then load these domains into a Go map for O(1) lookup time.
package main

import (
    "io/ioutil"
    "net/http"
    "strings"
    "time"
)

var blacklistedDomains map[string]bool

func fetchBlacklist(url string) {
    resp, err := http.Get(url)
    if err != nil {
        // Handle error
    }
    defer resp.Body.Close()

    body, err := ioutil.ReadAll(resp.Body)
    if err != nil {
        // Handle error
    }

    domains := strings.Split(string(body), "\n")
    for _, domain := range domains {
        blacklistedDomains[domain] = true
    }
}
  1. Blocking Logic:
    • For each incoming request, check if the requested domain is blacklisted.
    • Block or alert based on the result.
func isBlacklisted(domain string) bool {
    return blacklistedDomains[domain]
}

func handleRequest(domain string) {
    if isBlacklisted(domain) {
        // Block or alert
    }
}
  1. Periodic Blacklist Update:
    • Use Go's time package to periodically fetch the updated blacklist.
func main() {
    blacklistedDomains = make(map[string]bool)
    fetchBlacklist("https://get.domainsblacklists.com/blacklist.txt")

    // Update every hour
    ticker := time.NewTicker(1 * time.Hour)
    quit := make(chan struct{})
    go func() {
        for {
            select {
            case <-ticker.C:
                fetchBlacklist("https://get.domainsblacklists.com/blacklist.txt")
            case <-quit:
                ticker.Stop()
                return
            }
        }
    }()
}
  1. Integration with CrowdSec:

    • This largely depends on how you plan to deploy this bouncer and what exactly you need to interact with in CrowdSec. Typically, you would make API calls to CrowdSec to fetch decisions and then use the domain blacklist in conjunction with these decisions to make block/allow decisions.
  2. Configuration & Customization:

    • Consider using a configuration file or environment variables to allow users to specify settings like the blacklist URL.
  3. Logging & Error Handling:

    • Go has great logging libraries. Consider using the standard log package or third-party packages like logrus for more advanced features.
  4. Compile and Run:

    • Once your Go bouncer is ready, you can compile it using go build and then deploy the resulting binary.

This is a high-level outline to get you started. A production-ready bouncer would involve more complexities such as error handling, performance optimizations, integration with other systems, and so forth.

Remember that Go is statically typed, and error handling is explicit, so make sure you handle all possible error cases, especially when making network calls or IO operations.