consider using ja3 to block bots

[komuw]

https://github.com/salesforce/ja3

Do note that even though salesforce/ja3 is released under a BSD 3-Clause license, the ja3 fingerprinting is patented by salesforce: https://patents.google.com/patent/US11128606B2

The ja3 repo says: JA3 is a .. TLS fingerprinting .. inspired by research of Brotherston & his TLS Fingerprinting tool: FingerprinTLS. Looking at the linked research and tool , I'm surprised salesforce got the patent since there is prior art. Anyhow, I'm not a lawyer and have no clue how the patent system works.

• check to see if https://github.com/golang/go/blob/go1.20/src/crypto/tls/handshake_test.go#L27-L42 has TLS test vectors we can use.

[komuw]

Also see: https://github.com/komuw/ong/issues/124 (consider using hassh to block bots)

[komuw]

CircleCI recently got hacked, the entrypoint was[1]: "Our investigation indicates that the malware was able to execute session cookie theft, enabling them to impersonate the targeted employee in a remote location"

Loom also had an incident[4]. They said; "Some our users had their information exposed to other user accounts. A configuration change to our Content Delivery Network (CDN) caused incorrect session cookies to be sent back to our users."

I think the issue could have been mitigated by storing the client's IP address in the session cookie, and then failing any requests that have mismatched IP addresses. However, client IPs are easily spoofed[2]. So instead of storing IP address, how about we store the ja3 hash instead?

This still does not resolve the issue completely. This is because, whereas the attacker now cannot use their own machine/s to carry out the attack, they can still do so by proxying though the employees machine[3]

1. https://circleci.com/blog/jan-4-2023-incident-report/
2. https://adam-p.ca/blog/2022/03/x-forwarded-for/
3. https://news.ycombinator.com/item?id=34388099
4. https://www.loom.com/blog/march-7-incident-update

[komuw]

Amazon cloudfront now has ja3 support: https://aws.amazon.com/about-aws/whats-new/2022/11/amazon-cloudfront-supports-ja3-fingerprint-headers/

A Cloudfront-viewer-ja3-fingerprint header contains a 32-character hash fingerprint of the TLS Client Hello
packet of an incoming viewer request. The fingerprint encapsulates information about how the 
client communicates and can be used to profile clients that share the same pattern. 

As does clouflare: https://developers.cloudflare.com/bots/concepts/ja3-fingerprint/

[komuw]

repo with some know ja3 fingerprints including bots; https://github.com/LeargasSecurity/ja3-fingerprint-repository

[komuw]

https://pkg.go.dev/crypto/tls#ConnectionState

https://pkg.go.dev/crypto/tls#ClientHelloInfo

[komuw]

"JA3 gathers the decimal values of the bytes for the following fields in the Client Hello packet; SSL Version, Accepted Ciphers, List of Extensions, Elliptic Curves, and Elliptic Curve Formats. It then concatenates those values together in order, using a "," to delimit each field and a "-" to delimit each value in each field.

The field order is as follows:

SSLVersion,Cipher,SSLExtension,EllipticCurve,EllipticCurvePointFormat"

https://github.com/salesforce/ja3

[komuw]

https://github.com/sleeyax/ja3rp/blob/c6e640ae57ea705b992b61752ef264eae32e510d/crypto/tls/common.go#L462-L514

[komuw]

Caddy seems to have implemented something similar to what we want; https://github.com/caddyserver/caddy/pull/1430

This was pointed out by Dan Peterson[1] on gophers slack.

1. https://gophers.slack.com/archives/C0M8BLZAN/p1673793286287319

[komuw]

We should also consider whether we can use tls.Config.GetCertificate https://github.com/komuw/ong/blob/fd94ed712d9baa5b42d5ff16f1fe561337491328/server/tls_conf.go#L79

Dan Peterson[1] on gophers slack had said; maybe need to get into some kind of listener/conn wrapping and info passing that way So we should look into that

1. https://gophers.slack.com/archives/C0M8BLZAN/p1673793286287319

[komuw]

package main

import (
    "crypto/tls"
    "fmt"
    "io"
    "log"
    "net/http"
    "net/http/httptest"
    "sync"
)

func main() {
    var chisMu sync.Mutex
    chis := make(map[string]*tls.ClientHelloInfo)

    ts := httptest.NewUnstartedServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        chisMu.Lock()
        defer chisMu.Unlock()

        var supportedVersions []uint16
        chi := chis[r.RemoteAddr]
        if chi != nil {
            supportedVersions = chi.SupportedVersions
        }

        fmt.Fprintln(w, "Hello, client who supported", supportedVersions)
    }))
    ts.TLS = &tls.Config{
        GetConfigForClient: func(chi *tls.ClientHelloInfo) (*tls.Config, error) {
            chisMu.Lock()
            defer chisMu.Unlock()
            chis[chi.Conn.RemoteAddr().String()] = chi
            return nil, nil
        },
    }
    ts.StartTLS()
    defer ts.Close()

    client := ts.Client()
    res, err := client.Get(ts.URL)
    if err != nil {
        log.Fatal(err)
    }

    greeting, err := io.ReadAll(res.Body)
    res.Body.Close()
    if err != nil {
        log.Fatal(err)
    }

    fmt.Printf("%s", greeting)
}

https://go.dev/play/p/oQ-OaEoZUix

Problem is that the chis map still depends on r.RemoteAddr which can be spoofed

Suggested by Dan Peterson[1] on gophers slack.

1. https://gophers.slack.com/archives/C0M8BLZAN/p1673793286287319

[komuw]

The order of the various lists(ciphers, curves) etc from the client, I think, is of importance for fingerprinting. So even if we use tls.ClientHelloInfo we need to make sure that it is not like sorting those lists(or changing them from the order that the client presents them)

[komuw]

Accessing the underlying socket of a net/http response : https://stackoverflow.com/questions/29531993/accessing-the-underlying-socket-of-a-net-http-response

[komuw]

package main

import (
    "crypto/rand"
    "crypto/rsa"
    "crypto/tls"
    "crypto/x509"
    "encoding/pem"
    "io"
    "math/big"
    "net"
    "net/http"
    "net/url"
)

func generateTLSConfig() *tls.Config {
    key, err := rsa.GenerateKey(rand.Reader, 1024)
    if err != nil {
        panic(err)
    }
    template := x509.Certificate{SerialNumber: big.NewInt(1)}
    certDER, err := x509.CreateCertificate(rand.Reader, &template,
        &template, &key.PublicKey, key)
    if err != nil {
        panic(err)
    }
    keyPEM := pem.EncodeToMemory(&pem.Block{
        Type:  "RSA PRIVATE KEY",
        Bytes: x509.MarshalPKCS1PrivateKey(key)},
    )
    certPEM := pem.EncodeToMemory(&pem.Block{
        Type:  "CERTIFICATE",
        Bytes: certDER,
    })

    tlsCert, err := tls.X509KeyPair(certPEM, keyPEM)
    if err != nil {
        panic(err)
    }
    return &tls.Config{
        Certificates:       []tls.Certificate{tlsCert},
        GetConfigForClient: storeClientSNI,
    }
}

var (
    tlsCfg = generateTLSConfig()
)

func storeClientSNI(chi *tls.ClientHelloInfo) (*tls.Config, error) {
    conn := chi.Conn.(*hackedInnerConn)
    conn.sni = chi.ServerName
    return nil, nil
}

type hackedListener struct {
    net.Listener
}

type hackedInnerConn struct {
    net.Conn
    sni string
}

type hackedOuterConn struct {
    net.Conn
    hic *hackedInnerConn
}

type hackedAddr struct {
    net.Addr
    hoc *hackedOuterConn
}

func (hc hackedOuterConn) LocalAddr() net.Addr {
    return hackedAddr{
        hc.Conn.LocalAddr(),
        &hc,
    }
}

func (hln hackedListener) Accept() (net.Conn, error) {
    conn, err := hln.Listener.Accept()
    if err != nil {
        panic(err)
    }
    hoc := hackedOuterConn{}
    hic := hackedInnerConn{
        conn,
        "",
    }
    hoc.Conn = tls.Server(&hic, tlsCfg)
    hoc.hic = &hic
    return hoc, nil
}

func startServer(addr string, handler http.Handler) {
    netAddr, err := url.Parse(addr)
    if err != nil {
        panic(err)
    }

    server := &http.Server{
        Handler: handler,
    }
    ln, err := net.Listen("tcp", netAddr.Host)
    if err != nil {
        panic(err)
    }
    hackedLn := hackedListener{
        ln,
    }
    err = server.Serve(hackedLn)
    if err != nil {
        panic(err)
    }
}

func mustWrite(w io.Writer, p []byte) {
    _, err := w.Write(p)
    if err != nil {
        panic(err.Error())
    }
}

func main() {
    handler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        haddr := r.Context().Value(http.LocalAddrContextKey).(hackedAddr)
        mustWrite(w, []byte(haddr.hoc.hic.sni+"\r\n"))
        mustWrite(w, []byte(r.Host+"\r\n"))
    })
    startServer("https://127.0.0.1:4443", handler)
}

from: https://github.com/golang/go/issues/20956#issuecomment-535887459

[komuw]

From https://github.com/golang/go/issues/36337#issuecomment-582619266 , FiloSottile said:

We don't surface values to the application when there is no other use case than fingerprinting. 
Not because fingerprinting is not a valid use case, but because it asymptotically tends to 
require access to everything, polluting the API.

Instead, I usually recommend making a net.Conn wrapper that reads the 
ClientHello off the wire and makes all of the details available as needed.

So, we should consider doing that.

[komuw]

https://github.com/golang/go/issues/16100#issuecomment-834271923

[komuw]

CircleCI recently got hacked, the entrypoint was[1]: "Our investigation indicates that the malware was able to execute session cookie theft, enabling them to impersonate the targeted employee in a remote location"

Loom also had an incident[4]. They said; "Some our users had their information exposed to other user accounts. A configuration change to our Content Delivery Network (CDN) caused incorrect session cookies to be sent back to our users."

I think the issue could have been mitigated by storing the client's IP address in the session cookie, and then failing any requests that have mismatched IP addresses. However, client IPs are easily spoofed[2]. So instead of storing IP address, how about we store the ja3 hash instead?

This still does not resolve the issue completely. This is because, whereas the attacker now cannot use their own machine/s to carry out the attack, they can still do so by proxying though the employees machine[3]

1. https://circleci.com/blog/jan-4-2023-incident-report/
2. https://adam-p.ca/blog/2022/03/x-forwarded-for/
3. https://news.ycombinator.com/item?id=34388099
4. https://www.loom.com/blog/march-7-incident-update

John Althouse(one of the authors of JA3) says that using ja3 would have made session hijacking a pain in the a**

https://twitter.com/4A4133/status/1615103474739429377

[komuw]

There's an alternative format called TS1: https://github.com/lwthiker/ts1

• It is like ja3, except it incorporates more options from the ssl handshake
• It also has another fingerprinter that works on http2

[komuw]

Do note that even though salesforce/ja3 is released under a BSD 3-Clause license, the ja3 fingerprinting is patented by salesforce: https://patents.google.com/patent/US11128606B2

The ja3 repo says: JA3 is a .. TLS fingerprinting .. inspired by research of Brotherston & his TLS Fingerprinting tool: FingerprinTLS. Looking at the linked research and tool , I'm surprised salesforce got the patent since their is prior art. Anyhow, I'm not a lawyer and have no clue how the patent system works.

[komuw]

• check to see if https://github.com/golang/go/blob/go1.20/src/crypto/tls/handshake_test.go#L27-L42 has TLS test vectors we can use.

[komuw]

• https://github.com/golang/go/issues/32936#issuecomment-1454853964
• https://github.com/bpowers/go-fingerprint-example/blob/main/fingerprinting_server.go