"Fun" type system bug breaks `CustomFalsePositiveChecker` interface

rgmz commented 3 months ago

Please review the Community Note before submitting

TruffleHog Version

HEAD

Trace Output

N/A

Description

Consider the following file, example.json

{
  "type": "service_account",
  "project_id": "authenticated-image-pulling2",
  "private_key_id": "b9f2a664aa9b20484cc1586063fefda19224ac3b2",
  "private_key": "-----BEGIN PRIVATE KEY-----\nMIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQC7SHnKTEEiYLjf\nJfAPGmJ3wrBceI50JKLqKmFXNQ/tDXbQ+h9aYxjWIL8Dx0Je74mZ/KMnWgXF5KZS\noA6KnIO9b/RcSeWeiItRzI3/YXV+O6CcrjJIyxjqVjnfW2i3sa37t9A9TFdlfrrn\n4zRJb9ixyMX4bLtqFGvB03NIitA3sVZ588koQAfh3JhaBegMj+Z4RbJ4heiBQT03\nvUo5bEaPeT9DMzlwseaPWgrt6N0OUDcAE9xlcIzMn253Pn/K82HZrtLxjGvRHMUx\nx4f8pJxfCxxBSwgSNF+w9jdmtvoL0Fa7dgnpRe86VD66z3Yzrj4yKEtjshKdyyUd\nIyqXh7RRAgMBAAECggEAOzsdwZxCUVQTxAdkl/I5SDUbv/Mk4pifqb2DkagnhEpo\n1Ij2l4iV10r9/nzrgcjyVPAwzYZMIx1AeQtD7hS4GZapyvJYG76FiXZPRoCVPzou\nfr8dCiapl5tzrC9lvAsGwoCM7IYTcfcVt7cE12D3QKsF6Z7B2zfgKKnuYPf+CE6T\ncM0y0h+XE/d0DoHDhW/zaMrXHj8Toweuytkbbs4f/9Fj9PnSgDOYPwlalVTr+FQa\nJRwVjVlXpFAQmx3BrwnkZt3CiWWiF3d+Hi9EtUbtVrW1b6g+RQOIbqamr+8bRndX\n6VgqBAkJZ8RVydxUP0d11GjuOPDxBnHBnc4QokIrEQKBgQD1CeicudhWtg4+gSxb\nzejxtV1N41mducBzo2jyoWGo3PT8wrBO/yQE34qOVJ/id+8I8hZ4oIhu+JA00s6g\nTnIq+v/d/TEjY81nkZiCkmRPWbXxaYtxR21KPXrLNNQJKkm8tdyXyPql8MoyGfCW\n2viPJKNb6HZnv9CyjdJ9g2LDnQKBgQDDqSvyDmheb923Ioz4legMR+m9glXUgSKg\nEsfYemRfmNWB+C7vaIyURmY5NyMxfBVWswWFWKaxc+I+bqsflzzVYtZp18MGjsMD\nfeefAX6BZMsUt7Bl7Z9VJ85ntEdqACLpZ+Z/3tIRVugCWZQ1hknlGkGT024JEE++\nNyH1g3d3RQKBgQCRv1wJZI0mPlFIokKFNHua0Tp3KoRSSXsMDSVOM+lHrG1XrmF6\nC04cS+447GLRLG8UThJJm4qrHtN/Z+gY96/2mqb4HjJND37MXJBvEa3yeLS8q/+R\n2F9MKjdQiNKZxPpo8W8NJTDY5NkPZdhxkjsHwU4dS66p1TDIE40gtLZZDQKBgFjW\nKrnQiNq39/b6nP8RMTbCQAJndwjxSQNdA5fqmkA9aFOGl+jjk1CPVkKMIlKJgDbJ\nOax9v9G6R/MI1HGXfWt1YNzVthr4HtsrA0tSplmhpgNWE6Vz6nADjtfPJs2eGjvX\njPRp+v8ccmL+wSg8PLjk3vl7ee5rlYlMBwMuGcPxAoGAednxbW1RLmVnlAiHLu/L\nlmfAwDWmEiI0Ug+PLnoOvO5tQ5d4W1/xEN8lP4qkspkffMQnNh4SYGEeBT32ZqCT\nJRgf0Xjoyvvup9xXjMkXrpY/yc1zfqTZC0MO9/1Uc1bRGdZ2dy3lR5NWap7OXyfO\nPPpNFoPTXgv3qCqnlLHrGzM=\n-----END PRIVATE KEY-----\n",
  "client_email": "image-pulling2@authenticated-image-pulling2.iam.gserviceaccount.com",
  "client_id": "1137979145300732787122",
  "auth_uri": "https://accounts.google.com/o/oauth2/auth",
  "token_uri": "https://accounts.google.com/o/oauth2/token",
  "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
  "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/image-pulling2%40authenticated-image-pulling2.iam.gserviceaccount.com"
}

If you were to run the following command, what would you expect the output to be? Perhaps two results: GCP and PrivateKey?

./trufflehog filesystem /tmp/example.json --results=unverified

The answer is, surprisingly, nothing!

🐷🔑🐷  TruffleHog. Unearth your secrets. 🐷🔑🐷

2024-06-11T15:00:29-04:00       info-0  trufflehog      running source  {"source_manager_worker_id": "yNhaW", "with_units": true}
2024-06-11T15:00:30-04:00       info-0  trufflehog      finished scanning       {"chunks": 1, "bytes": 2386, "verified_secrets": 0, "unverified_secrets": 0, "scan_duration": "335.667903ms", "trufflehog_version": "dev"}

Running with --results=filtered_unverified yields a surprising result:

🐷🔑🐷  TruffleHog. Unearth your secrets. 🐷🔑🐷

2024-06-11T15:00:21-04:00       info-0  trufflehog      running source  {"source_manager_worker_id": "HoTyv", "with_units": true}
2024-06-11T15:00:21-04:00       info-0  trufflehog      Filtered out known false positive       {"verification_overlap_worker_id": "KzUOM", "result": {"DetectorType":15,"DetectorName":"","DecoderType":0,"Verified":false,"Raw":"LS0...ZLS0tLS0K","RawV2":null,"Redacted":"-----BEGIN PRIVATE KEY-----\nMIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcw","ExtraData":{},"StructuredData":null}}

How can this be? Both GCP and PrivateKey explicitly don't attempt to do any form of "false positive" check.

https://github.com/trufflesecurity/trufflehog/blob/433a57adaf8fedce312630ef593f5142227c5855/pkg/detectors/gcp/gcp.go#L136-L138

https://github.com/trufflesecurity/trufflehog/blob/433a57adaf8fedce312630ef593f5142227c5855/pkg/detectors/privatekey/privatekey.go#L154

Put your guesses in the comments now!

...

The problematic lines are:

https://github.com/trufflesecurity/trufflehog/blob/433a57adaf8fedce312630ef593f5142227c5855/pkg/engine/engine.go#L738

https://github.com/trufflesecurity/trufflehog/blob/433a57adaf8fedce312630ef593f5142227c5855/pkg/engine/engine.go#L848

The struct being passed in is actually DetectorMatch, and not gcp.Scanner as expected. For some reason, the compiler sees that the embedded struct matches detectors.Detector, but does not see that the embedded struct matches detectors.CustomFalsePositiveChecker.

This may not be a type system bug per se, but it's clearly contrary to the intentions of the code and understanding of the developer.

rgmz commented 3 months ago

Priyadhana commented 3 months ago

Running trufflehog on a .pem file to scan for privateKeys but it returns nothing. --results=filtered_unverified works only with filesystem and not with git.

Command tried: docker run -v "/test/private_key.pem:/test/private_key.pem" trufflesecurity/trufflehog:latest git file:///test/private_key.pem --results=filtered_unverified

Image version: "trufflehog_version": "3.78.1"

Can someone please help on how to scan for privateKeys on a git commit?

trufflesecurity / trufflehog