dheeban0111 commented 1 year ago

I want to scan my private repo and all my repo in the entire org. I followed the below commands and procedures, nothing worked. Could u please help me with the right direction?

Command followed for ssh into the private repo: (Added my public key generated from the hosted vm to the github)

docker run --rm -it -v "$PWD:/pwd" trufflesecurity/trufflehog:latest git git@<my-private repo> Error received: errortrufflehog error preparing git repo for scanning{"error": "failed to clone unauthenticated Git repo (ssh://): could not clone repo: ssh://, error running 'git clone': exit status 128"}

Command followed for scan private repo with personal access token: docker run --rm -it -v "$PWD:/pwd" trufflesecurity/trufflehog:latest github --repo <https-github-url> --token <my-token> Error received: "2023-07-19T13:43:19Zinfo-0trufflehogloaded decoders{"count": 3} 2023-07-19T13:43:19Zinfo-0trufflehogloaded detectors{"total": 745, "verification_enabled": 745, "verification_disabled": 0}  TruffleHog. Unearth your secrets. 2023-07-19T13:43:19Zinfo-0trufflehogCompleted enumeration{"source_type": "SOURCE_TYPE_GITHUB", "source_name": "github", "num_repos": 1, "num_or gs": 0, "num_members": 0} 2023-07-19T13:43:20Zinfo-0trufflehogscanning comments{"source_type": "SOURCE_TYPE_GITHUB", "source_name": "github"}"

Doc followed https://docs.trufflesecurity.com/docs/configuration/sources/

I’m not sure how do use the above doc for github config as a part of my self-hosted cli version running in ubuntu machine?

It would be a great support for smaller users like me. Also the self-hosted is available as a complete web-version? Or only cli.

Thanks in advance. Please help me on this

rgmz commented 1 year ago

docker run --rm -it -v "$PWD:/pwd" trufflesecurity/trufflehog:latest git git@<my-private repo> Error received: errortrufflehog error preparing git repo for scanning{"error": "failed to clone unauthenticated Git repo (ssh://): could not clone repo: ssh://, error running 'git clone': exit status 128"}

This is failing because SSH cloning requires your private key, and the Docker container does not have access to your private keys (unless you mount them).

Command followed for scan private repo with personal access token: docker run --rm -it -v "$PWD:/pwd" trufflesecurity/trufflehog:latest github --repo <https-github-url> --token <my-token> Error received: "2023-07-19T13:43:19Zinfo-0trufflehogloaded decoders{"count": 3} 2023-07-19T13:43:19Zinfo-0trufflehogloaded detectors{"total": 745, "verification_enabled": 745, "verification_disabled": 0}  TruffleHog. Unearth your secrets. 2023-07-19T13:43:19Zinfo-0trufflehogCompleted enumeration{"source_type": "SOURCE_TYPE_GITHUB", "source_name": "github", "num_repos": 1, "num_or gs": 0, "num_members": 0} 2023-07-19T13:43:20Zinfo-0trufflehogscanning comments{"source_type": "SOURCE_TYPE_GITHUB", "source_name": "github"}"

I do not see any errors in this. Can you clarify why you think that this didn't work?

Check out the first question in the FAQ: https://github.com/trufflesecurity/trufflehog#question-faq

dheeban0111 commented 1 year ago

Thanks for the reply @rgmz . Absolutely right on what u mentioned about the 1st command. The latter is the concern. I have hardcoded some secrets (dummy one's) in the repo which I'll paste here. It went undetected, that is the reason for my query.

`import boto3 import pymongo from pymongo import MongoClient

Sample AWS S3 bucket and MongoDB connection settings

AWS_ACCESS_KEY = "ASDFGHDJKLMTRE567IORNBFTCYEUNFUgfbfnbufhbgeut889nv" AWS_SECRET_KEY = "vgbruvn98GHJKJOIU9889()(*&^)" AWS_BUCKET_NAME = "my_bucket"

MONGODB_USERNAME = "dummy_001" MONGODB_PASSWORD = "hdhfnIUIIb&68B&*" MONGODB_HOST = "test" MONGODB_PORT = 27017 MONGODB_DATABASE = "mydatabase" MONGODB_COLLECTION = "mycollection"

Sample list of files in the AWS S3 bucket

aws_files = [ "file1.txt", "file2.txt", "file3.txt", ]

Sample user information

user = { "name": "Dheeban", "email": "dheeban@wpp.com", "age": 26, "address": "123 Gotham City", }

Function to copy files from AWS S3 bucket

def copy_files_from_aws(): s3_client = boto3.client( "s3", aws_access_key_id=AWS_ACCESS_KEY, aws_secret_access_key=AWS_SECRET_KEY, )

for file_name in aws_files:
    s3_client.download_file(AWS_BUCKET_NAME, file_name, file_name)

Function to upload files to MongoDB

def upload_files_to_mongodb(): mongo_uri = f"mongodb://{MONGODB_USERNAME}:{MONGODB_PASSWORD}@{MONGODB_HOST}:{MONGODB_PORT}/"

client = MongoClient(mongo_uri)
db = client[MONGODB_DATABASE]
collection = db[MONGODB_COLLECTION]

for file_name in aws_files[:3]:  # Upload the first 3 files to MongoDB
    with open(file_name, "r") as file:
        content = file.read()
        document = {"file_name": file_name, "content": content}
        collection.insert_one(document)

Main function

def main(): copy_files_from_aws() upload_files_to_mongodb()

# Display user information
print("User Information:")
print(f"Name: {user['name']}")
print(f"Email: {user['email']}")
print(f"Age: {user['age']}")
print(f"Address: {user['address']}")

if name == "main": main() `

dnicolacopoulos commented 1 year ago

Hello 👋

I'm also having a similar issue with running the git command on private repo (docker run --rm -it -v "$HOME/.ssh:/root/.ssh:ro" trufflesecurity/trufflehog:latest git ssh://github.com/<my-private-repo>).

Even with the SSH mounted volume , I get the error:

2023-08-02T15:42:33Z    error   trufflehog      error preparing git repo for scanning   {"error": "failed to clone unauthenticated Git repo (ssh://github.com/<my-private-repo>): could not clone repo: ssh://github.com/<my-private-repo>, error running 'git clone': exit status 128"}

trufflesecurity / trufflehog