bellingcat / auto-archiver

Automatically archive links to videos, images, and social media content from Google Sheets (and more).
https://pypi.org/project/auto-archiver/
MIT License
489 stars 53 forks source link

Add a timestamp authority client Step #91

Closed msramalho closed 4 months ago

msramalho commented 10 months ago

Following information from this timestamp-authority repo ( RFC3161 Timestamp Authority) implement a Step which connects to a timestamp authority server, one example that can be tested write away is https://freetsa.org/index_en.php

taken from there a full example with sha512 is:


###########################################################
# 1. create a tsq file (SHA 512)
###########################################################
openssl ts -query -data file.png -no_nonce -sha512 -out file.tsq

# Option -cert: FreeTSA is expected to include its signing certificate (Root + Intermediate Certificates) in the response. (Optional)
# If the tsq was created with the option "-cert", its verification does not require "-untrusted".
#$ openssl ts -query -data file.png -no_nonce -sha512 -cert -out file.tsq

# How to make Timestamps of many files?

# To timestamp multiple files, create a text file with all their SHA-512 hashes and timestamp it.
# Alternatively, you may pack all the files to be timestamped in a zip/rar/img/tar, etc file and timestamp it.

# Generate a text file with all the hashes of the /var/log/ files
$ find /var/log/ -type f -exec sha512sum {} + > compilation.txt

###########################################################
# 2. cURL Time Stamp Request Input (HTTP / HTTPS)
###########################################################

# HTTP 2.0 in cURL: Get the latest cURL release and use this command: curl --http2.
curl -H "Content-Type: application/timestamp-query" --data-binary '@file.tsq' https://freetsa.org/tsr > file.tsr

# Using the Tor-network.
#$ curl -k --socks5-hostname 127.0.0.1:9050 -H "Content-Type: application/timestamp-query" --data-binary '@file.tsq' https://4bvu5sj5xok272x6cjx4uurvsbsdigaxfmzqy3n3eita272vfopforqd.onion/tsr > file.tsr

# tsget is very useful to stamp multiple time-stamp-queries: https://www.openssl.org/docs/manmaster/apps/tsget.html
#$ tsget -h https://freetsa.org/tsr file1.tsq file2.tsq file3.tsq

###########################################################
# 3. Verify tsr file
###########################################################

wget https://freetsa.org/files/tsa.crt
wget https://freetsa.org/files/cacert.pem

# Timestamp Information.
openssl ts -reply -in file.tsr -text

# Verify (two diferent ways).
# openssl ts -verify -data file -in file.tsr -CAfile cacert.pem -untrusted tsa.crt 
openssl ts -verify -in file.tsr -queryfile file.tsq -CAfile cacert.pem -untrusted tsa.crt
# Verification: OK

Discussion topics

Given the cyclical definition of this, I wonder what is the best way to implement it as it needs to run after the HtmlFormatter which can only happen as a database meaning the formatter should only include/display the links if they actually exist.

msramalho commented 10 months ago

asked an LLM to convert the bash code into python, though it's still very bashy:

import subprocess

# Step 1: Create a tsq file (SHA 512)
subprocess.run(["openssl", "ts", "-query", "-data", "file.png", "-no_nonce", "-sha512", "-out", "file.tsq"])

# Step 2: cURL Time Stamp Request Input (HTTP / HTTPS)
curl_command = [
    "curl",
    "-H", "Content-Type: application/timestamp-query",
    "--data-binary", "@file.tsq",
    "https://freetsa.org/tsr"
]
with open("file.tsr", "wb") as tsr_file:
    subprocess.run(curl_command, stdout=tsr_file)

# Step 3: Verify tsr file
subprocess.run(["wget", "https://freetsa.org/files/tsa.crt"])
subprocess.run(["wget", "https://freetsa.org/files/cacert.pem"])

# Timestamp Information
subprocess.run(["openssl", "ts", "-reply", "-in", "file.tsr", "-text"])

# Verify (two different ways)
verify_command = [
    "openssl", "ts", "-verify",
    "-in", "file.tsr",
    "-queryfile", "file.tsq",
    "-CAfile", "cacert.pem",
    "-untrusted", "tsa.crt"
]
subprocess.run(verify_command)
msramalho commented 10 months ago

in a way this is connected to #90

msramalho commented 10 months ago

other RFC3161 servers listed here: https://gist.github.com/Manouchehri/fd754e402d98430243455713efada710