search

SSLMate / certspotter

Certificate Transparency Log Monitor
https://sslmate.com/certspotter
Mozilla Public License 2.0
983 stars 84 forks source link

Save progress #37

Closed anotherdevs closed 1 year ago

anotherdevs commented 4 years ago

Hi

I have started Certspotter and asked it to download from the beginning. It looks like this will take a few hundred years to traverse through. /s

Is there any way to save the progress it has achieved fx. every 1 minute, so in case of panic or I want to reboot, it can continue from that point (with the chance of needing to parse already parsed data)?

shoeper commented 3 years ago

See also #44 regarding how much time it'll take.

anotherdevs commented 3 years ago

@shoeper I am not sure if this helps you but here you go:

$ vnstat -m

   month        rx      |     tx      |    total    |   avg. rate
------------------------+-------------+-------------+---------------
  Jan '20      6.48 TiB |  143.49 GiB |    6.62 TiB |   21.23 Mbit/s
  Feb '20      4.49 TiB |  104.53 GiB |    4.60 TiB |   15.76 Mbit/s
  Mar '20      3.65 TiB |   85.65 GiB |    3.73 TiB |   11.98 Mbit/s
  Apr '20      3.22 TiB |   78.06 GiB |    3.30 TiB |   10.94 Mbit/s
  May '20      3.42 TiB |   83.97 GiB |    3.51 TiB |   11.24 Mbit/s
  Jun '20     16.70 TiB |  246.44 GiB |   16.94 TiB |   56.15 Mbit/s
  Jul '20     15.91 TiB |  343.25 GiB |   16.24 TiB |   52.09 Mbit/s
  Aug '20      9.80 TiB |  231.82 GiB |   10.02 TiB |   32.14 Mbit/s
  Sep '20      5.22 TiB |  129.54 GiB |    5.35 TiB |   17.73 Mbit/s
  Oct '20      6.12 TiB |  135.55 GiB |    6.25 TiB |   20.05 Mbit/s
  Nov '20      4.38 TiB |  104.10 GiB |    4.48 TiB |   14.84 Mbit/s
  Dec '20      1.12 TiB |   26.55 GiB |    1.15 TiB |    8.55 Mbit/s