s-rah / onionscan

OnionScan is a free and open source tool for investigating the Dark Web.
https://twitter.com/OnionScan
Other
2.89k stars 600 forks source link

Split Resources into Chunks for Storing #88

Open s-rah opened 8 years ago

s-rah commented 8 years ago

Currently we drop resources larger than 2MB because of limitations with the database - regardless of which backing store we end up with in the future, we are likely always going to need to chunk blobs.

We should