search

s-rah / onionscan

OnionScan is a free and open source tool for investigating the Dark Web.
https://twitter.com/OnionScan
Other
2.89k stars 599 forks source link

why onionscandb/crawls is so larger #128

Open mylamour opened 7 years ago

mylamour commented 7 years ago

when i use onionscan to discovery the darknet , i found there was a big problem when the program was running. it's exsume a lot of diskapce image you can easily see it on the picture, and i dont why it's so larger. Thanks

mylamour commented 7 years ago

After about two weeks, it become to about 100G

s-rah commented 7 years ago

Yes, The snapshot database stores a copy of every page that you visit - this is done to power the correlation engine and can be useful for post-processing of the OnionScan database. There is no way to turn off this behavior without modifying the code.