openaustralia / morph

Take the hassle out of web scraping
https://morph.io
GNU Affero General Public License v3.0
461 stars 74 forks source link

Reduce disk space overhead of storing connection_logs information #1079

Closed mlandauer closed 8 years ago

mlandauer commented 8 years ago

Currently the connection_logs table is far bigger (in disk space) by far than any of the other tables. Let's really figure out what the minimum information we need here is really and store that instead so that we can significantly reduce the disk space overhead of running the morph server