Retain the content and/or hash of linksharing files that we take down

[ferristocrat]

Goal

• We want to start retaining the content and/or hash of linksharing files that we take down
• This will help us backtest any solution we may implement for proactively identifying these files (rather than waiting for a takedown request)

Acceptance Criteria

• Manual or automated process implemented for storing the content and hash for files we take down

Links

• N/A

[amwolff]

@storj/edge-team any idea where we should store it?

[amwolff]

I think we should use eventkit for this.

[adaRn]

I'm not sure about the volume, so an extreme opinion here: I'd be OK even with storing these in a Google Sheet, if this makes it super simple

[amwolff]

From sprint planning: probably implement the functionality in authservice-admin to send a hash of the object to eventkit for later analysis.

v2 would be moving the offending object to a bucket that we control while scanning it beforehand with, e.g., ClamAV—if the object is malware/dangerous, we would keep it and discard it otherwise.

[amwolff]

@jtolio Related to #342, how do you feel about retaining the checksum of the content that we take down for legal reasons?

[storj-gerrit[bot]]

Change cmd/authservice-admin: send link information before taking down link mentions this issue.

[storj-gerrit[bot]]

Change cmd/authservice-admin: send link information before taking down link mentions this issue.

[storj-gerrit[bot]]

Change pkg/hashreader: add package mentions this issue.