:warning:

This introduces a credits exploit to get 3GB * 12 months of free credits, and will be postponed until this can be mitigated:

Create free account
Upload 3GB of data pinned for 12 months
Upgrade account to paid tier immediately
Remove the 3GB of pins
You will be refunded approximately ~11.75months * 3GB worth of credits

Possible ideas:

1) *When upgrading, have a database column in user account that indicates the time they upgraded at
- Prevent refunding any uploads whose created_at is before the upgrade time, but allow them to be removed
- Downside with this is the upgraded_at field would be introduced in this PR, and as such any uploads from before then wouldn't be eligible for refunds
2)
- Dont allow any refunds (tbh while this is the "easiest" it is definitely the shittiest to our users)
- Only do this if there is no other possible option

:construction_worker: Purpose

This functionality has been enabled in the database manager we use, but hasn't been exposed via the API.

Note that free tier account users are unable to remove pins, and must wait for their pins to expire. This is done because repeated pinning and pin removal can be used as a vector for DoS'ing IPFS nodes by enabling pinning massive amounts of pins.

The reason this is a DoS vector is because the more pins your IPFS nodes have, the slower they become when adding new data. Additionally it means that when garbage collect is run, it takes a lot more time. To give an idea on how long, due the amount of data our IPFS nodes have, running garbage collection takes days to complete, which is a blocking process requiring the node be inaccessible.

Additionally we only do a partial refund on the remaining storage costs, minus a 72 hour buffer to accomodate for the garbage collection process, and to also prevent paid accounts from using the same DoS vector. If there was a complete refund paid accounts could pin+unpin immediately being able to exploit the DoS vector.

While this may be upsetting to some, it is a necessary step. The go-ipfs garbage collection system is unbelievably slow, and sometimes does not work requiring a complete wipe of the data directory, and copying over the data again. It is unfortunate, but to keep our service fast for all users, it is a necessary measure.

:rocket: Changes

Enable pin removal and include a partial refund cost

:warning: Breaking Changes

None

TODOs

Enable pin removal for encrypted uploads
- To do this if the pin removal call fails with an upload not found error attempt to delete via the encrypted upload models, if that fails return an error
Add tests

Codecov Report

Merging #484 into master will decrease coverage by 0.22%. The diff coverage is 5.00%.

@@            Coverage Diff             @@
##           master     #484      +/-   ##
==========================================
- Coverage   52.83%   52.61%   -0.21%     
==========================================
  Files          43       43              
  Lines        4371     4391      +20     
==========================================
+ Hits         2309     2310       +1     
- Misses       1549     1568      +19     
  Partials      513      513

Impacted Files	Coverage Δ
api/v2/routes_rtfs.go	`34.85% <0.00%> (-2.70%)`	:arrow_down:
api/v2/api.go	`85.15% <100.00%> (+0.04%)`	:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 6369551...d077f0f. Read the comment docs.

RTradeLtd / Temporal

Enable Pin Removal Via API #484