search

storj / team-metainfo

GNU Affero General Public License v3.0
0 stars 0 forks source link

Object deletes need to use bounded memory regardless of the amount of segments per object #41

Open mniewrzal opened 2 years ago

mniewrzal commented 2 years ago

Object deletes need to use bounded memory regardless of the amount of segments per object. The project Ivan was deleting had 9,459,183 segments, but just 1,178 objects.
Migrated from https://storjlabs.atlassian.net/browse/PG-234

mniewrzal commented 2 years ago

Commented by Michal Niewrzal

fix for deleting bucket https://review.dev.storj.io/c/storj/storj/+/5090

mniewrzal commented 2 years ago

Commented by Michal Niewrzal

we still need to fix other places where we are deleting objects + segments

mniewrzal commented 2 years ago

Commented by Michal Niewrzal

File delete.go from metabase package contains methods for deleting. Most of methods returns DeleteObjectResult with contains array of objects and segments that were deleted. We need to refactor it to return only array of deleted objects and add give ability to delete segment pieces in a way like in https://review.dev.storj.io/c/storj/storj/+/5090 with DeletePieces func as an option for the method.

mniewrzal commented 2 years ago

Commented by Michal Niewrzal

fix for deleting objects https://review.dev.storj.io/c/storj/storj/+/5294