search

storj / uplink

Storj network Go library
MIT License
123 stars 17 forks source link

Update uplink CLI to not use multipart upload #143

Closed shaupt131 closed 9 months ago

shaupt131 commented 1 year ago

Goal:

Uplink CLI needs to be updated not to use multipart upload by default to avoid some round trips and make maximum use of the new upload refactor features.

Acceptance Criteria:

Links:

Project Schedule: https://docs.google.com/spreadsheets/d/1ok5WOYNB_qeVOlsVS4-394vqp_IbmpN8MBPThdez6ew/edit?usp=sharing

iglesiasbrandon commented 1 year ago

@zeebo @jtolio does this also apply to objects larger than 64 MB? if so can you update the description of the ticket?

iglesiasbrandon commented 1 year ago

Most of this ticket might be removing code, but we need to be thoughtful about it because we may need to change some API to support it.

iglesiasbrandon commented 1 year ago

Once we test this on uplink CLI we will determine how to expose this in libuplink.

shaupt131 commented 1 year ago

https://review.dev.storj.io/c/storj/storj/+/10157

mniewrzal commented 9 months ago

@egonelbre is that implemented already?

egonelbre commented 9 months ago

Ah, yes, this was implemented in https://review.dev.storj.io/c/storj/storj/+/12187

jtolio commented 9 months ago

@zeebo @jtolio does this also apply to objects larger than 64 MB? if so can you update the description of the ticket?

it only does if parallelism is off.

Once we test this on uplink CLI we will determine how to expose this in libuplink.

this actually was specific to how uplink CLI was using libuplink. there is no change required in libuplink. uplink cli was just using libuplink inefficiently in small file cases