Node Group and Region select for Buckets

[ferristocrat]

Background

Currently, all buckets are global by default, which may not be optimal for certain use cases and can result in slower performance for users in certain regions. Providing users with the option to choose between global, region-specific buckets or groups of nodes can improve their experience and even enable new customers to use Storj.

Problem Statement

As a Storj user, I want to be able to choose between a global bucket and region-specific buckets upon creation so that I can prioritize either global durability or regional performance for my uploads and downloads based on my specific needs.

OR

As a Storj user, I want to be able to choose a satellite-defined group of nodes when creating a bucket so that I can store my data on nodes that have a specific certification or meet a specific business requirement. For example, a group of nodes could be in SOC2-certified facilities.

Why do customers care about having this choice?

Region: Choosing a specific region can offer better performance, lower latency, and potentially lower costs for data transfers within that region. However, region-specific buckets might have slightly lower durability compared to global buckets due to being restricted to a single region.

Global: On the other hand, global buckets provide high durability and global access to the data at the cost of potentially slower performance, especially for users in regions far from the data storage nodes. Global buckets are suitable for use cases where data availability across different regions is more important than performance.

Group:

Criteria	Specific Region	Global
Performance	Better (lower latency) within the selected region	Slower (higher latency) due to global distribution
Durability	Slightly lower due to restriction to a single region	Higher, as data is distributed globally
Data access	Restricted to the selected region	Global access to data
Data transfer costs	Potentially lower within the selected region	Potentially higher due to global data transfers

Scenarios

1. Choose a specific region:
   • When your users are primarily located within a particular region, and low latency and faster performance are crucial.
   • When you want to minimize data transfer costs within the region.
2. Choose global:
   • When you need to ensure data is accessible from multiple regions and global durability is a higher priority.
   • When your users are distributed across the world, and you want to maintain a single, globally available data set.

Requirements

Assumptions

• Once a region is selected for a bucket, it cannot be changed.
• Existing buckets will default to the "Global" region selection, and users cannot modify the region selection for these buckets.

Acceptance Criteria

• A new region selection option is added to the bucket creation page on the Satellite UI
• The new region selection option is available in Uplink and S3 Gateway
• The new region selection option includes the options for "Global", "North America", and "Europe".
• Clear information is provided to users about the benefits and limitations of each region selection option.
• The default region selection is "Global" if no region is selected.
• Object storage is restricted to the selected region for region-specific buckets.
• High durability is retained across all regions, including region-specific buckets (no changes to RS numbers).
• The new region selection feature is documented in docs.storj.io.

Success Metrics

• The number of users who choose a region-specific bucket upon creation.
• The increase in user satisfaction and positive feedback related to improved performance.
• The decrease in support requests related to slow performance.
• The increase in revenue and usage of the service.

Out of Scope

• Customers having the ability to create or select an arbitrary group of nodes. The groupings and regions of nodes are created by the satellites.

Open Discussion

• The possibility of restricting region selection based on Satellite may be discussed further, but is not currently included in the scope of the feature.

[iglesiasbrandon]

@ferristocrat can you refine this roadmap item and work with Tome on the UX/UI part? and work with Artur to determine if anything on the edge services needs to be updated.

The satellite team recently implemented: https://github.com/storj/storj/issues/5879 Which changes how the placement settings work on the satellite side for accounts, projects and buckets.

[mobyvb]

My notes:

This task involves work that will touch code owned by the Satellite team. We may need to consult/work with them to do the metainfo-side work. That stuff can actually be worked on in parallel with the other tasks, which could speed up development.

Work	Details	Story Point Estimation
DB work	add field for region to bucket_metainfos	2 sp
Metainfo work	if new field is set to NA or EU for a bucket, uploads are restricted to that region	3 sp
UI work	functionality to create bucket with region is implemented on backend. UI for bucket creation updated	3 sp

Personal ROM estimation: 2-3 sprints (the actual coding should be small, but this should account for QA/Design/Product feedback throughout implementation

[iglesiasbrandon]

closing this issue in favor of https://github.com/storj/roadmap-private/issues/84