celestiaorg / knuu

Integration Test Framework
Apache License 2.0
38 stars 31 forks source link

Optimize Minio Client Usage by Reusing Across Requests #429

Closed mojtaba-esk closed 4 days ago

mojtaba-esk commented 3 weeks ago

Description

Currently, our application creates a new Minio client (miniogo.Client) for each request. This approach can lead to performance inefficiencies due to the overhead associated with repeatedly initializing new clients, especially under high load.

Proposed Change

Refactor the application to instantiate a single Minio client and reuse it across multiple requests. This change aims to reduce initialization overhead and improve overall performance by leveraging connection pooling and reducing setup times.

Benefits

Implementation Outline

  1. Create a MinioService struct that includes a miniogo.Client as a field.
  2. Instantiate MinioService at application startup with a single Minio client.
  3. Refactor existing code to use the Minio client from MinioService instead of creating new instances.

Code Snippet

type MinioService struct {
    Client miniogo.Client
}
func NewMinioService(endpoint, accessKey, secretKey string) (MinioService, error) {
    cli, err := miniogo.New(endpoint, &miniogo.Options{
        Creds: credentials.NewStaticV4(accessKey, secretKey, ""),
        Secure: false,
})
if err != nil {
   return nil, err
}
return &MinioService{
       Client: cli,
    }, nil
}