storj / gateway-st

Single-tenant, S3-compatible server to interact with the Storj network
Apache License 2.0
71 stars 19 forks source link

Http trace option #84

Closed Zetanova closed 2 months ago

Zetanova commented 2 months ago

What version of Gateway-ST are you using (gateway version)?

$ gateway version
Version: v1.9.0

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (uname -mrsv)?

docker

What did you do?

I connected nextcloud to gateway-st as S3 primary storage.
Single operations on small and large files are working fine, but the nextcloud feature to create office template files for the user leads to constantly to "Please reduce your request rate." error.

I want to enable the full http trace ouput. It is normally possible over mc admin trace Only errors are printed over storjlabs/gateway run --log.level=debug --log.output=stdout --log.development=true

It seams by the that nextcloud object id that it makes at least one request successful before
nextcloud command to reproduce (executing twice) $ php8.3 occ richdocuments:update-empty-templates

Output:

#first execution
API: PutObject(bucket=nextcloud, object=urn:oid:901)
UserAgent: aws-sdk-php/3.240.8 OS/Linux/5.15.0-112-generic lang/php/8.3.8 GuzzleHttp/7
Error: Please reduce your request rate. (minio.ErrorResponse)`
#...
#second execution
API: PutObject(bucket=nextcloud, object=urn:oid:903)
UserAgent: aws-sdk-php/3.240.8 OS/Linux/5.15.0-112-generic lang/php/8.3.8 GuzzleHttp/7
Error: Please reduce your request rate. (minio.ErrorResponse)
       4: /go/pkg/mod/storj.io/minio@v0.0.0-20230627070725-370d887aacde/cmd/api-errors.go:2012:cmd.toAPIErrorCode()
       3: /go/pkg/mod/storj.io/minio@v0.0.0-20230627070725-370d887aacde/cmd/api-errors.go:2037:cmd.ToAPIError()
       2: /go/pkg/mod/storj.io/minio@v0.0.0-20230627070725-370d887aacde/cmd/object-handlers.go:1578:cmd.ObjectAPIHandlers.PutObjectHandler()
       1: net/http/server.go:2122:http.HandlerFunc.ServeHTTP()`

The request limit comes from the storj network itself, to see all http requests to the gateway instance would be helpful.

I don't know why nextcloud would make 2 put's on the same object in sequence.

What did you expect to see?

all http requests to the instance

What did you see instead?

only errors are printed out

Nextcloud basic info

https://docs.nextcloud.com/server/latest/admin_manual/configuration_files/primary_storage.html

config

'objectstore' => 
array (
    'class' => '\\OC\\Files\\ObjectStore\\S3',
    'arguments' => 
    array (
      'bucket' => 'nextcloud',
      'hostname' => '127.0.0.1',
      'key' => 'thekey',
      'secret' => 'thesecret',
      'port' => 7777,
      'use_ssl' => false,
      'use_path_style' => true,
      'uploadPartSize' => 67108864,
      'putSizeLimit' => 67108864,
      'concurrency' => 1,
    ),
  ),
Zetanova commented 2 months ago

nextcloud simple makes a putObject on file create followed by a second putObject by the content write.

The issue not to have a trace on the gateway said is still missing, but i close this issue.