[Tech Request]: High concurrent S3 reads causes token limit error - Githubissues

matrixorigin / matrixone

Hyperconverged cloud-edge native database

https://docs.matrixorigin.cn/en

Apache License 2.0

1.71k stars 265 forks source link

[Tech Request]: High concurrent S3 reads causes token limit error #17329

Open reusee opened 3 days ago

reusee commented 3 days ago

Is there an existing issue for the same tech request?

[X] I have checked the existing issues.

Does this tech request not affect user experience?

[X] This tech request doesn't affect user experience.

What would you like to be added ?

Lower s3 max connections from 1000.

Why is this needed ?

No response

Additional information

No response

reusee commented 3 days ago

related failed metrics: https://grafana.ci.matrixorigin.cn/d/a873685a-bf2f-4049-9602-b09d6e728735/fileservice-metrics?orgId=1&refresh=30s&var-interval=1m&var-namespace=mo-search-nightly-9b98c2c7c-20240704&var-pod=nightly-regression-dis-tp-cn-9tdpg&var-pod=nightly-regression-dis-tp-cn-pphjh&var-pod=nightly-regression-dis-tp-cn-klb8p&from=1720068685911&to=1720090285911

reusee commented 2 days ago

fixed in the above PR. related OK test: https://github.com/matrixorigin/mo-nightly-regression/actions/runs/9805696583