AliyundriveOpen ExceedMaxConcurrency issue

jibwf commented 1 month ago

Please make sure of the following things

[X] I have read the documentation. 我已经阅读了文档。
[X] I'm sure there are no duplicate issues or discussions. 我确定没有重复的issue或讨论。
[X] I'm sure it's due to AList and not something else(such as Network ,Dependencies or Operational). 我确定是AList的问题，而不是其他原因（例如网络，依赖或操作）。
[X] I'm sure this issue is not fixed in the latest version. 我确定这个问题在最新版本中没有被修复。

AList Version / AList 版本

v3.37.4

Driver used / 使用的存储驱动

AliyundriveOpen

Describe the bug / 问题描述

S3 使用时遇到http 500错误. 打开debug模式后，发现alist http 500是由于aliyun ExceedMaxConcurrency.

[GIN] 2024/09/24 - 17:12:46 | 500 | 55.154846ms | 192.168.1.165 | GET "/adrive/xn0_428093f4e285aac68c0e0dedc4845c44-s19584b24968d317212d-c1" [37mDEBU[0m[2024-09-24 17:12:47]/home/runner/work/alist/alist/internal/net/serve.go:241 github.com/alist-org/alist/v3/internal/net.RequestHttp() <?xml version="1.0" encoding="UTF-8"?>

RequestDeniedByCallback Callback deny this request reason: ExceedMaxConcurrency 66F2828FC5BC5B353014E694 cn-beijing-data.aliyundrive.net 0007-00000209 https://api.aliyun.com/troubleshoot?q=0007-00000209

[GIN] 2024/09/24 - 17:12:47 | 500 | 4.282942778s | 192.168.1.165 | GET "/adrive/xn0_08c45d7b29bae97ef6a24cf8c7eea9ed-sdbc31acf8e79603112d-c1"

看了下应该是第三方权益包进行了限制，非权益用户最多只能同时下载2个文.。请求alist适配.

Reproduction / 复现链接

[Uploading s3_http500_debuglog.log…]()

Config / 配置

{ "force": false, "site_url": "", "cdn": "", "jwt_secret": "mNIstpFmBVEFR07R", "token_expires_in": 48, "database": { "type": "sqlite3", "host": "", "port": 0, "user": "", "password": "", "name": "", "db_file": "data/data.db", "tableprefix": "x", "ssl_mode": "", "dsn": "" }, "meilisearch": { "host": "http://localhost:7700", "api_key": "", "index_prefix": "" }, "scheme": { "address": "0.0.0.0", "http_port": 5244, "https_port": -1, "force_https": false, "cert_file": "", "key_file": "", "unix_file": "", "unix_file_perm": "" }, "temp_dir": "data/temp", "bleve_dir": "data/bleve", "dist_dir": "", "log": { "enable": true, "name": "data/log/log.log", "max_size": 10, "max_backups": 5, "max_age": 28, "compress": false }, "delayed_start": 0, "max_connections": 0, "tls_insecure_skip_verify": true, "tasks": { "download": { "workers": 5, "max_retry": 1, "task_persistant": true }, "transfer": { "workers": 5, "max_retry": 2, "task_persistant": true }, "upload": { "workers": 5, "max_retry": 0, "task_persistant": false }, "copy": { "workers": 5, "max_retry": 2, "task_persistant": true } }, "cors": { "allow_origins": [ "" ], "allow_methods": [ "" ], "allow_headers": [ "*" ] }, "s3": { "enable": true, "port": 5246, "ssl": false } }

Logs / 日志

[GIN] 2024/09/24 - 17:12:44 | 200 | 4.936391847s | 192.168.1.165 | GET "/adrive/xn0_0eed0632a430667840aa2a4f994b3049-s7aaafccd28c3c9cf12d-c1" [37mDEBU[0m[2024-09-24 17:12:44]/home/runner/work/alist/alist/internal/op/path.go:25 github.com/alist-org/alist/v3/internal/op.GetStorageAndActualPath() use storage: /adrive
[37mDEBU[0m[2024-09-24 17:12:44]/home/runner/work/alist/alist/internal/op/fs.go:165 github.com/alist-org/alist/v3/internal/op.Get() op.Get /nas/xn0_83fc79ecadb27f884151ab6a14c766d8-sb5d993497837d64d12d-c1 [37mDEBU[0m[2024-09-24 17:12:44]/home/runner/work/alist/alist/internal/op/fs.go:108 github.com/alist-org/alist/v3/internal/op.List() op.List /nas
[37mDEBU[0m[2024-09-24 17:12:44]/home/runner/work/alist/alist/internal/op/fs.go:112 github.com/alist-org/alist/v3/internal/op.List() use cache when list /nas
[37mDEBU[0m[2024-09-24 17:12:44]/home/runner/work/alist/alist/internal/op/path.go:25 github.com/alist-org/alist/v3/internal/op.GetStorageAndActualPath() use storage: /adrive
[37mDEBU[0m[2024-09-24 17:12:44]/home/runner/work/alist/alist/internal/op/fs.go:165 github.com/alist-org/alist/v3/internal/op.Get() op.Get /nas/xn0_83fc79ecadb27f884151ab6a14c766d8-sb5d993497837d64d12d-c1 [37mDEBU[0m[2024-09-24 17:12:44]/home/runner/work/alist/alist/internal/op/fs.go:108 github.com/alist-org/alist/v3/internal/op.List() op.List /nas
[37mDEBU[0m[2024-09-24 17:12:44]/home/runner/work/alist/alist/internal/op/fs.go:112 github.com/alist-org/alist/v3/internal/op.List() use cache when list /nas
[GIN] 2024/09/24 - 17:12:45 | 200 | 5.017430656s | 192.168.1.165 | GET "/adrive/xn0_75349a1ec6c08c9f5ff1c29cb611ca2d-s637b6d1be0ed53b812d-c1" [37mDEBU[0m[2024-09-24 17:12:45]/home/runner/work/alist/alist/internal/op/path.go:25 github.com/alist-org/alist/v3/internal/op.GetStorageAndActualPath() use storage: /adrive
[37mDEBU[0m[2024-09-24 17:12:45]/home/runner/work/alist/alist/internal/op/fs.go:165 github.com/alist-org/alist/v3/internal/op.Get() op.Get /nas/xn0_c1e747df4576f4eee3c3e9ed49553505-s98e86e9247ebe4cc12d-c1 [37mDEBU[0m[2024-09-24 17:12:45]/home/runner/work/alist/alist/internal/op/fs.go:108 github.com/alist-org/alist/v3/internal/op.List() op.List /nas
[37mDEBU[0m[2024-09-24 17:12:45]/home/runner/work/alist/alist/internal/op/fs.go:112 github.com/alist-org/alist/v3/internal/op.List() use cache when list /nas
[37mDEBU[0m[2024-09-24 17:12:45]/home/runner/work/alist/alist/internal/op/path.go:25 github.com/alist-org/alist/v3/internal/op.GetStorageAndActualPath() use storage: /adrive
[37mDEBU[0m[2024-09-24 17:12:45]/home/runner/work/alist/alist/internal/op/fs.go:165 github.com/alist-org/alist/v3/internal/op.Get() op.Get /nas/xn0_c1e747df4576f4eee3c3e9ed49553505-s98e86e9247ebe4cc12d-c1 [37mDEBU[0m[2024-09-24 17:12:45]/home/runner/work/alist/alist/internal/op/fs.go:108 github.com/alist-org/alist/v3/internal/op.List() op.List /nas
[37mDEBU[0m[2024-09-24 17:12:45]/home/runner/work/alist/alist/internal/op/fs.go:112 github.com/alist-org/alist/v3/internal/op.List() use cache when list /nas
[37mDEBU[0m[2024-09-24 17:12:46]/home/runner/work/alist/alist/internal/net/serve.go:241 github.com/alist-org/alist/v3/internal/net.RequestHttp() <?xml version="1.0" encoding="UTF-8"?>

RequestDeniedByCallback Callback deny this request reason: ExceedMaxConcurrency 66F2828EC5BC5B353081E494 cn-beijing-data.aliyundrive.net 0007-00000209 https://api.aliyun.com/troubleshoot?q=0007-00000209

[GIN] 2024/09/24 - 17:12:46 | 500 | 3.904611197s | 192.168.1.165 | GET "/adrive/xn0_428093f4e285aac68c0e0dedc4845c44-s19584b24968d317212d-c1" [37mDEBU[0m[2024-09-24 17:12:46]/home/runner/work/alist/alist/internal/op/path.go:25 github.com/alist-org/alist/v3/internal/op.GetStorageAndActualPath() use storage: /adrive
[37mDEBU[0m[2024-09-24 17:12:46]/home/runner/work/alist/alist/internal/op/fs.go:165 github.com/alist-org/alist/v3/internal/op.Get() op.Get /nas/xn0_428093f4e285aac68c0e0dedc4845c44-s19584b24968d317212d-c1 [37mDEBU[0m[2024-09-24 17:12:46]/home/runner/work/alist/alist/internal/op/fs.go:108 github.com/alist-org/alist/v3/internal/op.List() op.List /nas
[37mDEBU[0m[2024-09-24 17:12:46]/home/runner/work/alist/alist/internal/op/fs.go:112 github.com/alist-org/alist/v3/internal/op.List() use cache when list /nas
[37mDEBU[0m[2024-09-24 17:12:46]/home/runner/work/alist/alist/internal/op/path.go:25 github.com/alist-org/alist/v3/internal/op.GetStorageAndActualPath() use storage: /adrive
[37mDEBU[0m[2024-09-24 17:12:46]/home/runner/work/alist/alist/internal/op/fs.go:165 github.com/alist-org/alist/v3/internal/op.Get() op.Get /nas/xn0_428093f4e285aac68c0e0dedc4845c44-s19584b24968d317212d-c1 [37mDEBU[0m[2024-09-24 17:12:46]/home/runner/work/alist/alist/internal/op/fs.go:108 github.com/alist-org/alist/v3/internal/op.List() op.List /nas
[37mDEBU[0m[2024-09-24 17:12:46]/home/runner/work/alist/alist/internal/op/fs.go:112 github.com/alist-org/alist/v3/internal/op.List() use cache when list /nas
[37mDEBU[0m[2024-09-24 17:12:46]/home/runner/work/alist/alist/internal/net/serve.go:241 github.com/alist-org/alist/v3/internal/net.RequestHttp() <?xml version="1.0" encoding="UTF-8"?>

RequestDeniedByCallback Callback deny this request reason: ExceedMaxConcurrency 66F2828EC5BC5B35309DE494 cn-beijing-data.aliyundrive.net 0007-00000209 https://api.aliyun.com/troubleshoot?q=0007-00000209

[GIN] 2024/09/24 - 17:12:46 | 500 | 45.723688ms | 192.168.1.165 | GET "/adrive/xn0_428093f4e285aac68c0e0dedc4845c44-s19584b24968d317212d-c1" [37mDEBU[0m[2024-09-24 17:12:46]/home/runner/work/alist/alist/internal/op/path.go:25 github.com/alist-org/alist/v3/internal/op.GetStorageAndActualPath() use storage: /adrive
[37mDEBU[0m[2024-09-24 17:12:46]/home/runner/work/alist/alist/internal/op/fs.go:165 github.com/alist-org/alist/v3/internal/op.Get() op.Get /nas/xn0_428093f4e285aac68c0e0dedc4845c44-s19584b24968d317212d-c1 [37mDEBU[0m[2024-09-24 17:12:46]/home/runner/work/alist/alist/internal/op/fs.go:108 github.com/alist-org/alist/v3/internal/op.List() op.List /nas
[37mDEBU[0m[2024-09-24 17:12:46]/home/runner/work/alist/alist/internal/op/fs.go:112 github.com/alist-org/alist/v3/internal/op.List() use cache when list /nas
[37mDEBU[0m[2024-09-24 17:12:46]/home/runner/work/alist/alist/internal/op/path.go:25 github.com/alist-org/alist/v3/internal/op.GetStorageAndActualPath() use storage: /adrive
[37mDEBU[0m[2024-09-24 17:12:46]/home/runner/work/alist/alist/internal/op/fs.go:165 github.com/alist-org/alist/v3/internal/op.Get() op.Get /nas/xn0_428093f4e285aac68c0e0dedc4845c44-s19584b24968d317212d-c1 [37mDEBU[0m[2024-09-24 17:12:46]/home/runner/work/alist/alist/internal/op/fs.go:108 github.com/alist-org/alist/v3/internal/op.List() op.List /nas
[37mDEBU[0m[2024-09-24 17:12:46]/home/runner/work/alist/alist/internal/op/fs.go:112 github.com/alist-org/alist/v3/internal/op.List() use cache when list /nas
[37mDEBU[0m[2024-09-24 17:12:46]/home/runner/work/alist/alist/drivers/aliyundrive_open/upload.go:270 github.com/alist-org/alist/v3/drivers/aliyundrive_open.(*AliyundriveOpen).upload() [aliyundrive_open] create file success, resp: {FileId:66f28289bb5744f88b1c4f8d9559043367f9939a UploadId:A607AE2508AF4BC49EC8A896872DE407 RapidUpload:false PartInfoList:[{Etag: PartNumber:1 PartSize: UploadUrl:https://cn-beijing-data.aliyundrive.net/VM1TeUGw%2F770522%2F66f28289bb5744f88b1c4f8d9559043367f9939a%2F66f28289708cdd45caec4790ba3d2833aafff8c6?partNumber=1&security-token=CAISvgJ1q6Ft5B2yfSjIr5eEfvbH1ZJAj4mpbU3LvEsCZdpiqvbm1jz2IHhMf3NpBOkZvvQ1lGlU6%2Fcalq5rR4QAXlDfNW%2BcTSidq1HPWZHInuDox55m4cTXNAr%2BIhr%2F29CoEIedZdjBe%2FCrRknZnytou9XTfimjWFrXWv%2Fgy%2BQQDLItUxK%2FcCBNCfpPOwJms7V6D3bKMuu3OROY6Qi5TmgQ41Uh1jgjtPzkkpfFtkGF1GeXkLFF%2B97DRbG%2FdNRpMZtFVNO44fd7bKKp0lQLs0ARrv4r1fMUqW2X543AUgFLhy2KKMPY99xpFgh9a7j0iCbSGyUu%2FhcRm5sw9%2Byfo34lVYneA79W5VyN7uHwufJ7FxfIREfquk63pvSlHLcLPe0Kjzzleo2k1XRPVFF%2B535IaHXuToXDnvSiWRkPdvXtuMkagAFDZdhjZLrO6mcLZKPLq5PfghOOVOL4Gv1wlICyNWdgjSzG8x9JO2infDF3RlkcD9PFZS0MnToziLrq2diEiQ%2FJoemwk%2Fo%2BUCVvikf9%2BBVysb%2FY6Zj%2FhY2vhx3jlCOvOIhE52jjpzccpcPPpbl1mtRWde%2BQ4a%2B9Hmbexl7rOphYJSAA&uploadId=A607AE2508AF4BC49EC8A896872DE407&x-oss-access-key-id=STS.NT15Ls8Ma8KBokzXKWiUNE3M4&x-oss-expires=1727172762&x-oss-signature=L9gMCEv6OfHsZfbQfolbrb1YYu6RYEAtCex7ESmP8Zo%3D&x-oss-signature-version=OSS2 ContentType:} {Etag: PartNumber:2 PartSize: UploadUrl:https://cn-beijing-data.aliyundrive.net/VM1TeUGw%2F770522%2F66f28289bb5744f88b1c4f8d9559043367f9939a%2F66f28289708cdd45caec4790ba3d2833aafff8c6?partNumber=2&security-token=CAISvgJ1q6Ft5B2yfSjIr5eEfvbH1ZJAj4mpbU3LvEsCZdpiqvbm1jz2IHhMf3NpBOkZvvQ1lGlU6%2Fcalq5rR4QAXlDfNW%2BcTSidq1HPWZHInuDox55m4cTXNAr%2BIhr%2F29CoEIedZdjBe%2FCrRknZnytou9XTfimjWFrXWv%2Fgy%2BQQDLItUxK%2FcCBNCfpPOwJms7V6D3bKMuu3OROY6Qi5TmgQ41Uh1jgjtPzkkpfFtkGF1GeXkLFF%2B97DRbG%2FdNRpMZtFVNO44fd7bKKp0lQLs0ARrv4r1fMUqW2X543AUgFLhy2KKMPY99xpFgh9a7j0iCbSGyUu%2FhcRm5sw9%2Byfo34lVYneA79W5VyN7uHwufJ7FxfIREfquk63pvSlHLcLPe0Kjzzleo2k1XRPVFF%2B535IaHXuToXDnvSiWRkPdvXtuMkagAFDZdhjZLrO6mcLZKPLq5PfghOOVOL4Gv1wlICyNWdgjSzG8x9JO2infDF3RlkcD9PFZS0MnToziLrq2diEiQ%2FJoemwk%2Fo%2BUCVvikf9%2BBVysb%2FY6Zj%2FhY2vhx3jlCOvOIhE52jjpzccpcPPpbl1mtRWde%2BQ4a%2B9Hmbexl7rOphYJSAA&uploadId=A607AE2508AF4BC49EC8A896872DE407&x-oss-access-key-id=STS.NT15Ls8Ma8KBokzXKWiUNE3M4&x-oss-expires=1727172762&x-oss-signature=bhgZ72eSK6Ti3Q8qHMnKPl9sNRqyVIDVU4mlHv7XVoA%3D&x-oss-signature-version=OSS2 ContentType:}]} [37mDEBU[0m[2024-09-24 17:12:46]/home/runner/work/alist/alist/internal/net/serve.go:241 github.com/alist-org/alist/v3/internal/net.RequestHttp() <?xml version="1.0" encoding="UTF-8"?>

welcome[bot] commented 1 month ago

Thanks for opening your first issue here! Be sure to follow the issue template!

jibwf commented 1 month ago

s3_http500_debuglog.log.zip

jibwf commented 3 weeks ago

没有其他人遇到这个问题嘛？

roblues commented 3 weeks ago

遇到同样的情况。事实上是阿里云对最大连接数做了限制，超出之后就会拒绝连接。适应就好。

jibwf commented 3 weeks ago

我的想法是不是可以在alist上对阿里云访问进行限制或者排队，保持2个连接.

AlistGo / alist