[Help] s3 使用 contabo 失败

1PercentSync commented 3 months ago

这家在其他软件中需要选中强制路径样式，是否和这个有关系

1PercentSync commented 3 months ago

后端log

{
  "outcome": "ok",
  "scriptVersion": {
    "id": "a60eabfa-45dd-499e-b3e5-53e489b48eb2"
  },
  "scriptName": "rin-server",
  "diagnosticsChannelEvents": [],
  "exceptions": [],
  "logs": [
    {
      "message": [
        "char 'e' is not expected.:1:1\n  Deserialization error: to see the raw response, inspect the hidden field {error}.$response on this object."
      ],
      "level": "error",
      "timestamp": 1717819383229
    }
  ],
  "eventTimestamp": 1717819383200,
  "event": {
    "request": {
      "url": "https://rin-server.1percentsync.games/storage",
      "method": "POST",
      "headers": {
        "accept": "*/*",
        "accept-encoding": "gzip, br",
        "accept-language": "zh-CN,zh-TW;q=0.9,zh;q=0.8,en-US;q=0.7,en;q=0.6",
        "authorization": "REDACTED",
        "cf-connecting-ip": "2601:19c:5000:120:d92b:b6d5:bc89:7fae",
        "cf-ipcountry": "US",
        "cf-ray": "8905fd66cc2e9e1a",
        "cf-visitor": "{\"scheme\":\"https\"}",
        "connection": "Keep-Alive",
        "content-length": "1865858",
        "content-type": "multipart/form-data; boundary=----WebKitFormBoundaryfIfBdjZv9QozZbOz",
        "dnt": "1",
        "host": "rin-server.1percentsync.games",
        "origin": "https://rin.1percentsync.games",
        "priority": "u=1, i",
        "referer": "https://rin.1percentsync.games/",
        "sec-ch-ua": "\"Microsoft Edge\";v=\"125\", \"Chromium\";v=\"125\", \"Not.A/Brand\";v=\"24\"",
        "sec-ch-ua-mobile": "?0",
        "sec-ch-ua-platform": "\"Windows\"",
        "sec-fetch-dest": "empty",
        "sec-fetch-mode": "cors",
        "sec-fetch-site": "same-site",
        "user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.0.0 Safari/537.36 Edg/125.0.0.0",
        "x-forwarded-proto": "https",
        "x-real-ip": "2601:19c:5000:120:d92b:b6d5:bc89:7fae"
      },
      "cf": {
        "longitude": "-71.19840",
        "httpProtocol": "HTTP/3",
        "tlsCipher": "AEAD-AES128-GCM-SHA256",
        "continent": "NA",
        "asn": 7922,
        "clientAcceptEncoding": "gzip, deflate, br, zstd",
        "country": "US",
        "verifiedBotCategory": "",
        "tlsClientAuth": {
          "certIssuerDNLegacy": "",
          "certIssuerSKI": "",
          "certSubjectDNRFC2253": "",
          "certSubjectDNLegacy": "",
          "certFingerprintSHA256": "",
          "certNotBefore": "",
          "certSKI": "",
          "certSerial": "",
          "certIssuerDN": "",
          "certVerified": "NONE",
          "certNotAfter": "",
          "certSubjectDN": "",
          "certPresented": "0",
          "certRevoked": "0",
          "certIssuerSerial": "",
          "certIssuerDNRFC2253": "",
          "certFingerprintSHA1": ""
        },
        "tlsExportedAuthenticator": {
          "clientFinished": "fbbf24a818ea0b409eef0839b331f7f3bb902b7139b4cf4f1c51fda6fcb67b8b",
          "clientHandshake": "abdc5186fac35c791a78fa65ff2030401432028672dc2c8cfcd5d79b271e15ff",
          "serverHandshake": "8e9c977f06ee9cf272f8100e0fd5a54104716f64705feee9b8b171cae0a723a1",
          "serverFinished": "255c8fd35d1f609dca0f2355a4b5b8b91ee23e5b6a648eaae884399e155ebacd"
        },
        "tlsVersion": "TLSv1.3",
        "city": "Burlington",
        "timezone": "America/New_York",
        "colo": "EWR",
        "tlsClientHelloLength": "2075",
        "edgeRequestKeepAliveStatus": 1,
        "postalCode": "01803",
        "region": "Massachusetts",
        "latitude": "42.50900",
        "requestPriority": "",
        "regionCode": "MA",
        "asOrganization": "Comcast Cable",
        "metroCode": "506",
        "tlsClientExtensionsSha1": "QgXfefoqdcH2v14yfHZiKrQykVs=",
        "tlsClientRandom": "jxiiDr9bE7oOsbcPuwa4QsFKcv60WSQnt0qeNqwGjJ4=",
        "botManagement": {
          "corporateProxy": false,
          "verifiedBot": false,
          "jsDetection": {
            "passed": false
          },
          "staticResource": false,
          "detectionIds": {},
          "score": 99
        }
      }
    },
    "response": {
      "status": 400
    }
  },
  "id": 1
}

1PercentSync commented 3 months ago

需要contabo的s3测试的话我可以提供

1PercentSync commented 3 months ago

nope,就叫https://contabo.com/

OXeu commented 3 months ago

nope,就叫https://contabo.com/

抱歉😂

1PercentSync commented 3 months ago

测试了backblaze是没问题的

OXeu commented 3 months ago

没有使用过这家的服务，晚点我去看看他们的文档

1PercentSync commented 3 months ago

没有使用过这家的服务，晚点我去看看他们的文档

来自gpt补充的可能相关信息：在某些情况下，需要选中S3的强制路径样式（Path Style）是因为路径样式在特定的网络配置或旧版S3 API兼容性问题中能够确保更好的兼容性和连接成功。主要原因包括：

DNS解析问题：某些网络环境下，对象存储的虚拟主机（Virtual Host）样式的DNS解析可能会失败，而路径样式不依赖于DNS解析，因此可以避免这些问题。
兼容性：一些旧版的S3 SDK或工具可能不支持虚拟主机样式，只能使用路径样式进行访问。
跨区域访问：在某些跨区域访问的场景中，路径样式可能会比虚拟主机样式更稳定。

具体情况下，选择路径样式还是虚拟主机样式应根据实际需求和环境来决定。

1PercentSync commented 3 months ago

https://github.com/awslabs/aws-sdk-rust/issues/390#issue-1097160885

OXeu commented 3 months ago

你可以先在自己的仓库中修改 server/src/services/storage.ts 中 https://github.com/OXeu/Rin/blob/9b23ab6e09ba3d9149e06b3d6e4875c0c4fc3f57/server/src/services/storage.ts#L22-L29

添加一行参数forcePathStyle: true,如下所示：

    const s3 = new S3Client({
        region: region,
        endpoint: endpoint,
        forcePathStyle: true,
        credentials: {
            accessKeyId: accessKeyId,
            secretAccessKey: secretAccessKey,
        }
    });

然后测试是否能够正常工作，如果可以的话我后续加一个环境变量来实现这个

OXeu commented 3 months ago

你可以先在自己的仓库中修改 server/src/services/storage.ts 中

https://github.com/OXeu/Rin/blob/9b23ab6e09ba3d9149e06b3d6e4875c0c4fc3f57/server/src/services/storage.ts#L22-L29

添加一行参数forcePathStyle: true,如下所示：
    const s3 = new S3Client({
        region: region,
        endpoint: endpoint,
        forcePathStyle: true,
        credentials: {
            accessKeyId: accessKeyId,
            secretAccessKey: secretAccessKey,
        }
    });
然后测试是否能够正常工作，如果可以的话我后续加一个环境变量来实现这个

同时也欢迎 PR

1PercentSync commented 3 months ago

可以，成功了，我研究研究怎么整环境变量

1PercentSync commented 3 months ago

有个问题：我修改了两处，一个是 storage.ts: export const StorageService = (db: DB, env: Env) => { const region = env.S3_REGION; const endpoint = env.S3_ENDPOINT; const accessKeyId = env.S3_ACCESS_KEY_ID; const secretAccessKey = env.S3_SECRET_ACCESS_KEY; const accessHost = env.S3_ACCESS_HOST || endpoint; const bucket = env.S3_BUCKET; const folder = env.S3_FOLDER || ''; const forcePathStyle = env.FORCE_PATH_STYLE || false; //添加了FORCE_PATH_STYLE const s3 = new S3Client({ region: region, endpoint: endpoint, forcePathStyle: forcePathStyle, credentials: { accessKeyId: accessKeyId, secretAccessKey: secretAccessKey } });

然后在db.ts中添加了： export interface Env { DB: D1Database; RIN_GITHUB_CLIENT_ID: string; RIN_GITHUB_CLIENT_SECRET: string; GITHUB_CLIENT_ID: string; GITHUB_CLIENT_SECRET: string; JWT_SECRET: string; FRONTEND_URL: string; S3_REGION: string, S3_ENDPOINT: string, S3_ACCESS_KEY_ID: string, S3_SECRET_ACCESS_KEY: string, S3_ACCESS_HOST: string, S3_BUCKET: string, S3_FOLDER: string, S3_CACHE_FOLDER: string, WEBHOOK_URL: string, FORCE_PATH_STYLE: boolean, //这一行 } 然而配置完环境变量部署是不生效的

1PercentSync commented 3 months ago

环境变量是FORCE_PATH_STYLE=true

OXeu commented 3 months ago

添加的环境变量应以 S3_ 开头以便于确认用途，推荐使用 S3_FORCE_PATH_STYLE 作为环境变量名称
添加一个新的环境变量需要修改以下地方：

在使用处添加环境变量的引用变量并根据情况配置合适的默认值，此处建议：https://github.com/OXeu/Rin/blob/9b23ab6e09ba3d9149e06b3d6e4875c0c4fc3f57/server/src/services/storage.ts#L15-L21
```
const forcePathStyle = env.S3_FORCE_PATH_STYLE === "1" || false;
```
在 Env 类型处声明对应的环境变量，类型统一为 string（使用 string 而不是 boolean 防止出现错误的类型转换导致意料之外的结果）： https://github.com/OXeu/Rin/blob/9b23ab6e09ba3d9149e06b3d6e4875c0c4fc3f57/server/src/db/db.ts#L3-L20
在部署的脚本 scripts/migrator.ts 使用并注入环境变量，明文需要放置到生成 wrangler.toml 的样板文本中，密文需在 Put secrets 流程处执行添加 https://github.com/OXeu/Rin/blob/9b23ab6e09ba3d9149e06b3d6e4875c0c4fc3f57/scripts/migrator.ts#L21-L26 https://github.com/OXeu/Rin/blob/9b23ab6e09ba3d9149e06b3d6e4875c0c4fc3f57/scripts/migrator.ts#L47-L53 https://github.com/OXeu/Rin/blob/9b23ab6e09ba3d9149e06b3d6e4875c0c4fc3f57/scripts/migrator.ts#L112-L129
最后在 CI 文件 .github/workflows/deploy.yaml 中为其注入在 Github 中配置好的环境变量： https://github.com/OXeu/Rin/blob/9b23ab6e09ba3d9149e06b3d6e4875c0c4fc3f57/.github/workflows/deploy.yaml#L40-L45

同时，由于 SEO 也需要使用 S3 存储，你还需要修改 srcipts/render.ts 和 .github/workflows/seo.yaml 确保 SEO 也能正常运行 https://github.com/OXeu/Rin/blob/9b23ab6e09ba3d9149e06b3d6e4875c0c4fc3f57/scripts/render.ts#L8-L14 https://github.com/OXeu/Rin/blob/9b23ab6e09ba3d9149e06b3d6e4875c0c4fc3f57/.github/workflows/seo.yaml#L35-L43

openRin / Rin

[Help] s3 使用 contabo 失败 #64