run pipeline error: Forbidden

vanessayang819 commented 2 years ago

MacBook M1 pro

OS: macOS 12.1
nodejs: 14.17.0

sudo pipcook train ./objDect.json # => run pipeline error: Forbidden

{
  "specVersion": "2.0",
  "type": "ObjectDetection",
  "datasource": "https://cdn.jsdelivr.net/gh/imgcook/pipcook-script@59d3e2f/scripts/object-detection-yolo/build/datasource.js?format=pascalvoc&url=https://pipcook-cloud.oss-cn-hangzhou.aliyuncs.com/dataset/mask.zip",
  "dataflow": [
    "https://cdn.jsdelivr.net/gh/imgcook/pipcook-script@59d3e2f/scripts/object-detection-yolo/build/dataflow.js?size=416&size=416"
  ],
  "model": "https://cdn.jsdelivr.net/gh/imgcook/pipcook-script@59d3e2f/scripts/object-detection-yolo/build/model.js",
  "options": {
    "framework": "tfjs@3.8",
    "train": {
      "epochs": 100,
      "batchSize": 16,
      "patience": 10
    }
  }
}

FeelyChau commented 2 years ago

The jsdelivr is not so stable, sometimes you may need to try several times. If it still fails, you can add the --debug parameter to get the detailed error stack.

lilee commented 1 year ago

The jsdelivr is not so stable, sometimes you may need to try several times. If it still fails, you can add the --debug parameter to get the detailed error stack.

我的机器也是M1 Pro芯片应该不是jsdelivr的问题，我把datasource,dataflow和model全部删掉，运行时还是报这个错误。还没到下载脚本的环节

FeelyChau commented 1 year ago

The jsdelivr is not so stable, sometimes you may need to try several times. If it still fails, you can add the --debug parameter to get the detailed error stack.

我的机器也是M1 Pro芯片应该不是jsdelivr的问题，我把datasource,dataflow和model全部删掉，运行时还是报这个错误。还没到下载脚本的环节

可以加个 -d 参数看下具体的错误栈。

lilee commented 1 year ago

[03:26:08:600] ✖ run pipeline error: StatusError: Forbidden
    at ClientRequest.<anonymous> (/Users/laelli/.nvm/versions/node/v14.17.0/lib/node_modules/@pipcook/cli/node_modules/bent/src/nodejs.js:133:23)
    at Object.onceWrapper (events.js:483:26)
    at ClientRequest.emit (events.js:376:20)
    at HTTPParser.parserOnIncomingClient (_http_client.js:647:27)
    at HTTPParser.parserOnHeadersComplete (_http_common.js:126:17)
    at TLSSocket.socketOnData (_http_client.js:515:22)
    at TLSSocket.emit (events.js:376:20)
    at addChunk (internal/streams/readable.js:309:12)
    at readableAddChunk (internal/streams/readable.js:284:9)
    at TLSSocket.Readable.push (internal/streams/readable.js:223:10)

FeelyChau commented 1 year ago

[03:26:08:600] ✖ run pipeline error: StatusError: Forbidden
    at ClientRequest.<anonymous> (/Users/laelli/.nvm/versions/node/v14.17.0/lib/node_modules/@pipcook/cli/node_modules/bent/src/nodejs.js:133:23)
    at Object.onceWrapper (events.js:483:26)
    at ClientRequest.emit (events.js:376:20)
    at HTTPParser.parserOnIncomingClient (_http_client.js:647:27)
    at HTTPParser.parserOnHeadersComplete (_http_common.js:126:17)
    at TLSSocket.socketOnData (_http_client.js:515:22)
    at TLSSocket.emit (events.js:376:20)
    at addChunk (internal/streams/readable.js:309:12)
    at readableAddChunk (internal/streams/readable.js:284:9)
    at TLSSocket.Readable.push (internal/streams/readable.js:223:10)

应该是 framework 没有 M1 的实现，现在 tfjs 应该已经支持 m1 了，更新下 tfjs 版本应该就可以了。在我们更新验证完之前，你可以试试下载 https://pipcook-cloud.oss-cn-hangzhou.aliyuncs.com/framework/node14-py37/tfjs%403.8-darwin-x64-v8.zip ，解压后是一个 npm package，将 tfjs 升级成支持 M1 的版本，然后重新压缩成 zip。对应的 pipeline 改成:

{
"options": {
    "framework": "file:///path/to/your/tfjs.zip",
}

应该就可以绕过这个错误了。

lilee commented 1 year ago

[03:26:08:600] ✖ run pipeline error: StatusError: Forbidden
    at ClientRequest.<anonymous> (/Users/laelli/.nvm/versions/node/v14.17.0/lib/node_modules/@pipcook/cli/node_modules/bent/src/nodejs.js:133:23)
    at Object.onceWrapper (events.js:483:26)
    at ClientRequest.emit (events.js:376:20)
    at HTTPParser.parserOnIncomingClient (_http_client.js:647:27)
    at HTTPParser.parserOnHeadersComplete (_http_common.js:126:17)
    at TLSSocket.socketOnData (_http_client.js:515:22)
    at TLSSocket.emit (events.js:376:20)
    at addChunk (internal/streams/readable.js:309:12)
    at readableAddChunk (internal/streams/readable.js:284:9)
    at TLSSocket.Readable.push (internal/streams/readable.js:223:10)
应该是 framework 没有 M1 的实现，现在 tfjs 应该已经支持 m1 了，更新下 tfjs 版本应该就可以了。在我们更新验证完之前，你可以试试下载 https://pipcook-cloud.oss-cn-hangzhou.aliyuncs.com/framework/node14-py37/tfjs%403.8-darwin-x64-v8.zip ，解压后是一个 npm package，将 tfjs 升级成支持 M1 的版本，然后重新压缩成 zip。对应的 pipeline 改成:
{
"options": {
    "framework": "file:///path/to/your/tfjs.zip",
}
应该就可以绕过这个错误了。

我将zip包解压后，修改framework/framework.json中的arch为amd64，然后压缩，修改options.framework路径。最后运行，还是会报错，报错如下

platform darwin does not support gpu, enable backend `@tensorflow/tfjs-node`.
Platform node has already been set. Overwriting the platform with node.
[05:57:18:037] ✖ run pipeline error: Error: dlopen(/Users/laelli/Develop/test/pipcook/output/framework/node_modules/_@tensorflow_tfjs-node@3.8.0@@tensorflow/tfjs-node/lib/napi-v8/tfjs_binding.node, 0x0001): tried: '/Users/laelli/Develop/test/pipcook/output/framework/node_modules/_@tensorflow_tfjs-node@3.8.0@@tensorflow/tfjs-node/lib/napi-v8/tfjs_binding.node' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64')), '/System/Volumes/Preboot/Cryptexes/OS/Users/laelli/Develop/test/pipcook/output/framework/node_modules/_@tensorflow_tfjs-node@3.8.0@@tensorflow/tfjs-node/lib/napi-v8/tfjs_binding.node' (no such file), '/Users/laelli/Develop/test/pipcook/output/framework/node_modules/_@tensorflow_tfjs-node@3.8.0@@tensorflow/tfjs-node/lib/napi-v8/tfjs_binding.node' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64'))
    at Object.Module._extensions..node (node:internal/modules/cjs/loader:1210:18)
    at Module.load (node:internal/modules/cjs/loader:1004:32)
    at Function.Module._load (node:internal/modules/cjs/loader:839:12)
    at Module.require (node:internal/modules/cjs/loader:1028:19)
    at require (node:internal/modules/cjs/helpers:102:18)
    at Object.<anonymous> (/Users/laelli/Develop/test/pipcook/output/framework/node_modules/_@tensorflow_tfjs-node@3.8.0@@tensorflow/tfjs-node/dist/index.js:60:16)
    at Module._compile (node:internal/modules/cjs/loader:1126:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1180:10)
    at Module.load (node:internal/modules/cjs/loader:1004:32)
    at Function.Module._load (node:internal/modules/cjs/loader:839:12)

lilee commented 1 year ago

OK了，重新yarn一下

leozhang007 commented 1 year ago

这个问题还没有修复，mac m1 型片，报错：run pipeline error: Forbidden，有修复计划么？

FeelyChau commented 1 year ago

这个问题还没有修复，mac m1 型片，报错：run pipeline error: Forbidden，有修复计划么？

已经更新了 framework，现在应该可以在 m1 上的 node 16 环境下正常运行了。

FXHDHR commented 1 year ago

这个问题还没有修复，mac m1 型片，报错：run pipeline error: Forbidden，有修复计划么？

已经更新了 framework，现在应该可以在 m1 上的 node 16 环境下正常运行了。

我这里还是不行..，运行的命令是：pipcook train https://cdn.jsdelivr.net/gh/alibaba/pipcook@main/example/pipelines/image-classification-mobilenet.json -o ./output -d

vanessaZh commented 8 months ago

这个问题好像还没解决？ https://pipcook-cloud.oss-cn-hangzhou.aliyuncs.com/framework/node16-py37/node_modules%2F%40tensorflow%2Ftfsj-darwin-arm64-v8.zip 这个地址请求为403

cloudOfWorld commented 8 months ago

这个问题好像还没解决？ https://pipcook-cloud.oss-cn-hangzhou.aliyuncs.com/framework/node16-py37/node_modules%2F%40tensorflow%2Ftfsj-darwin-arm64-v8.zip 这个地址请求为403

+1

alibaba / pipcook

run pipeline error: Forbidden #931