ValueError: (InvalidArgument) The axis is expected to be in range of [0, 0), but got 0

[firaterdem]

问题确认 Search before asking

• [X] 我已经查询历史issue(包括open与closed)，没有发现相似的bug。I have searched the open and closed issues and found no similar bug report.

Bug描述 Describe the Bug

In Google Colab Quick Start code is giving the following error message at the training part. How can I solve it?

2023-07-26 12:56:35 [INFO] [TRAIN] epoch: 8, iter: 500/1000, loss: 0.0543, lr: 0.005369, batch_cost: 0.1380, reader_cost: 0.04589, ips: 28.9826 samples/sec | ETA 00:01:09 2023-07-26 12:56:35 [INFO] Start evaluating (total_samples: 76, total_iters: 76)... Traceback (most recent call last): File "/content/drive/MyDrive/SmartPol_PaddleSeg/PaddleSeg/tools/train.py", line 195, in <module> main(args) File "/content/drive/MyDrive/SmartPol_PaddleSeg/PaddleSeg/tools/train.py", line 170, in main train( File "/usr/local/lib/python3.10/dist-packages/paddleseg/core/train.py", line 315, in train mean_iou, acc, _, _, _ = evaluate( File "/usr/local/lib/python3.10/dist-packages/paddleseg/core/val.py", line 161, in evaluate intersect_area, pred_area, label_area = metrics.calculate_area( File "/usr/local/lib/python3.10/dist-packages/paddleseg/utils/metrics.py", line 57, in calculate_area pred_area = paddle.concat(pred_area) File "/usr/local/lib/python3.10/dist-packages/paddle/tensor/manipulation.py", line 1121, in concat return _C_ops.concat(input, axis) ValueError: (InvalidArgument) The axis is expected to be in range of [0, 0), but got 0 [Hint: Expected axis >= -rank && axis < rank == true, but received axis >= -rank && axis < rank:0 != true:1.] (at ../paddle/phi/infermeta/multiary.cc:954)

复现环境 Environment

• OS: Linux (Google Colab)
• PaddlePaddle: 2.5.0
• PaddleSeg: 2.8.0
• Python: 3.10.6
• CUDA: 11.8

Bug描述确认 Bug description confirmation

• [X] 我确认已经提供了Bug复现步骤、代码改动说明、以及环境信息，确认问题是可以复现的。I confirm that the bug replication steps, code change instructions, and environment information have been provided, and the problem can be reproduced.

是否愿意提交PR？ Are you willing to submit a PR?

• [ ] 我愿意提交PR！I'd like to help by submitting a PR!

[Asthestarsfalll]

You can attempt to fix the bug using this method, but finding the root cause of the issue will require additional effort.

[firaterdem]

You can attempt to fix the bug using this method, but finding the root cause of the issue will require additional effort.

@Asthestarsfalll I corrected the related part of the code ("/usr/local/lib/python3.10/dist-packages/paddleseg/utils/metrics.py") as suggested:

pred_area.append(paddle.sum(paddle.cast(pred_i, "int64")).unsqueeze(0))

But it is still giving the same error.

[Asthestarsfalll]

@firaterdem Can you share the Colab notebook so that I can investigate the reason?

[firaterdem]

@firaterdem Can you share the Colab notebook so that I can investigate the reason?

@Asthestarsfalll sure, here is the notebook:

https://colab.research.google.com/drive/1Uu4eUHcDNA7LThE5KGexG5HdEZR-vB2t?usp=sharing

[Asthestarsfalll]

@firaterdem I install paddleseg2.8 through source code instead of pip, It works well:

You can try this to install paddleseg

git clone https://github.com/PaddlePaddle/PaddleSeg
cd PaddleSeg
pip install -r requirements.txt
python setup.py install

[firaterdem]

@firaterdem I install paddleseg2.8 through source code instead of pip, It works well:

You can try this to install paddleseg

git clone https://github.com/PaddlePaddle/PaddleSeg
cd PaddleSeg
pip install -r requirements.txt
python setup.py install

@Asthestarsfalll Yes, installing paddleseg in this way solved the problem. Thank you very much!

[ToddBear]

以上回答已经充分解答了问题，如果有新的问题欢迎随时提交issue，或者在此条issue下继续回复～ 我们开启了飞桨套件的ISSUE攻关活动，欢迎感兴趣的开发者参加：https://github.com/PaddlePaddle/PaddleOCR/issues/10223

[Jerry3062]

@firaterdem I install paddleseg2.8 through source code instead of pip, It works well: You can try this to install paddleseg

git clone https://github.com/PaddlePaddle/PaddleSeg
cd PaddleSeg
pip install -r requirements.txt
python setup.py install

@Asthestarsfalll Yes, installing paddleseg in this way solved the problem. Thank you very much!

I have try this method, but not solve the problem. How can I solve this.

[Asthestarsfalll]

I have try this method, but not solve the problem. How can I solve this.

should give more details

[Jerry3062]

I have try this method, but not solve the problem. How can I solve this.

should give more details

I change my code from gitee to github and solve the problem, thanks.

[jason660519]

我也遇到同樣的問題，已經按照上面的步驟無論是重新安裝或downgrad版本重跑數變，都是一樣的問題: ValueError: (InvalidArgument) The axis is expected to be in range of [0, 0), but got 0 [Hint: Expected axis >= -rank && axis < rank == true, but received axis >= -rank && axis < rank:0 != true:1.] (at ..\paddle\phi\infermeta\multiary.cc:961)

我是follow 官方教程:https://github.com/PaddlePaddle/PaddleSeg/blob/release/2.8/docs/quick_start_cn.md

I0924 22:43:01.913477 15680 eager_method.cc:140] Warning:: 0D Tensor cannot be used as 'Tensor.numpy()[0]' . In order to avoid this problem, 0D Tensor will be changed to 1D numpy currently, but it's not correct and will be removed in release 2.6. For Tensor contain only one element, Please modify 'Tensor.numpy()[0]' to 'float(Tensor)' as soon as possible, otherwise 'Tensor.numpy()[0]' will raise error in release 2.6. 2023-09-24 22:43:01 [INFO] [TRAIN] epoch: 8, iter: 500/1000, loss: 0.0397, lr: 0.005369, batch_cost: 0.1381, reader_cost: 0.04371, ips: 28.9607 samples/sec | ETA 00:01:09 2023-09-24 22:43:01 [INFO] Start evaluating (total_samples: 76, total_iters: 76)... Traceback (most recent call last): File "C:\Users\a0922\Desktop\paddlepaddle\PaddleSeg\tools\train.py", line 195, in main(args) File "C:\Users\a0922\Desktop\paddlepaddle\PaddleSeg\tools\train.py", line 170, in main train( File "C:\Users\a0922\Desktop\paddlepaddle\Lib\site-packages\paddleseg\core\train.py", line 315, in train meaniou, acc, , , = evaluate( ^^^^^^^^^ File "C:\Users\a0922\Desktop\paddlepaddle\Lib\site-packages\paddleseg\core\val.py", line 161, in evaluate intersect_area, pred_area, label_area = metrics.calculate_area( ^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\a0922\Desktop\paddlepaddle\Lib\site-packages\paddleseg\utils\metrics.py", line 57, in calculate_area pred_area = paddle.concat(pred_area) ^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\a0922\Desktop\paddlepaddle\Lib\site-packages\paddle\tensor\manipulation.py", line 1121, in concat return _C_ops.concat(input, axis) ^^^^^^^^^^^^^^^^^^^^^^^^^^ ValueError: (InvalidArgument) The axis is expected to be in range of [0, 0), but got 0 [Hint: Expected axis >= -rank && axis < rank == true, but received axis >= -rank && axis < rank:0 != true:1.] (at ..\paddle\phi\infermeta\multiary.cc:961)

[Asthestarsfalll]

@jason660519

是按照这种方式安装的吗，请确认安装成功了，而不是使用的pypi源当中的paddleseg

git clone https://github.com/PaddlePaddle/PaddleSeg
cd PaddleSeg
pip install -r requirements.txt
python setup.py install

[jason660519]

我是按照官方網站的教學: https://github.com/PaddlePaddle/PaddleSeg/blob/release/2.8/docs/quick_start_cn.md 一步一步重複做了很多次， 每次在第三步驟執行: python tools/train.py --config configs/quick_start/pp_liteseg_optic_disc_512x512_1k.yml --save_interval 500 --do_eval --use_vdl --save_dir output 之後，都是一樣的結果。

我的硬體與安裝包如下: OS:Windows 10, AMD CPU, NVIDIA RTX 3090 24G GPU Compute Capability: 8.6 Driver API Version: 12.2, Runtime API Version: 12.0 cuDNN Version: 8.9. PaddlePaddle works well on 1 GPU. PaddlePaddle: 2.5.1 Paddleseg: 2.8.0

錯誤碼如下:

Warning:: 0D Tensor cannot be used as 'Tensor.numpy()[0]' . In order to avoid this problem, 0D Tensor will be changed to 1D numpy currently, but it's not correct and will be removed in release 2.6. For Tensor contain only one element, Please modify 'Tensor.numpy()[0]' to 'float(Tensor)' as soon as possible, otherwise 'Tensor.numpy()[0]' will raise error in release 2.6. (這一段的警告提示，應該無關訓練)

中止執行的時間點就是ve_interval 500的地方， iter設500 iter就停在500(epoch約跑到:8) iter設1000 iter就停在1000(epoch約跑到:17) Iter設10000 iter就停在10000(epoch約跑到:?? 截稿前，還在跑....)

epoch: 8, iter: 500/1000, loss: 0.0478, lr: 0.005369, batch_cost: 0.1344, reader_cost: 0.05471, ips: 29.7619 samples/sec | ETA 00:01:07 2023-09-25 16:04:53 [INFO] Start evaluating (total_samples: 76, total_iters: 76)... Traceback (most recent call last): File "C:\Users\a0922\Desktop\paddlepaddle\paddleseg\tools\train.py", line 195, in main(args) File "C:\Users\a0922\Desktop\paddlepaddle\paddleseg\tools\train.py", line 170, in main train( File "C:\Users\a0922\Desktop\paddlepaddle\Lib\site-packages\paddleseg\core\train.py", line 315, in train meaniou, acc, , , = evaluate( ^^^^^^^^^ File "C:\Users\a0922\Desktop\paddlepaddle\Lib\site-packages\paddleseg\core\val.py", line 161, in evaluate intersect_area, pred_area, label_area = metrics.calculate_area( ^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\a0922\Desktop\paddlepaddle\Lib\site-packages\paddleseg\utils\metrics.py", line 57, in calculate_area pred_area = paddle.concat(pred_area) ^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\a0922\Desktop\paddlepaddle\Lib\site-packages\paddle\tensor\manipulation.py", line 1121, in concat return _C_ops.concat(input, axis) ^^^^^^^^^^^^^^^^^^^^^^^^^^ ValueError: (InvalidArgument) The axis is expected to be in range of [0, 0), but got 0 [Hint: Expected axis >= -rank && axis < rank == true, but received axis >= -rank && axis < rank:0 != true:1.] (at ..\paddle\phi\infermeta\multiary.cc:961)

[Asthestarsfalll]

@jason660519 不要使用pip安装，确保本地的包是通过python setup.py安装上的

[jason660519]

用 python setup.py 安裝不了，安裝到一半會跳出錯誤碼...

[Asthestarsfalll]

用 python setup.py 安裝不了，安裝到一半會跳出錯誤碼...

那就使用更低版本的paddle试试

[jason660519]

(paddle2-4) PS C:\Users\a0922\Desktop\paddle2-4> python -m pip install paddlepaddle-gpu==2.4.2.post117 -f https://www.paddlepaddle.org.cn/whl/windows/mkl/avx/stable.html Looking in links: https://www.paddlepaddle.org.cn/whl/windows/mkl/avx/stable.html ERROR: Could not find a version that satisfies the requirement paddlepaddle-gpu==2.4.2.post117 (from versions: 2.5.1, 2.5.1.post102, 2.5.1.post112, 2.5.1.post116, 2.5.1.post117, 2.5.1.post120) ERROR: No matching distribution found for paddlepaddle-gpu==2.4.2.post117 (paddle2-4) PS C:\Users\a0922\Desktop\paddle2-4>

PS C:\Users\a0922\Desktop\paddle2-4> python -m pip install paddlepaddle-gpu==2.3.2.post116 -f https://www.paddlepaddle.org.cn/whl/windows/mkl/avx/stable.html Looking in links: https://www.paddlepaddle.org.cn/whl/windows/mkl/avx/stable.html ERROR: Could not find a version that satisfies the requirement paddlepaddle-gpu==2.3.2.post116 (from versions: 2.5.1, 2.5.1.post102, 2.5.1.post112, 2.5.1.post116, 2.5.1.post117, 2.5.1.post120) ERROR: No matching distribution found for paddlepaddle-gpu==2.3.2.post116

即使降了一代版本，官網回應satisfy的paddle版本全都2.5.1版本以上 2.4.2 版本，官網系統回應找不到該版本。 2.3.2 版本，官網系統回應找不到該版本。 2.4.2 CPU的版本雖可以安裝...但PaddleSeg會裝不了。 已經一個禮拜了...真想放棄!

[Asthestarsfalll]

@jason660519 请给出本地安装paddleseg的报错信息

[yuzu16]

方式安装paddleseg解决了这个问题。非常

解决了吗

[yuzu16]

@jason660519

是按照这种方式安装的吗，请确认安装成功了，而不是使用的pypi源使用的paddleseg

git clone https://github.com/PaddlePaddle/PaddleSeg
cd PaddleSeg
pip install -r requirements.txt
python setup.py install

这种方式安装的还是没有解决，

[yuzu16]

我已经尝试过这个方法，但没有解决问题。我该如何解决这个问题。

应该提供更多细节

我也是源代码下载 也是没解决问题 版本是2.8.0 是因为版本问题吗？

[FaberSober]

跟着官网的步骤，同样遇到了这个问题。

[Asthestarsfalll]

跟着官网的步骤，同样遇到了这个问题。

使用更新的版本2.9呢？

[pvinhd]

I have try this method, but not solve the problem. How can I solve this.

should give more details

I change my code from gitee to github and solve the problem, thanks.

where you change this code

[messy2333]

@firaterdem I install paddleseg2.8 through source code instead of pip, It works well:

You can try this to install paddleseg

git clone https://github.com/PaddlePaddle/PaddleSeg
cd PaddleSeg
pip install -r requirements.txt
python setup.py install

我按照您的方法重新装了paddleseg，还是报了这个错误，似乎不是metric的错误 Traceback (most recent call last): File "/home/PaddleSeg/contrib/QualityInspector/tools/det/train.py", line 197, in main() File "/home/PaddleSeg/contrib/QualityInspector/tools/det/train.py", line 193, in main run(FLAGS, cfg) File "/home/PaddleSeg/contrib/QualityInspector/tools/det/train.py", line 146, in run trainer.train(FLAGS.eval) File "/usr/local/lib/python3.10/dist-packages/ppdet/engine/trainer.py", line 539, in train outputs = model(data) File "/usr/local/lib/python3.10/dist-packages/paddle/nn/layer/layers.py", line 1429, in call return self.forward(*inputs, *kwargs) File "/usr/local/lib/python3.10/dist-packages/ppdet/modeling/architectures/meta_arch.py", line 60, in forward out = self.get_loss() File "/usr/local/lib/python3.10/dist-packages/ppdet/modeling/architectures/fasterrcnn.py", line 112, in get loss rpn_loss, bbox_loss = self._forward() File "/usr/local/lib/python3.10/dist-packages/ppdet/modeling/architectures/faster_rcnn.py", line 80, in _forw ard rois, rois_num, rpn_loss = self.rpn_head(body_feats, self.inputs) File "/usr/local/lib/python3.10/dist-packages/paddle/nn/layer/layers.py", line 1429, in call return self.forward(inputs, **kwargs) File "/usr/local/lib/python3.10/dist-packages/ppdet/modeling/proposal_generator/rpn_head.py", line 141, in fo rward rois, rois_num = self._gen_proposal(scores, deltas, anchors, inputs) File "/usr/local/lib/python3.10/dist-packages/ppdet/modeling/proposal_generator/rpn_head.py", line 234, in _g en_proposal bs_rois_num_collect = paddle.concat(bs_rois_num_collect) File "/usr/local/lib/python3.10/dist-packages/paddle/tensor/manipulation.py", line 1263, in concat return _C_ops.concat(input, axis) ValueError: (InvalidArgument) The axis is expected to be in range of [0, 0), but got 0

[guajioao]

@firaterdem I install paddleseg2.8 through source code instead of pip, It works well: You can try this to install paddleseg

git clone https://github.com/PaddlePaddle/PaddleSeg
cd PaddleSeg
pip install -r requirements.txt
python setup.py install

我按照您的方法重新装了paddleseg，还是报了这个错误，似乎不是metric的错误 Traceback (most recent call last): File "/home/PaddleSeg/contrib/QualityInspector/tools/det/train.py", line 197, in main() File "/home/PaddleSeg/contrib/QualityInspector/tools/det/train.py", line 193, in main run(FLAGS, cfg) File "/home/PaddleSeg/contrib/QualityInspector/tools/det/train.py", line 146, in run trainer.train(FLAGS.eval) File "/usr/local/lib/python3.10/dist-packages/ppdet/engine/trainer.py", line 539, in train outputs = model(data) File "/usr/local/lib/python3.10/dist-packages/paddle/nn/layer/layers.py", line 1429, in call return self.forward(*inputs, kwargs) File "/usr/local/lib/python3.10/dist-packages/ppdet/modeling/architectures/meta_arch.py", line 60, in forward out = self.get_loss() File "/usr/local/lib/python3.10/dist-packages/ppdet/modeling/architectures/fasterrcnn.py", line 112, in get loss rpn_loss, bbox_loss = self._forward() File "/usr/local/lib/python3.10/dist-packages/ppdet/modeling/architectures/faster_rcnn.py", line 80, in _forw ard rois, rois_num, rpn_loss = self.rpn_head(body_feats, self.inputs) File "/usr/local/lib/python3.10/dist-packages/paddle/nn/layer/layers.py", line 1429, in call* return self.forward(inputs, **kwargs) File "/usr/local/lib/python3.10/dist-packages/ppdet/modeling/proposal_generator/rpn_head.py", line 141, in fo rward rois, rois_num = self._gen_proposal(scores, deltas, anchors, inputs) File "/usr/local/lib/python3.10/dist-packages/ppdet/modeling/proposal_generator/rpn_head.py", line 234, in _g en_proposal bs_rois_num_collect = paddle.concat(bs_rois_num_collect) File "/usr/local/lib/python3.10/dist-packages/paddle/tensor/manipulation.py", line 1263, in concat return _C_ops.concat(input, axis) ValueError: (InvalidArgument) The axis is expected to be in range of [0, 0), but got 0

我调试了一下看到在metric.py第53-55行的paddle.sum(*)计算结果shape=[]，在for循环结束后进行concat就会报错。我将sum的结果用升了一维，之后就没再报错了。如果你也是这种情况可以试试 pred_area.append(paddle.sum(paddle.cast(pred_i, "int64")).unsqueeze(0))