ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.

[kyleaitgroup]

Describe the bug A clear and concise description of what the bug is.

Environment Environment you use when bug appears:

1. Python version: 3.8
2. PyTorch Version: 2.0.0+cu118
3. MMCV Version: 2.0.1
4. EdgeLab Version
5. Code you run: python tools/train.py configs/fomo/fomo_person.py --cfg-options \work_dir=work_dirs/fomo_300 num_classes=1 epochs=300 height=416 width=416 data_root=datasets/coco_person/
6. The detailed error

Traceback (most recent call last): File "tools/train.py", line 226, in main() File "tools/train.py", line 221, in main runner.train() File "/home/ubuntu/.local/lib/python3.8/site-packages/mmengine/runner/runner.py", line 1777, in train model = self.train_loop.run() # type: ignore File "/home/ubuntu/.local/lib/python3.8/site-packages/mmengine/runner/loops.py", line 96, in run self.run_epoch() File "/home/ubuntu/.local/lib/python3.8/site-packages/mmengine/runner/loops.py", line 112, in run_epoch self.run_iter(idx, data_batch) File "/home/ubuntu/.local/lib/python3.8/site-packages/mmengine/runner/loops.py", line 128, in run_iter outputs = self.runner.model.train_step( File "/home/ubuntu/Documents/ModelAssistant/sscma/models/detectors/fomo.py", line 99, in train_step losses = self._run_forward(data, mode='loss') # type: ignore File "/home/ubuntu/.local/lib/python3.8/site-packages/mmengine/model/base_model/base_model.py", line 361, in _run_forward results = self(data, mode=mode) File "/home/ubuntu/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, *kwargs) File "/home/ubuntu/Documents/ModelAssistant/sscma/models/detectors/fomo.py", line 70, in forward return self.loss(inputs, data_samples) File "/home/ubuntu/.local/lib/python3.8/site-packages/mmdet/models/detectors/single_stage.py", line 78, in loss losses = self.bbox_head.loss(x, batch_data_samples) File "/home/ubuntu/Documents/ModelAssistant/sscma/models/heads/fomo_head.py", line 115, in loss loss = self.loss_by_feat(pred, batch_gt_instances, batch_img_metas, batch_gt_instances_ignore) File "/home/ubuntu/Documents/ModelAssistant/sscma/models/heads/fomo_head.py", line 150, in loss_by_feat loss, cls_loss, bg_loss, P, R, F1 = multi_apply(self.lossFunction, preds, target) File "/home/ubuntu/.local/lib/python3.8/site-packages/mmdet/models/utils/misc.py", line 219, in multi_apply return tuple(map(list, zip(map_results))) File "/home/ubuntu/Documents/ModelAssistant/sscma/models/heads/fomo_head.py", line 188, in lossFunction P, R, F1 = self.get_pricsion_recall_f1(preds, data) File "/home/ubuntu/Documents/ModelAssistant/sscma/models/heads/fomo_head.py", line 229, in get_pricsion_recall_f1 site = np.sum([ti, po], axis=0) File "<__array_function__ internals>", line 200, in sum File "/home/ubuntu/anaconda3/envs/model2/lib/python3.8/site-packages/numpy/core/fromnumeric.py", line 2324, in sum return _wrapreduction(a, np.add, 'sum', axis, dtype, out, keepdims=keepdims, File "/home/ubuntu/anaconda3/envs/model2/lib/python3.8/site-packages/numpy/core/fromnumeric.py", line 86, in _wrapreduction return ufunc.reduce(obj, axis, dtype, out, passkwargs) ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.

Additional context I'm using latest version of ModelAssistant .

after I change site = np.sum([ti, po], axis=0)in ModelAssistant/sscma/models/heads/fomo_head.pywith site = np.concatenate([ti, po], axis=0) it stat training,but I get another error when training the model

Traceback (most recent call last): File "tools/train.py", line 226, in main() File "tools/train.py", line 221, in main runner.train() File "/home/ubuntu/.local/lib/python3.8/site-packages/mmengine/runner/runner.py", line 1777, in train model = self.train_loop.run() # type: ignore File "/home/ubuntu/.local/lib/python3.8/site-packages/mmengine/runner/loops.py", line 102, in run self.runner.val_loop.run() File "/home/ubuntu/.local/lib/python3.8/site-packages/mmengine/runner/loops.py", line 371, in run self.run_iter(idx, data_batch) File "/home/ubuntu/.local/lib/python3.8/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(*args, kwargs) File "/home/ubuntu/.local/lib/python3.8/site-packages/mmengine/runner/loops.py", line 392, in run_iter self.evaluator.process(data_samples=outputs, data_batch=data_batch) File "/home/ubuntu/.local/lib/python3.8/site-packages/mmengine/evaluator/evaluator.py", line 60, in process metric.process(data_batch, _data_samples) File "/home/ubuntu/Documents/ModelAssistant/sscma/evaluation/fomo_metric.py", line 74, in process tp, fp, fn = multi_apply(self.compute_ftp, preds, target) File "/home/ubuntu/.local/lib/python3.8/site-packages/mmdet/models/utils/misc.py", line 219, in multi_apply return tuple(map(list, zip(map_results))) File "/home/ubuntu/Documents/ModelAssistant/sscma/evaluation/fomo_metric.py", line 48, in compute_ftp confusion = confusion_matrix( File "/home/ubuntu/.local/lib/python3.8/site-packages/sklearn/utils/_param_validation.py", line 214, in wrapper return func(args, kwargs) File "/home/ubuntu/.local/lib/python3.8/site-packages/sklearn/metrics/_classification.py", line 326, in confusion_matrix y_type, y_true, y_pred = _check_targets(y_true, y_pred) File "/home/ubuntu/.local/lib/python3.8/site-packages/sklearn/metrics/_classification.py", line 84, in _check_targets check_consistent_length(y_true, y_pred) File "/home/ubuntu/.local/lib/python3.8/site-packages/sklearn/utils/validation.py", line 407, in check_consistent_length raise ValueError( ValueError: Found input variables with inconsistent numbers of samples: [144, 24]

Additional context if i set tp = fp = fn = 0 ,it works. but when I convert the model to binary and deploy it on the ESP32-S3 Eye, I encounter the following errors:

When I convert the model to binary and deploy it on the ESP32-S3 Eye, I encounter the following errors:

Didn't find op for builtin opcode 'TRANSPOSE' Failed to get registration from op code TRANSPOSE AllocateTensors() failed

and

W (4927) cam_hal: Failed to get the frame on time!

ERROR A stack overflow in task app_camera has been detected.

Backtrace: 0x40375b5e:0x3fca8680 0x4037cc1d:0x3fca86a0 0x4037f64a:0x3fca86c0 0x4037e2bf:0x3fca8740 0x4037f758:0x3fca8760 0x4037f74e:0xa5a5a5a5 |<-CORRUPTED 0x40375b5e: panic_abort at /home/ubuntu/esp/v5.1.2/esp-idf/components/esp_system/panic.c:452 0x4037cc1d: esp_system_abort at /home/ubuntu/esp/v5.1.2/esp-idf/components/esp_system/port/esp_system_chip.c:84 0x4037f64a: vApplicationStackOverflowHook at /home/ubuntu/esp/v5.1.2/esp-idf/components/freertos/FreeRTOS-Kernel/portable/xtensa/port.c:581 0x4037e2bf: vTaskSwitchContext at /home/ubuntu/esp/v5.1.2/esp-idf/components/freertos/FreeRTOS-Kernel/tasks.c:3729 0x4037f758: _frxt_dispatch at /home/ubuntu/esp/v5.1.2/esp-idf/components/freertos/FreeRTOS-Kernel/portable/xtensa/portasm.S:450 0x4037f74e: _frxt_int_exit at /home/ubuntu/esp/v5.1.2/esp-idf/components/freertos/FreeRTOS-Kernel/portable/xtensa/portasm.S:245

[LynnL4]

Hi, Which deployment code are you using, it looks like the registration of the TRANSPOSE operator is missing. https://github.com/Seeed-Studio/SSCMA-Micro/blob/dd57e1cdd0317b7ae97838ae53fbd18f92acb020/porting/espressif/el_config_porting.h#L69 also thanks for trying, we will fix the training issue you mentioned as soon as possible, thanks!

[kyleaitgroup]

Hi, Thanks for the quick response!

I'm currently using https://github.com/Seeed-Studio/sscma-example-esp32

I've checked that the line#define CONFIG_EL_TFLITE_OP_TRANSPOSE does exist in the code, but the error persists.

[LynnL4]

LynnL4 commented 21 minutes ago Do you use this example? I will check it https://github.com/Seeed-Studio/sscma-example-esp32/blob/main/examples/fomo_detection_demo/main/app_main.cpp

[kyleaitgroup]

Yes ,but I didn't notice that I'm not using the latest version of sscma-example-esp32. I'll try it first .

[LynnL4]

Hi, If you're using an older version of the repository, perhaps the problem is here https://github.com/Seeed-Studio/sscma-example-esp32/blob/d926450d530216340c46a0daf0f9435294abce27/components/modules/algorithm/algo_fomo.cpp#L223

[kyleaitgroup]

Thanks. I've tested the latest version, but unfortunately, I encountered a new error.

Failed to resize buffer. Requested: 2092448, available 1044760, missing: 1047688 [ASSERT] Failed assertion 'interpreter != nullptr' E (5922) task_wdt: Task watchdog got triggered. The following tasks/users did not reset the watchdog in time: E (5922) task_wdt: - IDLE (CPU 0) E (5922) task_wdt: Tasks currently running: E (5922) task_wdt: CPU 0: main E (5922) task_wdt: CPU 1: IDLE E (5922) task_wdt: Print CPU 0 (current core) backtrace

Could it be caused by setting the epoch too large? I set the epoch to 300.

[LynnL4]

I see that your configuration is height=416 width=416, this will cause the model to require a lot of running memory, suggest changing it to 192x192

[MILK-BIOS]

Hi @kyleaitgroup, the problem in training is caused by different types of ti and po. We fixed it in the latest version. Thank you!