errors when converting det_2d.json to coco format.

[JACK-Chen-2019]

when runnning # SHIFT discrete (images, detection-like) SET_NAME=val

SHIFT discrete (images, detection-like)

python -m scalabel.label.to_coco -m box_track -i discrete/images/$SET_NAME/front/det_2d.json -o discrete/images/$SET_NAME/front/det_2d_cocoformat.json

SHIFT continuous (videos, tracking-like)

python -m scalabel.label.to_coco -m box_track -i continuous/videos/1x/$SET_NAME/front/det_2d.json -o continuous/videos/1x/$SET_NAME/front/det_2d_cocoformat.json

bash transfer.sh /home/liangzc/miniconda3/envs/TTADET/lib/python3.9/runpy.py:127: RuntimeWarning: 'scalabel.label.to_coco' found in sys.modules after import of package 'scalabel.label', but prior to execution of 'scalabel.label.to_coco'; this may result in unpredictable behaviour warn(RuntimeWarning(msg)) [2023-07-14 19:56:52,323 to_coco.py:516 run] Loading Scalabel jsons... Traceback (most recent call last): File "/home/liangzc/miniconda3/envs/TTADET/lib/python3.9/runpy.py", line 197, in _run_module_as_main return _run_code(code, main_globals, None, File "/home/liangzc/miniconda3/envs/TTADET/lib/python3.9/runpy.py", line 87, in _run_code exec(code, run_globals) File "/home/liangzc/miniconda3/envs/TTADET/lib/python3.9/site-packages/scalabel/label/to_coco.py", line 551, in run(parse_arguments()) File "/home/liangzc/miniconda3/envs/TTADET/lib/python3.9/site-packages/scalabel/label/to_coco.py", line 517, in run dataset = load(args.input, args.nproc) File "/home/liangzc/miniconda3/envs/TTADET/lib/python3.9/site-packages/scalabel/label/io.py", line 115, in load config = Config(**cfg) File "/home/liangzc/miniconda3/envs/TTADET/lib/python3.9/site-packages/pydantic/main.py", line 150, in init pydantic_self.pydantic_validator.validate_python(data, self_instance=__pydantic_self__) pydantic_core._pydantic_core.ValidationError: 14 validation errors for Config attributes Field required [type=missing, input_value={'imageSize': {'width': 1...}, {'name': 'bicycle'}]}, input_type=dict] For further information visit https://errors.pydantic.dev/2.1.2/v/missing categories.0.subcategories Field required [type=missing, input_value={'name': 'pedestrian'}, input_type=dict] For further information visit https://errors.pydantic.dev/2.1.2/v/missing categories.0.color Field required [type=missing, input_value={'name': 'pedestrian'}, input_type=dict] For further information visit https://errors.pydantic.dev/2.1.2/v/missing categories.1.subcategories Field required [type=missing, input_value={'name': 'car'}, input_type=dict] For further information visit https://errors.pydantic.dev/2.1.2/v/missing categories.1.color Field required [type=missing, input_value={'name': 'car'}, input_type=dict] For further information visit https://errors.pydantic.dev/2.1.2/v/missing categories.2.subcategories Field required [type=missing, input_value={'name': 'truck'}, input_type=dict] For further information visit https://errors.pydantic.dev/2.1.2/v/missing categories.2.color Field required [type=missing, input_value={'name': 'truck'}, input_type=dict] For further information visit https://errors.pydantic.dev/2.1.2/v/missing categories.3.subcategories Field required [type=missing, input_value={'name': 'bus'}, input_type=dict] For further information visit https://errors.pydantic.dev/2.1.2/v/missing categories.3.color Field required [type=missing, input_value={'name': 'bus'}, input_type=dict] For further information visit https://errors.pydantic.dev/2.1.2/v/missing categories.4.subcategories Field required [type=missing, input_value={'name': 'motorcycle'}, input_type=dict] For further information visit https://errors.pydantic.dev/2.1.2/v/missing categories.4.color Field required [type=missing, input_value={'name': 'motorcycle'}, input_type=dict] For further information visit https://errors.pydantic.dev/2.1.2/v/missing categories.5.subcategories Field required [type=missing, input_value={'name': 'bicycle'}, input_type=dict] For further information visit https://errors.pydantic.dev/2.1.2/v/missing categories.5.color Field required [type=missing, input_value={'name': 'bicycle'}, input_type=dict] For further information visit https://errors.pydantic.dev/2.1.2/v/missing poseSigmas Field required [type=missing, input_value={'imageSize': {'width': 1...}, {'name': 'bicycle'}]}, input_type=dict] For further information visit https://errors.pydantic.dev/2.1.2/v/missing

[mattiasegu]

Hi @JACK-Chen-2019, this problem seems related to the new scalabel version.

Can you try re-installing scalabel with pip install --upgrade git+https://github.com/scalabel/scalabel.git@v0.3.0 or directly re-installing the environment from scratch with that scalabel version?

Please let me know if it works

[JACK-Chen-2019]

when i run sudo docker build -t shift_dataset_decompress .

Sending build context to Docker daemon 23.02MB Step 1/13 : FROM python:3.8 ---> db2bf59cb82f Step 2/13 : WORKDIR /usr/src/app ---> Using cache ---> 0ea061a9be0d Step 3/13 : RUN apt -y update ---> Using cache ---> 9988a5e33986 Step 4/13 : RUN apt -y install software-properties-common dirmngr apt-transport-https lsb-release ca-certificates ---> Using cache ---> 2d57763bd12b Step 5/13 : RUN apt -y install python3-h5py pkg-config libhdf5-dev ---> Using cache ---> deb7ac1f4dd7 Step 6/13 : RUN apt -y install ffmpeg=7:4.3.4-0+deb11u1 ---> Running in 48cbedce9a3f

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

Reading package lists... Building dependency tree... Reading state information... Package ffmpeg is not available, but is referred to by another package. This may mean that the package is missing, has been obsoleted, or is only available from another source

E: Version '7:4.3.4-0+deb11u1' for 'ffmpeg' was not found The command '/bin/sh -c apt -y install ffmpeg=7:4.3.4-0+deb11u1' returned a non-zero code: 100

[birdomi]

I had a same problem and https://github.com/scalabel/scalabel/issues/515 works for me.

Also, for your docker issue, I delete the line "RUN apt -y install ffmpeg=7:4.3.4-0+deb11u1" in Dockerfile in shite-dev folder. I thinks it works for me, because I already installed ffmpeg.

Hope it works.

[JACK-Chen-2019]

Thank you very much. pip install -U pydantic==1.9.1. It works!!!

[mattiasegu]

Kudos to @birdomi for solving this issue :) thanks a lot, I updated the requirements of our repo to downgrade pydantic to the previous version