Waymo open dataset docker image pip install issue

[mgmike]

When building a docker image from the Dockerfile, I am getting the following error: Could not find a version that satisfies the requirement tensorflow==2.5.0 (from versions: 0.12.1, 1.0.0, 1.0.1, 1.1.0, 1.2.0, 1.2.1, 1.3.0, 1.4.0, 1.4.1, 1.5.0, 1.5.1, 1.6.0, 1.7.0, 1.7.1, 1.8.0, 1.9.0, 1.10.0, 1.10.1, 1.11.0, 1.12.0, 1.12.2, 1.12.3, 1.13.1, 1.13.2, 1.14.0) No matching distribution found for tensorflow==2.5.0

The only thing changed from the original Dockerfile is changing the Ubuntu version from 20.04 to 18.04

I fixed this by adding RUN python3 -m pip install --upgrade pip

to the Dockerfile before the install tensorflow line, but then I get this error: ERROR: Could not find a version that satisfies the requirement waymo-open-dataset-tf-2-5-0==1.4.1 (from versions: none) ERROR: No matching distribution found for waymo-open-dataset-tf-2-5-0==1.4.1

I dont know what to do because the latest version is 1.4.1 according to pypi which I also have locally.

I am running on Ubuntu 18.04.

[mgmike]

I did a version check on python and found out python is Python 3.6.9. This is the problem, python is not updating to 3.8.

[mgmike]

Fixed by adding

RUN update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.6 1 RUN update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.8 2 RUN python3 -m pip install --upgrade pip

before the tensorflow install command, but another issue popped up.

After RUN python3 -m pip install git+https://github.com/philferriere/cocoapi.git#subdirectory=PythonAPI, I get the error:

Collecting git+https://github.com/philferriere/cocoapi.git#subdirectory=PythonAPI Cloning https://github.com/philferriere/cocoapi.git to /tmp/pip-req-build-k3pmfnhn Running command git clone --filter=blob:none -q https://github.com/philferriere/cocoapi.git /tmp/pip-req-build-k3pmfnhn Resolved https://github.com/philferriere/cocoapi.git to commit 2929bd2ef6b451054755dfd7ceb09278f935f7ad Preparing metadata (setup.py): started Preparing metadata (setup.py): finished with status 'done' Building wheels for collected packages: pycocotools Building wheel for pycocotools (setup.py): started Building wheel for pycocotools (setup.py): finished with status 'error' ERROR: Command errored out with exit status 1: command: /usr/bin/python3 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-req-build-k3pmfnhn/PythonAPI/setup.py'"'"'; file='"'"'/tmp/pip-req-build-k3pmfnhn/PythonAPI/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(file) if os.path.exists(file) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-pua4jq_n cwd: /tmp/pip-req-build-k3pmfnhn/PythonAPI Complete output (65 lines): running bdist_wheel running build running build_py creating build creating build/lib.linux-x86_64-3.8 creating build/lib.linux-x86_64-3.8/pycocotools copying pycocotools/coco.py -> build/lib.linux-x86_64-3.8/pycocotools copying pycocotools/init.py -> build/lib.linux-x86_64-3.8/pycocotools copying pycocotools/mask.py -> build/lib.linux-x86_64-3.8/pycocotools copying pycocotools/cocoeval.py -> build/lib.linux-x86_64-3.8/pycocotools running build_ext creating build/common creating build/temp.linux-x86_64-3.8 creating build/temp.linux-x86_64-3.8/pycocotools x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -I/usr/local/lib/python3.8/dist-packages/numpy/core/include -I../common -I/usr/include/python3.8 -c ../common/maskApi.c -o build/temp.linux-x86_64-3.8/../common/maskApi.o ../common/maskApi.c: In function ‘rleDecode’: ../common/maskApi.c:46:7: warning: this ‘for’ clause does not guard... [-Wmisleading-indentation] for( k=0; k<R[i].cnts[j]; k++ ) (M++)=v; v=!v; }} ^~~ ../common/maskApi.c:46:49: note: ...this statement, but the latter is misleadingly indented as if it were guarded by the ‘for’ for( k=0; k<R[i].cnts[j]; k++ ) (M++)=v; v=!v; }} ^ ../common/maskApi.c: In function ‘rleFrPoly’: ../common/maskApi.c:166:3: warning: this ‘for’ clause does not guard... [-Wmisleading-indentation] for(j=0; j<k; j++) x[j]=(int)(scalexy[j2+0]+.5); x[k]=x[0]; ^~~ ../common/maskApi.c:166:54: note: ...this statement, but the latter is misleadingly indented as if it were guarded by the ‘for’ for(j=0; j<k; j++) x[j]=(int)(scalexy[j2+0]+.5); x[k]=x[0]; ^ ../common/maskApi.c:167:3: warning: this ‘for’ clause does not guard... [-Wmisleading-indentation] for(j=0; j<k; j++) y[j]=(int)(scalexy[j2+1]+.5); y[k]=y[0]; ^~~ ../common/maskApi.c:167:54: note: ...this statement, but the latter is misleadingly indented as if it were guarded by the ‘for’ for(j=0; j<k; j++) y[j]=(int)(scalexy[j2+1]+.5); y[k]=y[0]; ^ ../common/maskApi.c: In function ‘rleToString’: ../common/maskApi.c:212:7: warning: this ‘if’ clause does not guard... [-Wmisleading-indentation] if(more) c |= 0x20; c+=48; s[p++]=c; ^~ ../common/maskApi.c:212:27: note: ...this statement, but the latter is misleadingly indented as if it were guarded by the ‘if’ if(more) c |= 0x20; c+=48; s[p++]=c; ^ ../common/maskApi.c: In function ‘rleFrString’: ../common/maskApi.c:220:3: warning: this ‘while’ clause does not guard... [-Wmisleading-indentation] while( s[m] ) m++; cnts=malloc(sizeof(uint)m); m=0; ^~~~~ ../common/maskApi.c:220:22: note: ...this statement, but the latter is misleadingly indented as if it were guarded by the ‘while’ while( s[m] ) m++; cnts=malloc(sizeof(uint)m); m=0; ^~~~ ../common/maskApi.c:228:5: warning: this ‘if’ clause does not guard... [-Wmisleading-indentation] if(m>2) x+=(long) cnts[m-2]; cnts[m++]=(uint) x; ^~ ../common/maskApi.c:228:34: note: ...this statement, but the latter is misleadingly indented as if it were guarded by the ‘if’ if(m>2) x+=(long) cnts[m-2]; cnts[m++]=(uint) x; ^~~~ ../common/maskApi.c: In function ‘rleToBbox’: ../common/maskApi.c:141:31: warning: ‘xp’ may be used uninitialized in this function [-Wmaybe-uninitialized] if(j%2==0) xp=x; else if(xp<x) { ys=0; ye=h-1; } ^ x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -I/usr/local/lib/python3.8/dist-packages/numpy/core/include -I../common -I/usr/include/python3.8 -c pycocotools/_mask.c -o build/temp.linux-x86_64-3.8/pycocotools/_mask.o pycocotools/_mask.c:27:10: fatal error: Python.h: No such file or directory

include "Python.h"

^~~~~~ compilation terminated. error: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1

ERROR: Failed building wheel for pycocotools Running setup.py clean for pycocotools ERROR: Command errored out with exit status 1: command: /usr/bin/python3 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-req-build-k3pmfnhn/PythonAPI/setup.py'"'"'; file='"'"'/tmp/pip-req-build-k3pmfnhn/PythonAPI/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(file) if os.path.exists(file) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' clean --all cwd: /tmp/pip-req-build-k3pmfnhn Complete output (11 lines): Traceback (most recent call last): File "", line 1, in File "/tmp/pip-req-build-k3pmfnhn/PythonAPI/setup.py", line 25, in cythonize(ext_modules) File "/usr/local/lib/python3.8/dist-packages/Cython/Build/Dependencies.py", line 965, in cythonize module_list, module_metadata = create_extension_list( File "/usr/local/lib/python3.8/dist-packages/Cython/Build/Dependencies.py", line 815, in create_extension_list for file in nonempty(sorted(extended_iglob(filepattern)), "'%s' doesn't match any files" % filepattern): File "/usr/local/lib/python3.8/dist-packages/Cython/Build/Dependencies.py", line 114, in nonempty raise ValueError(error_msg) ValueError: 'pycocotools/_mask.pyx' doesn't match any files

ERROR: Failed cleaning build dir for pycocotools Failed to build pycocotools Installing collected packages: pycocotools Running setup.py install for pycocotools: started Running setup.py install for pycocotools: finished with status 'error' ERROR: Command errored out with exit status 1: command: /usr/bin/python3 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-req-build-k3pmfnhn/PythonAPI/setup.py'"'"'; file='"'"'/tmp/pip-req-build-k3pmfnhn/PythonAPI/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(file) if os.path.exists(file) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record /tmp/pip-record-hg6lrs1l/install-record.txt --single-version-externally-managed --compile --install-headers /usr/local/include/python3.8/pycocotools cwd: /tmp/pip-req-build-k3pmfnhn/PythonAPI Complete output (57 lines): running install /usr/local/lib/python3.8/dist-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools. warnings.warn( running build running build_py running build_ext x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -I/usr/local/lib/python3.8/dist-packages/numpy/core/include -I../common -I/usr/include/python3.8 -c ../common/maskApi.c -o build/temp.linux-x86_64-3.8/../common/maskApi.o ../common/maskApi.c: In function ‘rleDecode’: ../common/maskApi.c:46:7: warning: this ‘for’ clause does not guard... [-Wmisleading-indentation] for( k=0; k<R[i].cnts[j]; k++ ) (M++)=v; v=!v; }} ^~~ ../common/maskApi.c:46:49: note: ...this statement, but the latter is misleadingly indented as if it were guarded by the ‘for’ for( k=0; k<R[i].cnts[j]; k++ ) (M++)=v; v=!v; }} ^ ../common/maskApi.c: In function ‘rleFrPoly’: ../common/maskApi.c:166:3: warning: this ‘for’ clause does not guard... [-Wmisleading-indentation] for(j=0; j<k; j++) x[j]=(int)(scalexy[j2+0]+.5); x[k]=x[0]; ^~~ ../common/maskApi.c:166:54: note: ...this statement, but the latter is misleadingly indented as if it were guarded by the ‘for’ for(j=0; j<k; j++) x[j]=(int)(scalexy[j2+0]+.5); x[k]=x[0]; ^ ../common/maskApi.c:167:3: warning: this ‘for’ clause does not guard... [-Wmisleading-indentation] for(j=0; j<k; j++) y[j]=(int)(scalexy[j2+1]+.5); y[k]=y[0]; ^~~ ../common/maskApi.c:167:54: note: ...this statement, but the latter is misleadingly indented as if it were guarded by the ‘for’ for(j=0; j<k; j++) y[j]=(int)(scalexy[j2+1]+.5); y[k]=y[0]; ^ ../common/maskApi.c: In function ‘rleToString’: ../common/maskApi.c:212:7: warning: this ‘if’ clause does not guard... [-Wmisleading-indentation] if(more) c |= 0x20; c+=48; s[p++]=c; ^~ ../common/maskApi.c:212:27: note: ...this statement, but the latter is misleadingly indented as if it were guarded by the ‘if’ if(more) c |= 0x20; c+=48; s[p++]=c; ^ ../common/maskApi.c: In function ‘rleFrString’: ../common/maskApi.c:220:3: warning: this ‘while’ clause does not guard... [-Wmisleading-indentation] while( s[m] ) m++; cnts=malloc(sizeof(uint)m); m=0; ^~~~~ ../common/maskApi.c:220:22: note: ...this statement, but the latter is misleadingly indented as if it were guarded by the ‘while’ while( s[m] ) m++; cnts=malloc(sizeof(uint)m); m=0; ^~~~ ../common/maskApi.c:228:5: warning: this ‘if’ clause does not guard... [-Wmisleading-indentation] if(m>2) x+=(long) cnts[m-2]; cnts[m++]=(uint) x; ^~ ../common/maskApi.c:228:34: note: ...this statement, but the latter is misleadingly indented as if it were guarded by the ‘if’ if(m>2) x+=(long) cnts[m-2]; cnts[m++]=(uint) x; ^~~~ ../common/maskApi.c: In function ‘rleToBbox’: ../common/maskApi.c:141:31: warning: ‘xp’ may be used uninitialized in this function [-Wmaybe-uninitialized] if(j%2==0) xp=x; else if(xp<x) { ys=0; ye=h-1; } ^ x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -I/usr/local/lib/python3.8/dist-packages/numpy/core/include -I../common -I/usr/include/python3.8 -c pycocotools/_mask.c -o build/temp.linux-x86_64-3.8/pycocotools/_mask.o pycocotools/_mask.c:27:10: fatal error: Python.h: No such file or directory

include "Python.h"

^~~~~~ compilation terminated. error: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1

ERROR: Command errored out with exit status 1: /usr/bin/python3 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-req-build-k3pmfnhn/PythonAPI/setup.py'"'"'; file='"'"'/tmp/pip-req-build-k3pmfnhn/PythonAPI/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(file) if os.path.exists(file) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record /tmp/pip-record-hg6lrs1l/install-record.txt --single-version-externally-managed --compile --install-headers /usr/local/include/python3.8/pycocotools Check the logs for full command output. The command '/bin/bash -c python3 -m pip install git+https://github.com/philferriere/cocoapi.git#subdirectory=PythonAPI' returned a non-zero code: 1