consider not using pycocotools since it doesn't work on windows

woolfel commented 6 years ago

Please go to Stack Overflow for help and support:

http://stackoverflow.com/questions/tagged/tensorflow

Also, please understand that many of the models included in this repository are experimental and research-style code. If you open a GitHub issue, here is our policy:

It must be a bug, a feature request, or a significant problem with documentation (for small docs fixes please send a PR instead).
The form below must be filled out.

Here's why we have that policy: TensorFlow developers respond to issues. We want to focus on work that benefits the whole community, e.g., fixing bugs and adding features. Support only helps individuals. GitHub also notifies thousands of people when issues are filed. We want them to see you communicating an interesting problem, rather than being redirected to Stack Overflow.

System information

What is the top-level directory of the model you are using:
Have I written custom code (as opposed to using a stock example script provided in TensorFlow):
OS Platform and Distribution (e.g., Linux Ubuntu 16.04):
TensorFlow installed from (source or binary):
TensorFlow version (use command below):
Bazel version (if compiling from source):
CUDA/cuDNN version:
GPU model and memory:
Exact command to reproduce:

You can collect some of this information using our environment capture script:

https://github.com/tensorflow/tensorflow/tree/master/tools/tf_env_collect.sh

_== cat /etc/issue =============================================== MINGW64_NT-10.0 Beast 2.6.0(0.304/5/3) 2016-09-09 09:46 x86_64 Msys

== are we in docker ============================================= No

== compiler ===================================================== bash: c++: command not found

== uname -a ===================================================== MINGW64_NT-10.0 Beast 2.6.0(0.304/5/3) 2016-09-09 09:46 x86_64 Msys

== check pips =================================================== numpy 1.14.2
protobuf 3.6.1
tensorflow 1.7.0
tensorflow-gpu 1.10.0
tensorflow-hub 0.1.1

== check for virtualenv ========================================= False

== tensorflow import ============================================ tf.VERSION = 1.10.0 tf.GIT_VERSION = b'v1.10.0-rc1-19-g656e7a2b34' tf.COMPILERVERSION = b'v1.10.0-rc1-19-g656e7a2b34' Sanity check: array([1])

You can obtain the TensorFlow version with

python -c "import tensorflow as tf; print(tf.GIT_VERSION, tf.VERSION)"

Describe the problem

the current model_main.py script for object detection uses pycoco and pycocotools. There is a known bug in pycocotools and the maintainers of that project have no plans to support windows. This means anyone using windows has to use the legacy.trainer.py instead. On MacOS model_main.py works fine. Until pycocotools maintainers decide to support windows, I suggest not using pycocotools or at least let users know it only works on Mac and Linux.

Source code / logs

The exact issue with pycocotools can be found at this link. https://github.com/cocodataset/cocoapi/issues/169 The error when you try to run pip install pycocotools on windows.

Running setup.py clean for pycocotools Failed to build pycocotools Installing collected packages: pycocotools Running setup.py install for pycocotools ... error Complete output from command d:\python\python36\python.exe -u -c "import setuptools, tokenize;file='C:\Users\P ETERL~1\AppData\Local\Temp\pip-install-qwoxf3_t\pycocotools\setup.py';f=getattr(tokenize, 'open', open)(file); code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record C:\Users\PETERL~1\ AppData\Local\Temp\pip-record-nkxviuzj\install-record.txt --single-version-externally-managed --compile: running install running build running build_py creating build creating build\lib.win-amd64-3.6 creating build\lib.win-amd64-3.6\pycocotools copying pycocotools\coco.py -> build\lib.win-amd64-3.6\pycocotools copying pycocotools\cocoeval.py -> build\lib.win-amd64-3.6\pycocotools copying pycocotools\mask.py -> build\lib.win-amd64-3.6\pycocotools copying pycocotools__init__.py -> build\lib.win-amd64-3.6\pycocotools running build_ext building 'pycocotools._mask' extension creating build\temp.win-amd64-3.6 creating build\temp.win-amd64-3.6\Release creating build\temp.win-amd64-3.6\Release\pycocotools creating build\temp.win-amd64-3.6\Release\common C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\bin\HostX86\x64\cl.exe /c /n ologo /Ox /W3 /GL /DNDEBUG /MD -Id:\python\python36\lib\site-packages\numpy\core\include -Icommon -Id:\python\python36\i nclude -Id:\python\python36\include "-IC:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14 .26428\ATLMFC\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\includ e" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.6.1\include\um" "-IC:\Program Files (x86)\Windows Kits\10\include\1 0.0.17134.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17134.0\shared" "-IC:\Program Files (x86)\Windo ws Kits\10\include\10.0.17134.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17134.0\winrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.17134.0\cppwinrt" /Tcpycocotools/_mask.c /Fobuild\temp.win-amd64-3.6\Release\py cocotools/_mask.obj -Wno-cpp -Wno-unused-function -std=c99 cl : Command line error D8021 : invalid numeric argument '/Wno-cpp' error: command 'C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\bin \HostX86\x64\cl.exe' failed with exit status 2

----------------------------------------

baluyotraf commented 6 years ago

As a work around can try using this https://github.com/philferriere/cocoapi

Li-ReDBox commented 6 years ago

I would think it does not work well on MacOS neither. See this issue

woolfel commented 6 years ago

I did test model_main.py MacOS Mojave 10.14 and it worked fine with pycoco and pycocotools. I've only had issues on windows. I haven't tried the workaround cocoapi on windows.

Li-ReDBox commented 6 years ago

@woolfel I had the problem when I was using gcloud suggested by an article, which needs to copy a repackaged pycocotools:

gcloud ml-engine jobs submit training `whoami`_object_detection_`date +%s` \
--job-dir=gs://${YOUR_GCS_BUCKET}/train \
--packages dist/object_detection-0.1.tar.gz,slim/dist/slim-0.1.tar.gz,/tmp/pycocotools/pycocotools-2.0.tar.gz \
--module-name object_detection.model_tpu_main \
--runtime-version 1.8 \
--scale-tier BASIC_TPU \
--region us-central1 \
-- \
--model_dir=gs://${YOUR_GCS_BUCKET}/train \
--tpu_zone us-central1 \
--pipeline_config_path=gs://${YOUR_GCS_BUCKET}/data/pipeline.config

On mac, repackaging pycocotools using sed does not work. The detail can be seen in the issue

zbiswas3 commented 5 years ago

From conda command prompt do this two commands. Just copy and paste and enter to run.. first: conda install git then: pip3 install "git+https://github.com/philferriere/cocoapi.git#egg=pycocotools&subdirectory=PythonAPI"

tensorflowbutler commented 4 years ago

Hi There, We are checking to see if you still need help on this, as this seems to be considerably old issue. Please update this issue with the latest information, code snippet to reproduce your issue and error you are seeing. If we don't hear from you in the next 7 days, this issue will be closed automatically. If you don't need help on this issue any more, please consider closing this.

anoldmaninthesea commented 3 years ago

I have the same problem...

Building wheels for collected packages: pycocotools Building wheel for pycocotools (setup.py) ... error ERROR: Command errored out with exit status 1: command: 'C:\Users\iavta\anaconda3\envs\PythonCPU\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\Users\iavta\AppData\Local\Temp\pip-install-23wmyn5i\pycocotools_2add5938f11b43b19dbf841cc3df73f8\setup.py'"'"'; file='"'"'C:\Users\iavta\AppData\Local\Temp\pip-install-23wmyn5i\pycocotools_2add5938f11b43b19dbf841cc3df73f8\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' bdist_wheel -d 'C:\Users\iavta\AppData\Local\Temp\pip-wheel-11vbpz4j' cwd: C:\Users\iavta\AppData\Local\Temp\pip-install-23wmyn5i\pycocotools_2add5938f11b43b19dbf841cc3df73f8\ Complete output (16 lines): running bdist_wheel running build running build_py creating build creating build\lib.win-amd64-3.8 creating build\lib.win-amd64-3.8\pycocotools copying pycocotools\coco.py -> build\lib.win-amd64-3.8\pycocotools running build_ext cythoning pycocotools/_mask.pyx to pycocotools_mask.c C:\Users\iavta\anaconda3\envs\PythonCPU\lib\site-packages\Cython\Compiler\Main.py:369: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: C:\Users\iavta\AppData\Local\Temp\pip-install-23wmyn5i\pycocotools_2add5938f11b43b19dbf841cc3df73f8\pycocotools_mask.pyx tree = Parsing.p_module(s, pxd, full_module_name) building 'pycocotools._mask' extension error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/

ERROR: Failed building wheel for pycocotools Running setup.py clean for pycocotools Failed to build pycocotools

I've even installed the visual-cpp-build-tools from the indicated site as requested in the error message...

tensorflow / models