search

tensorflow / models

Models and examples built with TensorFlow
Other
77.01k stars 45.78k forks source link

object_detection API protoc installation step fails #4002

Closed radzfoto closed 6 years ago

radzfoto commented 6 years ago

System information

Describe the problem

I followed object_detection installation instructions step by step. When I tried protoc command, .proto files could not be compiled to .py files.

I issued this command:

protoc object_detection/protos/*.proto --python_out=.

And obtained these errors:

object_detection/protos/ssd.proto:87:3: Expected "required", "optional", or "repeated". object_detection/protos/ssd.proto:87:12: Expected field name. object_detection/protos/model.proto: Import "object_detection/protos/ssd.proto" was not found or had errors. object_detection/protos/model.proto:12:5: "Ssd" is not defined.

Note that all the .proto files exist including ssd.proto.

Source code / logs

No custom source code. Just following object_detection API installation instructions

he44 commented 6 years ago

I encountered the same issue and I think it's because of the update/latest commit. I bypassed the protoc error by checking out an older version (2018-04-13).

git reset --hard 490813bdb3499290633919a9867eb0bb6d346d87

(How to checkout previous commit: https://stackoverflow.com/questions/3555107/git-clone-particular-version-of-remote-repository/18037563 )

denglixi commented 6 years ago

Upgrade your protoc.

derekjchow commented 6 years ago

@denglixi is correct. Update your protobuf compiler/runtime to 3.x.x

radzfoto commented 6 years ago

@denglixi and @derekjchow, you were both correct. I removed the 2.6 protobuf-compiler and installed the 3.5.1 protobuf-compiler from github. What is frustrating is that the installation instructions in the object_detection api github still say to use the 2.6 version. I hope that this gets updated. Thank you!

derekjchow commented 6 years ago

Will update the docs for next release

lechatthecat commented 6 years ago

related? https://github.com/tensorflow/models/issues/4047

lechatthecat commented 6 years ago

I have protobuf 3.5.2 and still fail to compile. Why?

object_detection/protos/ssd.proto:87:3: Expected "required", "optional", or "repeated". object_detection/protos/ssd.proto:87:12: Expected field name. object_detection/protos/model.proto: Import "object_detection/protos/ssd.proto" was not found or had errors. object_detection/protos/model.proto:12:5: "Ssd" is not defined.

sandunadhikari commented 6 years ago

Make sure you grab the latest version

curl -OL https://github.com/google/protobuf/releases/download/v3.2.0/protoc-3.2.0-linux-x86_64.zip

Unzip

unzip protoc-3.2.0-linux-x86_64.zip -d protoc3 change your protobuf to protobuf3

Move protoc to /usr/local/bin/

sudo mv protoc3/bin/* /usr/local/bin/

Move protoc3/include to /usr/local/include/

sudo mv protoc3/include/* /usr/local/include/

Optional: change owner

sudo chwon [user] /usr/local/bin/protoc sudo chwon -R [user] /usr/local/include/google

AliceDinh commented 6 years ago

@radzfoto: please show how did you install the 3.5.1 protobuf-compiler from github please! I grab the latest version from: https://github.com/google/protobuf/releases/download/v3.2.0/protoc-3.2.0-linux-x86_64.zip then I don't know what to do next

radzfoto commented 6 years ago

Hi @AliceDinh, here are some basic steps that may help to get you started: To download version 3.5.1 you need to go here: then download file protoc-3.5.1-osx-x86_64.zip. Unzip the file in some directory, let's say something like ~/protobuf/

If you are using Ubuntu or similar, then use the archive manager to unzip everything in the file, or install unzip. Make sure that you put the protoc zip file in your target directory, then with unzip in a terminal window

$ cd ~/protobuf
$ sudo apt-get install unzip
$ unzip protoc-3.5.1-osx-x86_64.zip

You will now see a readme.txt file and two directories, bin and include. The protobuf compiler is a file in the bin directory called protoc. It doesn't hurt to make sure that protoc is executable, so do

$ cd bin
$ chmod +x protoc

Now you could add this directory to your PATH so you can execute from anywhere, for example:

$ export PATH=$PATH:~/protobuf/bin

If you want this to be permanent, add that line to your .bashrc file

To use the protobuf compiler now, all you have to do is

$ protoc /path/to/the/protobuf/file/you/want/to/compile

I hope this helps.

Thaiph1308 commented 6 years ago

@radzfoto I follow you but only 3.3 version have bin and include folder, and when I using protoc --version , it still returns libprotoc 2.6.1 so I still got the same issue like #4223. It's mean that I failed when installing new protoc right? Is there any way I can check I follow exactly what you say? And how can I install protobuf 3.5.1 version (newest version)? It's doesn't contain bin and include folders. I tried install like @sandunadhikari but it still return libprotoc 2.6.1 i'm using ubuntu on Window 10. Thank for your help.

radzfoto commented 6 years ago

@Thaiph1308, if you download the protoc zip file, it should have everything inside. Please make sure you are downloading the correct file and unzipping it correctly. It is very important that you then execute this new version of protoc. You can make sure by putting the directory in the PATH environment variable. And make sure that you remove any other versions of protoc. I am very confused about your comment that you are using "ubuntu on Windows 10". What does that mean? Are you using Ubuntu Linux? Or are your using Windows 10? If you are using a virtualized version of Ubuntu running on something like VirtualBox on Windows, then you are just using Ubuntu.

huyutong commented 6 years ago

@radzfoto I use "which protoc ,sudo rm /usr/local/bin/protoc" , uninstall the protoc2.6.1 , then follow the way you give as above to install protoc3.2.0. when I run " protoc object_detection/protos/*.proto --python_out=." ,it shows "sudo apt install protobuf-compiler", but even if I run ''sudo apt install protobuf-compiler'', it still does not work, it means the protoc was't installed correctly. so,do u have any idea about this ? thans very much

radzfoto commented 6 years ago

Hi @huyutong, if you read my suggested procedure more carefully, I suggest that you do NOT use apt to install the protobuf compiler. Instead go directly to the download website here: https://developers.google.com/protocol-buffers/docs/downloads And then click on the "release page" to select the version for your operating system. Once you download the zip file, just unzip it to a folder, look for the "bin" subdirectory and you will find the protoc executable there. Put that in your PATH and you should be all set.

huyutong commented 6 years ago

@radzfoto I have followed the way you given, but it still shows ''protoc is not installed，you can install by sudo apt install protobuf-compiler'' ,then I tried ''sudo apt install protobuf-compiler'', it didnot work

Thaiph1308 commented 6 years ago

@radzfoto Thanks for your help, I have successfully installed protoc. But I have some concern about file protobuf-python-3.5.1.tar.gz. It's used in Window OS right? How can we know a file is used for what OS? And I'm using Ubuntu from Microsoft Store, direct Ubuntu from Window. @huyutong What OS did u use? If Ubuntu, u have to install some library (make,gcc ...) to compile protobuf-compiler before use it. More details here https://github.com/google/protobuf/blob/master/src/README.md

edisonchensy commented 6 years ago

I got the same problem, please give me a help. my protobuf is 3.5.2 version.

edison@amax-server:~/google_TF_object_detection/models/research$ protoc object_detection/protos/*.proto --python_out=. object_detection/protos/ssd.proto:87:3: Expected "required", "optional", or "repeated". object_detection/protos/ssd.proto:87:12: Expected field name. object_detection/protos/model.proto: Import "object_detection/protos/ssd.proto" was not found or had errors. object_detection/protos/model.proto:12:5: "Ssd" is not defined. edison@amax-server:~/google_TF_object_detection/models/research$ pip install protobuf Requirement already satisfied: protobuf in /home/edison/anaconda3/lib/python3.6/site-packages (3.5.2.post1) Requirement already satisfied: setuptools in /home/edison/anaconda3/lib/python3.6/site-packages (from protobuf) (39.0.1) Requirement already satisfied: six>=1.9 in /home/edison/anaconda3/lib/python3.6/site-packages (from protobuf) (1.11.0)

ETucks commented 6 years ago

I had the same problem, and attempted resolve things by updating to protobuf 3.5.2, but was unsuccessful. However, commmenting out line 87 of object_detection/protos/ssd.proto resolved the issue (As mentioned by mastash3ff in issue #4047 : )

HyperWayne commented 6 years ago

Following @sandunadhikari steps, I check my protoc version, it's still 2.6.1 (--version).
After 

source ~/.bashrc

error message for this compilation disappear.

protoc object_detection/protos/*.proto --python_out=.

Update version of protoc to 3.2.0 works for me.

dogod621 commented 5 years ago

try this pyscript... change the researchRoot for yourself. It works perfectly for me ( I use protobuf 3.6 ). you can execute the script at any folder.

import sys
import os
import subprocess

researchRoot = os.path.normpath( "D:/Program Files (x86)/Microsoft Visual Studio/Shared/Python36_64/Lib/site-packages/tensorflow/models/research/" )
protoRoots_ = os.listdir(researchRoot)
protoRoots = []

for protoRoot_ in protoRoots_:
    if ( protoRoot_ in protoRoots ):
        continue

    for root, dirs, files in os.walk(os.path.join(researchRoot, protoRoot_)):
        if ( protoRoot_ in protoRoots ):
            break

        for name in files:
            if ( os.path.splitext(name)[1] == ".proto" ):
                protoRoots.append( protoRoot_ )
                break

for protoRoot in protoRoots:
    for root, dirs, files in os.walk(os.path.join(researchRoot, protoRoot)):
        for name in files:
            if ( os.path.splitext(name)[1] == ".proto" ):
                cmdStr = \
                        "protoc " + name + \
                        " --proto_path=\"" + root + "\" " + \
                        " --proto_path=\"" + os.path.join(researchRoot, protoRoot) + "\" " + \
                        " --proto_path=\"" + researchRoot + "\" " + \
                        "--python_out=."

                print(cmdStr)

                with subprocess.Popen(cmdStr, stdout=subprocess.PIPE, shell=True, cwd=researchRoot) as proc:
                    stdout, stderr = proc.communicate()
                    cmdExit = proc.returncode

                    if ( cmdExit != 0 ):
                        print ("cmdExit == " + str(cmdExit) )
                        print ("stdout")
                        print (stdout)
                        print ("stderr")
                        print (stderr)
                    else:  
                        print ("cmdExit == 0")
bozcani commented 4 years ago

I downloaded and installed protoc from source, and it worked for me:

# You should be in folder tensorflow/models/research/
$ wget -O protobuf.zip https://github.com/google/protobuf/releases/download/v3.0.0/protoc-3.0.0-linux-x86_64.zip
$ unzip protobuf.zip
$ ./bin/protoc object_detection/protos/*.proto --python_out=.

Then, test the installation:

python object_detection/builders/model_builder_test.py
upasana-mittal commented 4 years ago

I downloaded and installed protoc from source, and it worked for me:

# You should be in folder tensorflow/models/research/
$ wget -O protobuf.zip https://github.com/google/protobuf/releases/download/v3.0.0/protoc-3.0.0-linux-x86_64.zip
$ unzip protobuf.zip
$ ./bin/protoc object_detection/protos/*.proto --python_out=.

Then, test the installation:

python object_detection/builders/model_builder_test.py

It is not working for me @bozcani I tried with the version you have given 3.0.0 also with 3.4.0 but it gave me same error

object_detection/protos/*.proto: No such file or directory

Can you please help?

AliceDinh commented 4 years ago

@upasana-mittal Try to run these: protoc object_detection/protos/anchor_generator.proto --python_out=. protoc object_detection/protos/argmax_matcher.proto --python_out=. protoc object_detection/protos/bipartite_matcher.proto --python_out=. protoc object_detection/protos/box_coder.proto --python_out=. protoc object_detection/protos/box_predictor.proto --python_out=. protoc object_detection/protos/eval.proto --python_out=. protoc object_detection/protos/faster_rcnn.proto --python_out=. protoc object_detection/protos/faster_rcnn_box_coder.proto --python_out=. protoc object_detection/protos/grid_anchor_generator.proto --python_out=. protoc object_detection/protos/hyperparams.proto --python_out=. protoc object_detection/protos/image_resizer.proto --python_out=. protoc object_detection/protos/input_reader.proto --python_out=. protoc object_detection/protos/keypoint_box_coder.proto --python_out=. protoc object_detection/protos/losses.proto --python_out=. protoc object_detection/protos/matcher.proto --python_out=. protoc object_detection/protos/mean_stddev_box_coder.proto --python_out=. protoc object_detection/protos/model.proto --python_out=. protoc object_detection/protos/multiscale_anchor_generator.proto --python_out=. protoc object_detection/protos/optimizer.proto --python_out=. protoc object_detection/protos/pipeline.proto --python_out=. protoc object_detection/protos/post_processing.proto --python_out=. protoc object_detection/protos/preprocessor.proto --python_out=. protoc object_detection/protos/region_similarity_calculator.proto --python_out=. protoc object_detection/protos/square_box_coder.proto --python_out=. protoc object_detection/protos/ssd.proto --python_out=. protoc object_detection/protos/ssd_anchor_generator.proto --python_out=. protoc object_detection/protos/string_int_label_map.proto --python_out=. protoc object_detection/protos/train.proto --python_out=.