Error in running "transfer_feature_extraction.py"

[JasonLiu-TJU]

Hi, I am quite interested in your chest X ray project. But when I run your program of "transfer_feature_extraction.py", I come across the following error. I don't know if it's the right way to bring about the problem. I would be greatly appreciated if you can help me solve this problem.

lj@lj-OptiPlex-790:~/CNN-chest-x-ray-abnormalities-localization$ python learning/transfer_feature_extraction.py data/images data/transfer_features.pickle data/vgg16.tfmodel Images Shape: (74, 224, 224, 3) Traceback (most recent call last): File "learning/transfer_feature_extraction.py", line 52, in main(sys.argv[1], sys.argv[2], sys.argv[3]) File "learning/transfer_feature_extraction.py", line 38, in main features = get_features(images, session) File "learning/transfer_feature_extraction.py", line 16, in get_features 'vgg/images:0': images File "/home/lj/.local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 900, in run run_metadata_ptr) File "/home/lj/.local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1120, in _run self._graph, fetches, feed_dict_tensor, feed_handles=feed_handles) File "/home/lj/.local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 427, in init self._fetch_mapper = _FetchMapper.for_fetch(fetches) File "/home/lj/.local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 253, in for_fetch return _ElementFetchMapper(fetches, contraction_fn) File "/home/lj/.local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 292, in init 'Tensor. (%s)' % (fetch, str(e))) ValueError: Fetch argument 'a - vgg/fc6/Reshape:0' cannot be interpreted as a Tensor. ("The name 'a - vgg/fc6/Reshape:0' refers to a Tensor which does not exist. The operation, 'a - vgg/fc6/Reshape', does not exist in the graph.")

[TomaszRewak]

Hi Jason, I will be more then happy to help you. I just want to make sure that you are using the latest version of this project. What concerns me is that the line numbers if the provided stack trace don't line up with what I see in the learning/transfer_feature_extraction.py file. The main() function in the source code form the repo is called form the line 49 not 52, etc. Can you also confirm that the vgg16.tfmodel file and all images in the images/ directory have been downloaded correctly? I hope that it's not a TF version related problem.

[JasonLiu-TJU]

Hi Tomasz,

Thank you very much for your timely reply and your kindness to help.

1. Versions of software I use.

Ubuntu: 16.04.5 LTS

Tensorflow: 1.9.0

python: 2.7.12

2.line number line up problem

Because I tried to find some solutions from the Internet. A possible solution was to create the graph in the main function instead of in the sub-function, so I added 3 sentences in the script to create the graph. But this turned out to be helpless, so I '#' them, making the system functions the same as your original one. This led to the line number inconsistency.

3. Dataset

Yes. I have downloaded the dataset, thanks to the precise instructions of your page. I have a data folder which has the following structure.

data/

- images

- images-description.json

- vgg16.tfmodel

What's need to be noticed is that I encountered a problem when I was downloading the images, which was a system slow down and it was stuck at the interface of login and I could not login to the system. I used tty1 command line to check the number of items under the images folder. I found the number of items is 7311, and that number kept for 5 minutes without change. I thought the downloading must have been finished so I restarted the computer and run the "transfer_feature_extraction.py" program. And I encountered the problem.

[TomaszRewak]

OK, as I can see I have a little bit older configuration:

• Ubuntu: 14.04.5 LTS
• Tensorflow: 1.6.0 (and this might a source of the problem if it's not elsewhere)
• Python: 2.7.14

As to the number of images. I have 7470 examples downloaded in my images/ dir. I believe that the process should have ended gently, so it might have been interrupted in your case. It's actually easy to check if all of the images have been downloaded. Open images-description.json and check the file name of the last entry (the "name" field of the last object). If it's not present in the images/ dir, then you will have to start the process over. I'm sorry, as one has to download all examples just once, I haven't implemented any safety measures for an occasion like this. But it might also be that the dataset has changed and they have removed those 160 images, so please, consult the images-description.json file.

Can you also confirm that the vgg16.tfmodel has over 540 MB? Maybe the model has not been downloaded fully.

Except for that everything seems to be in order.

[JasonLiu-TJU]

Yes, I found the dataset has not been downloaded completely. I downloaded all the images and made sure there are 7470 images under the folder. And I have checked the vgg16.tfmodel file is over 540MB.

Unfortunately, when I rerun the program, the problem remained. If the program runs well in your computer, I guess it should be the problem of the environment.

[TomaszRewak]

Hi, sorry for the late response, but I've been quite busy lately. Yes, you are right, it seems as it's an environment related problem. Unfortunately there is not much I can do about it at this moment. You might try to run it in venv, though.