Closed atrlbck closed 4 years ago
mikeyEcology
commented
4 years ago I'm not sure why you're getting that error on your machine. I'm concerned that the code might not be finding your tensorflow, but let's see if this helps:
Can you go into the MLWIC2_helper_files folder and open the file called run.py in a text editor. Go to line 23 where it reads tf.compat.v1.logging.set_verbosity(tf.compat.v1.logging.ERROR) and put a # in front of this to comment it out?
atrlbck
commented
4 years ago After commenting it out and rerunning classify I get this error:
Error in read.table(file = file, header = header, sep = sep, quote = quote, : no lines available in input with this traceback:
| stop("no lines available in input") |
|---|
| read.table(file = file, header = header, sep = sep, quote = quote, dec = dec, fill = fill, comment.char = comment.char, ...) |
|---|
| utils::read.csv(paste0(wd, "/", save_predictions), header = FALSE) |
|---|
| classify(path_prefix = paste0(getwd(), "/images"), data_info = paste0(getwd(), "/image_labels.csv"), model_dir = paste0(getwd(), "/MLWIC2_helper_files/"), log_dir = "empty_animal", save_predictions = "model_predictions.txt", python_loc = "/anaconda3/bin/", os = "Mac", num_classes = 2, ... |
|---|
mikeyEcology
commented
4 years ago Does the output begin with something like: Your data_info file exists...?
atrlbck
commented
4 years ago Before commenting out your suggestion, the output begins like this: Your data_info file exists: /Users/armandotoralbecker/Desktop/MLWIC2/image_labels.csv.
Your `path_prefix exists: /Users/armandotoralbecker/Desktop/MLWIC2/images.
You are not using a Windows computer.
If I run the function after commenting out that line then I get the begining of the output saying Your data_file exists and the error in my last comment.
If I run the function again, it gives me this error: Error in classify(path_prefix = paste0(getwd(), "/images"), data_info = paste0(getwd(), : Your data_info file (containing file names and classifications) does not exist.
but if I go into the helper files folder, the data_info.csv file is there
mikeyEcology
commented
4 years ago Sorry you're having so much trouble with this. Just to confirm that you haven't moved things, so after commenting that out, do you still have this file here /Users/armandotoralbecker/Desktop/MLWIC2/image_labels.csv.
Also, what happens if you just remove the getwd() stuff and replace it with the full path (i.e., /Users/armandotoralbecker/Desktop/MLWIC2/image_labels.csv.)?
atrlbck
commented
4 years ago No worries, I appreciate your help.
Yes. I still have the image_labels.csv file in /Users/armandotoralbecker/Desktop/MLWIC2/image_labels.csv. The MLWIC2 folder I have on my desktop contains a folder with all the images, the MLWIC2 helper files, and the image_labels.csv I created using option 4 with the make_input function.
If I remove the getwd() stuff and use the full paths:
classify( path_prefix = '/Users/armandotoralbecker/Desktop/MLWIC2/images', data_info = '/Users/armandotoralbecker/Desktop/MLWIC2/image_labels.csv', model_dir = '/Users/armandotoralbecker/Desktop/MLWIC2/MLWIC2_helper_files', log_dir = "empty_animal", save_predictions = "model_predictions.txt", python_loc = "/anaconda3/bin/", os = "Mac", num_classes = 2, num_cores = 1, delimiter = ",", architecture = "resnet", depth = "18", top_n = 5, batch_size = 128, num_gpus = 2, make_output = TRUE, output_location = NULL, output_name = "MLWIC2_output.csv", test_tensorflow = FALSE, shiny = FALSE, print_cmd = FALSE )
I get the same error:
Error in read.table(file = file, header = header, sep = sep, quote = quote, : no lines available in input
atrlbck
commented
4 years ago If it helps, here's what comes up in my console (I put test_tensorflow: TRUE just to be sure):
Your data_info file exists: /Users/armandotoralbecker/Desktop/MLWIC2/image_labels.csv.
Your `path_prefix exists: /Users/armandotoralbecker/Desktop/MLWIC2/images.
You are not using a Windows computer.
Tensorflow and Python are properly installed.
You are running tensorflow version 1.4.0
Now proceeding to run classify.
2020-03-27 11:21:30.522257: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
2020-03-27 11:21:36.422284: W tensorflow/core/framework/op_kernel.cc:1192] Not found: /Users/armandotoralbecker/Desktop/MLWIC2/images/images/GC100_17_04170001.JPG; No such file or directory
Namespace(LR_details='19, 30, 44, 53, 0.01, 0.005, 0.001, 0.0005, 0.0001', LR_policy='piecewise_linear', WD_details='30, 0.0005, 0.0', WD_policy='piecewise_linear', architecture='resnet', batch_size=128, chunked_batch_size=64, command='eval', delimiter=',', depth=18, log_debug_info=False, log_device_placement=False, log_dir='empty_animal', max_to_keep=5, num_batches=-1, num_classes=1000, num_epochs=55, num_gpus=2, num_prefetch=2000, num_threads=1, optimizer='momentum', path_prefix='/Users/armandotoralbecker/Desktop/MLWIC2/images', processed_size=[224, 224, 3], raw_size=[256, 256, 3], retrain_from=None, run_metadata=None, run_name='Run-27-03-2020_11-21-30', run_options=None, save_predictions='/Users/armandotoralbecker/Desktop/MLWIC2/MLWIC2_helper_files/model_predictions.txt', shuffle=True, snapshot_prefix='empty_animal', top_n=5, train_info=None, transfer_mode=[0], val_info='/Users/armandotoralbecker/Desktop/MLWIC2/image_labels.csv')
found 1 classes
{0: 0}
Filling queue with 2000 images before starting to train. This may take some time.
Traceback (most recent call last):
File "/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1323, in _do_call
return fn(*args)
File "/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1302, in _run_fn
status, run_metadata)
File "/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/errors_impl.py", line 473, in exit
c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.OutOfRangeError: FIFOQueue '_1_batch/fifo_queue' is closed and has insufficient elements (requested 128, current size 0)
[[Node: batch = QueueDequeueUpToV2[component_types=[DT_FLOAT, DT_INT64, DT_STRING], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](batch/fifo_queue, PlaceholderWithDefault)]]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "run.py", line 412, in
Caused by op 'batch', defined at:
File "run.py", line 412, in
OutOfRangeError (see above for traceback): FIFOQueue '_1_batch/fifo_queue' is closed and has insufficient elements (requested 128, current size 0) [[Node: batch = QueueDequeueUpToV2[component_types=[DT_FLOAT, DT_INT64, DT_STRING], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](batch/fifo_queue, PlaceholderWithDefault)]]
Error in read.table(file = file, header = header, sep = sep, quote = quote, : no lines available in input
mikeyEcology
commented
4 years ago This does help. The problem is that the function is not able to find your image files (or at least some of them). Are you using the example images or your own dataset?
atrlbck
commented
4 years ago I'm using my own data set. What else can I do to make the function find my images besides specifying the path to the folder containing all the images?
mikeyEcology
commented
4 years ago I would first try running to model with the example images just to ensure that everything is set up properly.
Then you'll want to make sure that the file names listed in your image_labels.csv file exist in the path to your images.
Another potential problem could exist if your image file names contain spaces. Is this the case? If so, there is a function called remove_spaces that can get rid of spaces from file names.
atrlbck
commented
4 years ago I was able to figure it out. The csv file created with the make_input function adds the folder name to the name of the image (images/GC100_17_04170001.JPG instead of GC100_17_04170001.JPG). All I had to do is remove the folder name on the image_labels.csv file and the model ran completely.
Thank you for your help @mikeyEcology
mikeyEcology
commented
4 years ago Oh yes. make_input provides the relative path from where you ran the function. Your solution worked, but an alternative solution would be to modify your path_prefix to /Users/armandotoralbecker/Desktop/MLWIC2.
Hi @mikeyEcology,
I'm trying to run the classify function for the empty_animal model but I keep getting this error:
Traceback (most recent call last): File "run.py", line 23, in
tf.compat.v1.logging.set_verbosity(tf.compat.v1.logging.ERROR)
AttributeError: module 'tensorflow.python.util.compat' has no attribute 'v1'
The classify function did not run properly.
I have python v3.6.8, tensorflow v1.14, and Anaconda Navigator v1.9.7
my wd: /Users/armandotoralbecker/Desktop/MLWIC2' with images in "/images" classify( path_prefix = paste0(getwd(), "/images"), data_info = paste0(getwd(), "/image_labels.csv"), model_dir = paste0(getwd(), "/MLWIC2_helper_files/"), log_dir = "empty_animal", save_predictions = "model_predictions.txt", python_loc = "/anaconda3/bin/", os = "Mac", num_classes = 2, num_cores = 1, delimiter = ",", architecture = "resnet", depth = "18", top_n = 5, batch_size = 128, num_gpus = 2, make_output = TRUE, output_location = NULL, output_name = "MLWIC2_output.csv", test_tensorflow = FALSE, shiny = FALSE, print_cmd = FALSE )
thank you