Empty cvs file from make_input

[Bartski831]

Hey @mikeyEcology I'm trying to set up MLWIC2 on my windows machine and I've gotten through step 4 with a successful run of the example. However, when I get to Step 5 and try Option 4 I get an empty csv file. Any clues what could be wrong?

make_input( output_dir = "C:/Users/Bartek/Desktop/MLWIC2_helper_files", option = 4, find_file_names = TRUE, path_prefix = "C:/Users/Bartek/Pictures/RemoteCam/103RECNX", image_file_suffixes = c(".jpg", ".JPG"), recursive = FALSE )

[mikeyEcology]

If you go to this path C:/Users/Bartek/Pictures/RemoteCam/103RECNX are there jpg files in that folder? Or are they in sub-folders within that directory?

[mikeyEcology]

Actually, there was an error in the code there and I just updated it. Can you try re-installing MLWIC2 (following step 1) and running step 5 again? You don't need to re-do steps 2-4 again.

[Bartski831]

Still the same issue. I do have a bunch of jpg files in that folder.

[mikeyEcology]

What happens if you run:

path_prefix = "C:/Users/Bartek/Pictures/RemoteCam/103RECNX"
image_file_suffixes = c(".jpg", ".JPG")
pattern <- paste0(image_file_suffixes, collapse="|")
recursive <- FALSE
list.files(path = path_prefix,  
                             pattern=pattern,
                             full.names=FALSE, recursive=recursive)

[Bartski831]

I get a list of the jpg files [1] "IMG_0051.JPG" "IMG_0052.JPG" "IMG_0053.JPG" "IMG_0054.JPG" "IMG_0055.JPG" "IMG_0056.JPG" "IMG_0057.JPG" etc

[mikeyEcology]

@dwwolfson and @Nova-Scotia have you been able to use the make_input function? It's working fine for me so I'm wondering if this is another Windows issue. To test it, could you run make_input with path_prefix = [the location of some image files (i.e., .../MLWIC_examples/images/)] and option=4. You can leave the remaining options as default and then see if the file it created has images and labels in it.

[dwwolfson]

Yes, make_input has worked fine for me in the past using option=4.

[mikeyEcology]

@Bartski831 what happens if you run:

output_dir = "C:/Users/Bartek/Desktop/MLWIC2_helper_files"
path_prefix = "C:/Users/Bartek/Pictures/RemoteCam/103RECNX"
image_file_suffixes = c(".jpg", ".JPG")
pattern <- paste0(image_file_suffixes, collapse="|")
recursive <- FALSE
file_names <- list.files(path = path_prefix,  
                             pattern=pattern,
                             full.names=FALSE, recursive=recursive)
df <- data.frame(file_names, rep(0, length(file_names)))
head(df)
output.file <- file(paste0(output_dir, "/","image_labels.csv"), "wb")
utils::write.table(df, row.names = FALSE, col.names = FALSE, file = output.file, quote = FALSE,append = TRUE, sep = ",")

What is the output of head(df) and does this write a file?

[mitch-fen]

@mikeyEcology @Bartski831 I was also having this problem last week (on Windows), but found that it only occurred if my image files were located outside of the /MLWIC2_helper_files folder. After moving the images to be nested within this folder, the make_input function then worked as expected.

[Bartski831]

@mikeyEcology head(df) produces 1 IMG_0051.JPG 0 2 IMG_0052.JPG 0 3 IMG_0053.JPG 0 4 IMG_0054.JPG 0 5 IMG_0055.JPG 0 6 IMG_0056.JPG 0 It writes a file, but it appears empty. I am unable to delete it as it says it's used by RStudio. rm(list=ls()) doesn't remove the block. After closing RStudio, the 'image_labels.csv' has two columns, one with image file names, and the other with a bunch of zeros, so it works, except I can't view the file with R open.

[Bartski831]

@mikeyEcology @mitch-fen When I copied the image folder into the MLWIC2_helper_files folder and run make_input( output_dir = "C:/Users/Bartek/Desktop/MLWIC2_helper_files", option = 4, find_file_names = TRUE, path_prefix = "C:/Users/Bartek/Desktop/MLWIC2_helper_files/103RECNX", image_file_suffixes = c(".jpg", ".JPG"), recursive = FALSE ) again, I got Error in file(paste0(output_dir, "/", "image_labels.csv"), "wb") : cannot open the connection In addition: Warning message: In file(paste0(output_dir, "/", "image_labels.csv"), "wb") : cannot open file 'C:/Users/Bartek/Desktop/MLWIC2_helper_files/image_labels.csv': Permission denied

[mikeyEcology]

@Bartski831 can you try running this and see if you can open that output file? Note that it will be on your Desktop instead.

output_dir = "C:/Users/Bartek/Desktop"
path_prefix = "C:/Users/Bartek/Pictures/RemoteCam/103RECNX"
image_file_suffixes = c(".jpg", ".JPG")
pattern <- paste0(image_file_suffixes, collapse="|")
recursive <- FALSE
file_names <- list.files(path = path_prefix,  
                             pattern=pattern,
                             full.names=FALSE, recursive=recursive)
df <- data.frame(file_names, rep(0, length(file_names)))
#head(df)
output.file <- file(paste0(output_dir, "/","image_labels.csv"), "wb")
utils::write.table(df, row.names = FALSE, col.names = FALSE, file = output.file, quote = FALSE,append = TRUE, sep = ",")
    close(output.file)
    rm(output.file) 

[Bartski831]

That worked! Could this be a windows permission issue?

[mikeyEcology]

Good! It does look like a permission issue. I assume that the unzipping software is doing something silly with permissions when it unzips. I'm not sure of a good way around this. If anyone else is having this problem try running the make_input function with output_dir as somewhere outside the MLWIC2_helper_files folder

[Bartski831]

Hey @mikeyEcology After taking a break, I am now trying the classify function. I keep getting the output file that is identical to the example from step 4. I am also getting the following message File "run.py", line 21, in <module> import tensorflow as tf ModuleNotFoundError: No module named 'tensorflow' This along with previous issue makes me wonder if I installed everything correctly. Any suggestions on what to check 1st? I run this code classify(path_prefix = "C:/Users/Bartek/Desktop/MLWIC2_helper_files/103RECNX", data_info = "C:/Users/Bartek/Desktop/MLWIC2_helper_files/image_labels2.csv", model_dir = "C:/Users/Bartek/Desktop/MLWIC2_helper_files", python_loc = "C:/Users/Bartek/anaconda3", save_predictions = "model_predictions.txt", make_output = TRUE, output_name = "MLWIC2_output3.csv", num_cores = 4 )

[mikeyEcology]

This means that tensorflow is not properly installed.