Create predictions only for non-labeled images

[olgaliak]

When we're doing Active Learning "cycle" model will be used to get predictions for bbox locations. Currently it's done even for images that's been already reviewed by human experts. This change excludes those reviewed ("tagged") images.

Unit test has been added to test the change. run_all_test.py also is passing.

[yashpande]

Hi Olga! Long time no see 😄 The only suggestion I would have here is that in train/create_predictions.py, line 158, you are doing all_images = np.concatenate((all_images,all_images_this_folder), axis=0). The np.concatenate allocates a new array with memory equal to the sum of the arrays it is concatenating, then makes a copy of all the data in both existing arrays into the new array. This makes concatenating one folder at a time a very time-and-memory-intensive task. Two ways of improving this would be:

1) Instead of get_images_for_prediction returning an np array, it can simply return a list of filenames. Then, this list of filenames can keep being appended to, and once get_images_for_prediction has been run on every folder, you can simply allocate one np array with enough space for all the images from all the folders, then populate that np array. I would suggest this option, since it requires the least additional memory (keep in mind that np.concatente essentially requires twice as much memory because you need space for the existing arrays and the new ones. This sounds super stupid, but the whole reason I am creating np.zeros arrays in the first place instead of creating a list and then converting it into an np array is because I was facing out of memory errors when making a list first (even on the 128GB DSVM).

2) Create a list of np arrays that you append to when you iterate through the folders i.e. all_images = [] then all_images.append(get_images_for_prediction(..)). Then at the end of the iteration you can do all_images = np.concatenate(all_images). This still takes twice as much memory, but it at least saves a lot of time because you only do the concatenation once instead of once for each folder.

Let me know if you have any questions/comments!