Skip truncated images during novelty detection and instance segmentation. Errors below.
Novelty detection (job ID 7):
Error while executing python command '/var/www/vendor/biigle/maia/src/config/../resources/scripts/novelty-detection/DetectionRunner.py /var/www/storage/maia_jobs/maia-7-novelty-detection/input.json':
Traceback (most recent call last):
File "/var/www/vendor/biigle/maia/src/config/../resources/scripts/novelty-detection/DetectionRunner.py", line 120, in <module>
runner.run()
File "/var/www/vendor/biigle/maia/src/config/../resources/scripts/novelty-detection/DetectionRunner.py", line 57, in run
clusters = images.make_clusters(number=self.clusters)
File "/var/www/vendor/biigle/maia/src/resources/scripts/novelty-detection/ImageCollection.py", line 53, in make_clusters
pca_features = np.array(list(pca_features))
File "/usr/lib/python3.5/concurrent/futures/_base.py", line 556, in result_iterator
yield future.result()
File "/usr/lib/python3.5/concurrent/futures/_base.py", line 398, in result
return self.__get_result()
File "/usr/lib/python3.5/concurrent/futures/_base.py", line 357, in __get_result
raise self._exception
File "/usr/lib/python3.5/concurrent/futures/thread.py", line 55, in run
result = self.fn(*self.args, **self.kwargs)
File "/var/www/vendor/biigle/maia/src/resources/scripts/novelty-detection/ImageCollection.py", line 10, in extract_image_pca_features
return image.extract_pca_features()
File "/var/www/vendor/biigle/maia/src/resources/scripts/novelty-detection/Image.py", line 37, in extract_pca_features
return np.array(self._get_resized_image()).flatten()
File "/var/www/vendor/biigle/maia/src/resources/scripts/novelty-detection/Image.py", line 34, in _get_resized_image
return img.resize((width, height), PIL.Image.BILINEAR)
File "/usr/local/lib/python3.5/dist-packages/PIL/Image.py", line 1782, in resize
self.load()
File "/usr/local/lib/python3.5/dist-packages/PIL/ImageFile.py", line 236, in load
len(b))
OSError: image file is truncated (92 bytes not processed)
Instance segmentation (job ID 8):
Error while executing python command '/var/www/vendor/biigle/maia/src/config/../resources/scripts/instance-segmentation/InferenceRunner.py /var/www/storage/maia_jobs/maia-8-instance-segmentation/input-inference.json /var/www/storage/maia_jobs/maia-8-instance-segmentation/output-dataset.json /var/www/storage/maia_jobs/maia-8-instance-segmentation/output-training.json':
Using TensorFlow backend.
Configurations:
AUGMENTATION Sometimes(p=Binomial(Deterministic(float 0.50000000)), name=UnnamedSometimes, then_list=Sequential(name=UnnamedSometimes-then, random_order=False, children=[Fliplr(name=UnnamedFliplr, parameters=[Binomial(Deterministic(float 0.50000000))], deterministic=False), Flipud(name=UnnamedFlipud, parameters=[Binomial(Deterministic(float 0.50000000))], deterministic=False), Affine(name=UnnamedAffine, parameters=[Deterministic(float 1.00000000), Deterministic(int 0), Uniform(Deterministic(int -180), Deterministic(int 180)), Deterministic(float 0.00000000), Deterministic(int 1), Deterministic(int 0), Deterministic(constant), auto], deterministic=False), GaussianBlur(name=UnnamedGaussianBlur, parameters=[Uniform(Deterministic(float 0.00000000), Deterministic(float 2.00000000))], deterministic=False)], deterministic=False), else_list=Sequential(name=UnnamedSometimes-else, random_order=False, children=[], deterministic=False), deterministic=False)
BACKBONE resnet101
BACKBONE_STRIDES [4, 8, 16, 32, 64]
BATCH_SIZE 1
BBOX_STD_DEV [0.1 0.1 0.2 0.2]
COMPUTE_BACKBONE_SHAPE None
DETECTION_MAX_INSTANCES 100
DETECTION_MIN_CONFIDENCE 0.7
DETECTION_NMS_THRESHOLD 0.3
FPN_CLASSIF_FC_LAYERS_SIZE 1024
GPU_COUNT 1
GRADIENT_CLIP_NORM 5.0
IMAGES_PER_GPU 1
IMAGE_CHANNEL_COUNT 3
IMAGE_MAX_DIM 1024
IMAGE_META_SIZE 14
IMAGE_MIN_DIM 64
IMAGE_MIN_SCALE 0
IMAGE_RESIZE_MODE pad64
IMAGE_SHAPE [1024 1024 3]
LEARNING_MOMENTUM 0.9
LEARNING_RATE 0.001
LOSS_WEIGHTS {'mrcnn_class_loss': 1.0, 'mrcnn_mask_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'rpn_bbox_loss': 1.0, 'rpn_class_loss': 1.0}
MASK_POOL_SIZE 14
MASK_SHAPE [28, 28]
MAX_GT_INSTANCES 100
MEAN_PIXEL [147.95957908 164.82751785 173.19284938]
MINI_MASK_SHAPE (56, 56)
NAME maia_inference
NUM_CLASSES 2
POOL_SIZE 7
POST_NMS_ROIS_INFERENCE 1000
POST_NMS_ROIS_TRAINING 2000
PRE_NMS_LIMIT 6000
ROI_POSITIVE_RATIO 0.33
RPN_ANCHOR_RATIOS [0.5, 1, 2]
RPN_ANCHOR_SCALES (32, 64, 128, 256, 512)
RPN_ANCHOR_STRIDE 1
RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2]
RPN_NMS_THRESHOLD 0.7
RPN_TRAIN_ANCHORS_PER_IMAGE 256
STEPS_PER_EPOCH 1000
TOP_DOWN_PYRAMID_SIZE 256
TRAIN_BN False
TRAIN_ROIS_PER_IMAGE 200
USE_MINI_MASK True
USE_RPN_ROIS True
VALIDATION_STEPS 0
WEIGHT_DECAY 0.0001
WARNING:tensorflow:From /usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/sparse_ops.py:1165: sparse_to_dense (from tensorflow.python.ops.sparse_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Create a `tf.sparse.SparseTensor` and use `tf.sparse.to_dense` instead.
2019-01-11 23:16:50.811200: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 AVX512F FMA
2019-01-11 23:16:51.368945: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:964] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2019-01-11 23:16:51.369295: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1432] Found device 0 with properties:
name: Tesla P100-PCIE-16GB major: 6 minor: 0 memoryClockRate(GHz): 1.3285
pciBusID: 0000:00:07.0
totalMemory: 15.90GiB freeMemory: 15.61GiB
2019-01-11 23:16:51.369355: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1511] Adding visible gpu devices: 0
2019-01-11 23:16:51.666487: I tensorflow/core/common_runtime/gpu/gpu_device.cc:982] Device interconnect StreamExecutor with strength 1 edge matrix:
2019-01-11 23:16:51.666556: I tensorflow/core/common_runtime/gpu/gpu_device.cc:988] 0
2019-01-11 23:16:51.666565: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1001] 0: N
2019-01-11 23:16:51.666810: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 15123 MB memory) -> physical GPU (device: 0, name: Tesla P100-PCIE-16GB, pci bus id: 0000:00:07.0, compute capability: 6.0)
Image 1 of 54 (#171024)
Image 2 of 54 (#171031)
Image 3 of 54 (#171023)
Image 4 of 54 (#171027)
Image 5 of 54 (#171035)
Image 6 of 54 (#171034)
Image 7 of 54 (#171022)
Image 8 of 54 (#171038)
Image 9 of 54 (#171039)
Image 10 of 54 (#171037)
Image 11 of 54 (#171041)
Image 12 of 54 (#171042)
Image 13 of 54 (#171043)
Image 14 of 54 (#171044)
Image 15 of 54 (#171045)
Image 16 of 54 (#171046)
Image 17 of 54 (#171047)
Image 18 of 54 (#171028)
Image 19 of 54 (#171049)
Image 20 of 54 (#171050)
Image 21 of 54 (#171051)
Image 22 of 54 (#171052)
Image 23 of 54 (#171053)
Image 24 of 54 (#171025)
Image 25 of 54 (#171055)
Image 26 of 54 (#171056)
Image 27 of 54 (#171054)
Image 28 of 54 (#171058)
Image 29 of 54 (#171059)
Image 30 of 54 (#171060)
Image 31 of 54 (#171061)
Image 32 of 54 (#171062)
Image 33 of 54 (#171026)
Image 34 of 54 (#171064)
Image 35 of 54 (#171065)
Image 36 of 54 (#171066)
Image 37 of 54 (#171067)
Image 38 of 54 (#171068)
Image 39 of 54 (#171069)
Image 40 of 54 (#171070)
Image 41 of 54 (#171071)
Image 42 of 54 (#171072)
Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/skimage/io/_plugins/pil_plugin.py", line 53, in pil_to_ndarray
image.getdata()[0]
File "/usr/local/lib/python3.5/dist-packages/PIL/Image.py", line 1253, in getdata
self.load()
File "/usr/local/lib/python3.5/dist-packages/PIL/ImageFile.py", line 236, in load
len(b))
OSError: image file is truncated (92 bytes not processed)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/var/www/vendor/biigle/maia/src/config/../resources/scripts/instance-segmentation/InferenceRunner.py", line 108, in <module>
runner.run()
File "/var/www/vendor/biigle/maia/src/config/../resources/scripts/instance-segmentation/InferenceRunner.py", line 65, in run
image, info = job.result()
File "/usr/lib/python3.5/concurrent/futures/_base.py", line 398, in result
return self.__get_result()
File "/usr/lib/python3.5/concurrent/futures/_base.py", line 357, in __get_result
raise self._exception
File "/usr/lib/python3.5/concurrent/futures/thread.py", line 55, in run
result = self.fn(*self.args, **self.kwargs)
File "/var/www/vendor/biigle/maia/src/config/../resources/scripts/instance-segmentation/InferenceRunner.py", line 76, in load_image
return self.dataset.load_image(i), self.dataset.image_info[i]
File "/var/www/vendor/biigle/maia/src/resources/scripts/instance-segmentation/mrcnn/utils.py", line 358, in load_image
image = skimage.io.imread(self.image_info[image_id]['path'])
File "/usr/local/lib/python3.5/dist-packages/skimage/io/_io.py", line 62, in imread
img = call_plugin('imread', fname, plugin=plugin, **plugin_args)
File "/usr/local/lib/python3.5/dist-packages/skimage/io/manage_plugins.py", line 214, in call_plugin
return func(*args, **kwargs)
File "/usr/local/lib/python3.5/dist-packages/skimage/io/_plugins/pil_plugin.py", line 37, in imread
return pil_to_ndarray(im, dtype=dtype, img_num=img_num)
File "/usr/local/lib/python3.5/dist-packages/skimage/io/_plugins/pil_plugin.py", line 61, in pil_to_ndarray
raise ValueError(error_message)
ValueError: Could not load ""
Reason: "image file is truncated (92 bytes not processed)"
Please see documentation at: http://pillow.readthedocs.org/en/latest/installation.html#external-libraries
Skip truncated images during novelty detection and instance segmentation. Errors below.
Novelty detection (job ID 7):
Instance segmentation (job ID 8):