aliechoes
commented
3 years ago Hi Pearl! Thanks for showing interest to our work! @DLMLDW can you please help her?
Closed pearlryder closed 3 years ago
aliechoes
commented
3 years ago Hi Pearl! Thanks for showing interest to our work! @DLMLDW can you please help her?
DLMLDW
commented
3 years ago Dear Pearl,
thank you so much for your interest in InstantDL! Sure this is possible.
You can download the dataset from here: https://hmgubox2.helmholtz-muenchen.de/index.php/s/YXRD4a7qHnCa9x5 The one you with the DSB challenge data is called "Multi-Organ_SemanticSegmentation" or "Multi-Organ_InstanceSegmentation", depending on if you want to use semantic or instance segmentation. The .zip files contain the pretrained models (in /logs/Multi-organ_Semantic_Segmentation_weights.hdf5 or /logs/Multi-organ_Instance_Segmentation_weights.h5)
In the config file you then can set: "pretrained_weights": "/....//logs/Multi-organ_Semantic_Segmentation_weights.hdf5", to load the model. If you set: "iterations_over_dataset": 0 it will only evaluate the model on the test set without retraining it.
I hope this answered your question. Please let us know how it goes and don't hesitate to ask if you have any further questions!
Best Dominik
pearlryder
commented
3 years ago Thanks for your help! I'm excited to check it out and was able to download the data and model without any trouble.
I'm running into some errors. I created a conda environment with python 3.6 to run InstantDL. I ran the python setup.py install command without any errors. I then used pip to install the requirements.txt packages (changing the tensorflow package to tensorflow==1.14.0 since I don't have a GPU).
I tried to run from the command line using a config file from the instantdl directory. My config file reads:
{
"use_algorithm": "InstanceSegmentation",
"path": "/Users/pryder/Documents/InstantDL/instantdl-testing",
"iterations_over_dataset": 0,
"pretrained_weights": "/Users/pryder/Documents/InstantDL/Multi-organ_Instance_Segmentation/logs/Multi-organ_Instance_Segmentation_weights.h5"
}When I run python main.py --config ./config.json from the command line, I get several tensorflow related errors. Any idea what's going wrong here?
Cheers, Pearl
Errors:
Using TensorFlow backend.
INFO:root:use_algorithm : InstanceSegmentation
INFO:root:path : /Users/pryder/Documents/InstantDL/instantdl-testing
INFO:root:iterations_over_dataset : 0
INFO:root:pretrained_weights : /Users/pryder/Documents/InstantDL/Multi-organ_Instance_Segmentation/logs/Multi-organ_Instance_Segmentation_weights.h5
WARNING:root:Batchsize has not been set. Setting batchsize = 1
INFO:root:Start learning
INFO:root:InstanceSegmentation
INFO:root:
Configurations:
INFO:root:BACKBONE resnet50
INFO:root:BACKBONE_STRIDES [4, 8, 16, 32, 64]
INFO:root:BATCH_SIZE 1
INFO:root:BBOX_STD_DEV [0.1 0.1 0.2 0.2]
INFO:root:COMPUTE_BACKBONE_SHAPE None
INFO:root:DETECTION_MAX_INSTANCES 400
INFO:root:DETECTION_MIN_CONFIDENCE 0.85
INFO:root:DETECTION_NMS_THRESHOLD 0.3
INFO:root:FPN_CLASSIF_FC_LAYERS_SIZE 1024`
INFO:root:GPU_COUNT 1
INFO:root:GRADIENT_CLIP_NORM 5.0
INFO:root:IMAGES_PER_GPU 1
INFO:root:IMAGE_CHANNEL_COUNT 3
INFO:root:IMAGE_MAX_DIM 512
INFO:root:IMAGE_META_SIZE 14
INFO:root:IMAGE_MIN_DIM 256
INFO:root:IMAGE_MIN_SCALE 0
INFO:root:IMAGE_RESIZE_MODE crop
INFO:root:IMAGE_SHAPE [256 256 3]
INFO:root:LEARNING_MOMENTUM 0.9
INFO:root:LEARNING_RATE 0.0002
INFO:root:LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0}
INFO:root:MASK_POOL_SIZE 14
INFO:root:MASK_SHAPE [28, 28]
INFO:root:MAX_GT_INSTANCES 200
INFO:root:MEAN_PIXEL [43.53 39.56 48.22]
INFO:root:MINI_MASK_SHAPE (56, 56)
INFO:root:NAME Image
INFO:root:NUM_CLASSES 2
INFO:root:POOL_SIZE 7
INFO:root:POST_NMS_ROIS_INFERENCE 2000
INFO:root:POST_NMS_ROIS_TRAINING 1000
INFO:root:PRE_NMS_LIMIT 6000
INFO:root:ROI_POSITIVE_RATIO 0.33
INFO:root:RPN_ANCHOR_RATIOS [0.5, 1, 2]
INFO:root:RPN_ANCHOR_SCALES (8, 16, 32, 64, 256)
INFO:root:RPN_ANCHOR_STRIDE 1
INFO:root:RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2]
INFO:root:RPN_NMS_THRESHOLD 0.8
INFO:root:RPN_TRAIN_ANCHORS_PER_IMAGE 128
INFO:root:STEPS_PER_EPOCH 2
INFO:root:TOP_DOWN_PYRAMID_SIZE 256
INFO:root:TRAIN_BN False
INFO:root:TRAIN_ROIS_PER_IMAGE 200
INFO:root:USE_MINI_MASK True
INFO:root:USE_RPN_ROIS True
INFO:root:VALIDATION_STEPS 10
INFO:root:WEIGHT_DECAY 0.0001
INFO:root:classdefs ('Image', 1, 'Classes')
INFO:root:data_gen_args {}
INFO:root:
WARNING:tensorflow:From /Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/Keras-2.2.4-py3.6.egg/keras/backend/tensorflow_backend.py:517: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.
WARNING:tensorflow:From /Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/Keras-2.2.4-py3.6.egg/keras/backend/tensorflow_backend.py:517: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.
WARNING:tensorflow:From /Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/Keras-2.2.4-py3.6.egg/keras/backend/tensorflow_backend.py:74: The name tf.get_default_graph is deprecated. Please use tf.compat.v1.get_default_graph instead.
WARNING:tensorflow:From /Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/Keras-2.2.4-py3.6.egg/keras/backend/tensorflow_backend.py:74: The name tf.get_default_graph is deprecated. Please use tf.compat.v1.get_default_graph instead.
WARNING:tensorflow:From /Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/Keras-2.2.4-py3.6.egg/keras/backend/tensorflow_backend.py:4138: The name tf.random_uniform is deprecated. Please use tf.random.uniform instead.
WARNING:tensorflow:From /Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/Keras-2.2.4-py3.6.egg/keras/backend/tensorflow_backend.py:4138: The name tf.random_uniform is deprecated. Please use tf.random.uniform instead.
WARNING:tensorflow:From /Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/Keras-2.2.4-py3.6.egg/keras/backend/tensorflow_backend.py:1919: The name tf.nn.fused_batch_norm is deprecated. Please use tf.compat.v1.nn.fused_batch_norm instead.
WARNING:tensorflow:From /Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/Keras-2.2.4-py3.6.egg/keras/backend/tensorflow_backend.py:1919: The name tf.nn.fused_batch_norm is deprecated. Please use tf.compat.v1.nn.fused_batch_norm instead.
WARNING:tensorflow:From /Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/Keras-2.2.4-py3.6.egg/keras/backend/tensorflow_backend.py:3976: The name tf.nn.max_pool is deprecated. Please use tf.nn.max_pool2d instead.
WARNING:tensorflow:From /Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/Keras-2.2.4-py3.6.egg/keras/backend/tensorflow_backend.py:3976: The name tf.nn.max_pool is deprecated. Please use tf.nn.max_pool2d instead.
WARNING:tensorflow:From /Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/Keras-2.2.4-py3.6.egg/keras/backend/tensorflow_backend.py:2018: The name tf.image.resize_nearest_neighbor is deprecated. Please use tf.compat.v1.image.resize_nearest_neighbor instead.
WARNING:tensorflow:From /Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/Keras-2.2.4-py3.6.egg/keras/backend/tensorflow_backend.py:2018: The name tf.image.resize_nearest_neighbor is deprecated. Please use tf.compat.v1.image.resize_nearest_neighbor instead.
WARNING:tensorflow:From /Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/tensorflow/python/ops/array_ops.py:1354: add_dispatch_support.<locals>.wrapper (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
WARNING:tensorflow:From /Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/tensorflow/python/ops/array_ops.py:1354: add_dispatch_support.<locals>.wrapper (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
WARNING:tensorflow:From /Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/instantdl-0.0.1-py3.6.egg/instantdl/segmentation/utilsRCNN.py:201: The name tf.log is deprecated. Please use tf.math.log instead.
WARNING:tensorflow:From /Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/instantdl-0.0.1-py3.6.egg/instantdl/segmentation/utilsRCNN.py:201: The name tf.log is deprecated. Please use tf.math.log instead.
WARNING:tensorflow:From /Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/instantdl-0.0.1-py3.6.egg/instantdl/segmentation/RCNNmodel.py:609: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version.
Instructions for updating:
box_ind is deprecated, use box_indices instead
WARNING:tensorflow:From /Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/instantdl-0.0.1-py3.6.egg/instantdl/segmentation/RCNNmodel.py:609: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version.
Instructions for updating:
box_ind is deprecated, use box_indices instead
WARNING:tensorflow:From /Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/Keras-2.2.4-py3.6.egg/keras/backend/tensorflow_backend.py:174: The name tf.get_default_session is deprecated. Please use tf.compat.v1.get_default_session instead.
WARNING:tensorflow:From /Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/Keras-2.2.4-py3.6.egg/keras/backend/tensorflow_backend.py:174: The name tf.get_default_session is deprecated. Please use tf.compat.v1.get_default_session instead.
2020-10-30 13:00:15.809230: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
WARNING:tensorflow:From /Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/instantdl-0.0.1-py3.6.egg/instantdl/segmentation/RCNNmodel.py:729: The name tf.sets.set_intersection is deprecated. Please use tf.sets.intersection instead.
WARNING:tensorflow:From /Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/instantdl-0.0.1-py3.6.egg/instantdl/segmentation/RCNNmodel.py:729: The name tf.sets.set_intersection is deprecated. Please use tf.sets.intersection instead.
WARNING:tensorflow:From /Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/instantdl-0.0.1-py3.6.egg/instantdl/segmentation/RCNNmodel.py:781: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.cast` instead.
WARNING:tensorflow:From /Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/instantdl-0.0.1-py3.6.egg/instantdl/segmentation/RCNNmodel.py:781: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.cast` instead.
Traceback (most recent call last):
File "main.py", line 73, in <module>
start_learning(configs)
File "main.py", line 22, in start_learning
pipeline.run()
File "/Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/instantdl-0.0.1-py3.6.egg/instantdl/segmentation/InstanceSegmentation.py", line 126, in run
File "/Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/instantdl-0.0.1-py3.6.egg/instantdl/segmentation/RCNNmodel.py", line 2092, in find_last
StopIteration```Dear Pearl,
Thank you very much for taking the time and patience! We have thought that our users would most probable retrain the models on their own data and have therefore evaluate the model in the „/logs/„ folder created while training with InstantDL.
However it is possible to also run it in your case. I have reproduced the error and I think the solution is:
You can add „maskrcnn“ to beginning of the filename of the pretrained weights, and put this into your data directory. So I believe it should look like this:
"pretrained_weights": "/Users/pryder/Documents/InstantDL/instantdl-testing/logs/mask_rcnn_Multi-organ_Instance_Segmentation_weights.h5“,
To explain how we thought it would be used: When training InstantDL, with a pretrained model as initialization, it would not matter where you put the weights and how you name them, as InstantDL will save the InstanceSegmentation model in the logs-folder in the data directory ("/Users/pryder/Documents/InstantDL/instantdl-testing/logs/“ in your case) and save the model with the name "mask_rcnn.h5“, which it will then open during interference.
Thank you very much for pointing this out, we will update InstantDL in the next days, to solve this bug.
We are working on updating InstantDL to Tensorflow 2.XX, then hopefully the tensorflow error messages will be gone.
I hope this helps! Please let us know if you experience any other issues - I am also very happy to have a call with you.
Best Dominik
pearlryder
commented
3 years ago Thanks Dominik! That does seem to have fixed the error and I'm able to progress further - I'm able to load the weights and then start testing with them. After that, though, I run into a File Not Found error: FileNotFoundError: No such file: '/Users/pryder/Documents/InstantDL/instantdl-testing/test/images/image/images.png'
It's not critical for to me to be able to test InstantDL with my images if that feature is not yet available, so no worries if you don't have time to work on this!
Full error trace:
Using TensorFlow backend.
INFO:root:use_algorithm : InstanceSegmentation
INFO:root:path : /Users/pryder/Documents/InstantDL/instantdl-testing
INFO:root:iterations_over_dataset : 0
INFO:root:pretrained_weights : /Users/pryder/Documents/InstantDL/instantdl-testing/logs/mask_rcnn_Multi-organ_Instance_Segmentation_weights.h5
WARNING:root:Batchsize has not been set. Setting batchsize = 1
INFO:root:Start learning
INFO:root:InstanceSegmentation
INFO:root:
Configurations:
INFO:root:BACKBONE resnet50
INFO:root:BACKBONE_STRIDES [4, 8, 16, 32, 64]
INFO:root:BATCH_SIZE 1
INFO:root:BBOX_STD_DEV [0.1 0.1 0.2 0.2]
INFO:root:COMPUTE_BACKBONE_SHAPE None
INFO:root:DETECTION_MAX_INSTANCES 400
INFO:root:DETECTION_MIN_CONFIDENCE 0.85
INFO:root:DETECTION_NMS_THRESHOLD 0.3
INFO:root:FPN_CLASSIF_FC_LAYERS_SIZE 1024
INFO:root:GPU_COUNT 1
INFO:root:GRADIENT_CLIP_NORM 5.0
INFO:root:IMAGES_PER_GPU 1
INFO:root:IMAGE_CHANNEL_COUNT 3
INFO:root:IMAGE_MAX_DIM 512
INFO:root:IMAGE_META_SIZE 14
INFO:root:IMAGE_MIN_DIM 256
INFO:root:IMAGE_MIN_SCALE 0
INFO:root:IMAGE_RESIZE_MODE crop
INFO:root:IMAGE_SHAPE [256 256 3]
INFO:root:LEARNING_MOMENTUM 0.9
INFO:root:LEARNING_RATE 0.0002
INFO:root:LOSS_WEIGHTS {'rpn_class_loss': 1.0, 'rpn_bbox_loss': 1.0, 'mrcnn_class_loss': 1.0, 'mrcnn_bbox_loss': 1.0, 'mrcnn_mask_loss': 1.0}
INFO:root:MASK_POOL_SIZE 14
INFO:root:MASK_SHAPE [28, 28]
INFO:root:MAX_GT_INSTANCES 200
INFO:root:MEAN_PIXEL [43.53 39.56 48.22]
INFO:root:MINI_MASK_SHAPE (56, 56)
INFO:root:NAME Image
INFO:root:NUM_CLASSES 2
INFO:root:POOL_SIZE 7
INFO:root:POST_NMS_ROIS_INFERENCE 2000
INFO:root:POST_NMS_ROIS_TRAINING 1000
INFO:root:PRE_NMS_LIMIT 6000
INFO:root:ROI_POSITIVE_RATIO 0.33
INFO:root:RPN_ANCHOR_RATIOS [0.5, 1, 2]
INFO:root:RPN_ANCHOR_SCALES (8, 16, 32, 64, 256)
INFO:root:RPN_ANCHOR_STRIDE 1
INFO:root:RPN_BBOX_STD_DEV [0.1 0.1 0.2 0.2]
INFO:root:RPN_NMS_THRESHOLD 0.8
INFO:root:RPN_TRAIN_ANCHORS_PER_IMAGE 128
INFO:root:STEPS_PER_EPOCH 2
INFO:root:TOP_DOWN_PYRAMID_SIZE 256
INFO:root:TRAIN_BN False
INFO:root:TRAIN_ROIS_PER_IMAGE 200
INFO:root:USE_MINI_MASK True
INFO:root:USE_RPN_ROIS True
INFO:root:VALIDATION_STEPS 10
INFO:root:WEIGHT_DECAY 0.0001
INFO:root:classdefs ('Image', 1, 'Classes')
INFO:root:data_gen_args {}
INFO:root:
WARNING:tensorflow:From /Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/Keras-2.2.4-py3.6.egg/keras/backend/tensorflow_backend.py:517: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.
WARNING:tensorflow:From /Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/Keras-2.2.4-py3.6.egg/keras/backend/tensorflow_backend.py:517: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.
WARNING:tensorflow:From /Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/Keras-2.2.4-py3.6.egg/keras/backend/tensorflow_backend.py:74: The name tf.get_default_graph is deprecated. Please use tf.compat.v1.get_default_graph instead.
WARNING:tensorflow:From /Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/Keras-2.2.4-py3.6.egg/keras/backend/tensorflow_backend.py:74: The name tf.get_default_graph is deprecated. Please use tf.compat.v1.get_default_graph instead.
WARNING:tensorflow:From /Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/Keras-2.2.4-py3.6.egg/keras/backend/tensorflow_backend.py:4138: The name tf.random_uniform is deprecated. Please use tf.random.uniform instead.
WARNING:tensorflow:From /Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/Keras-2.2.4-py3.6.egg/keras/backend/tensorflow_backend.py:4138: The name tf.random_uniform is deprecated. Please use tf.random.uniform instead.
WARNING:tensorflow:From /Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/Keras-2.2.4-py3.6.egg/keras/backend/tensorflow_backend.py:1919: The name tf.nn.fused_batch_norm is deprecated. Please use tf.compat.v1.nn.fused_batch_norm instead.
WARNING:tensorflow:From /Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/Keras-2.2.4-py3.6.egg/keras/backend/tensorflow_backend.py:1919: The name tf.nn.fused_batch_norm is deprecated. Please use tf.compat.v1.nn.fused_batch_norm instead.
WARNING:tensorflow:From /Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/Keras-2.2.4-py3.6.egg/keras/backend/tensorflow_backend.py:3976: The name tf.nn.max_pool is deprecated. Please use tf.nn.max_pool2d instead.
WARNING:tensorflow:From /Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/Keras-2.2.4-py3.6.egg/keras/backend/tensorflow_backend.py:3976: The name tf.nn.max_pool is deprecated. Please use tf.nn.max_pool2d instead.
WARNING:tensorflow:From /Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/Keras-2.2.4-py3.6.egg/keras/backend/tensorflow_backend.py:2018: The name tf.image.resize_nearest_neighbor is deprecated. Please use tf.compat.v1.image.resize_nearest_neighbor instead.
WARNING:tensorflow:From /Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/Keras-2.2.4-py3.6.egg/keras/backend/tensorflow_backend.py:2018: The name tf.image.resize_nearest_neighbor is deprecated. Please use tf.compat.v1.image.resize_nearest_neighbor instead.
WARNING:tensorflow:From /Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/tensorflow/python/ops/array_ops.py:1354: add_dispatch_support.<locals>.wrapper (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
WARNING:tensorflow:From /Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/tensorflow/python/ops/array_ops.py:1354: add_dispatch_support.<locals>.wrapper (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
WARNING:tensorflow:From /Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/instantdl-0.0.1-py3.6.egg/instantdl/segmentation/utilsRCNN.py:201: The name tf.log is deprecated. Please use tf.math.log instead.
WARNING:tensorflow:From /Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/instantdl-0.0.1-py3.6.egg/instantdl/segmentation/utilsRCNN.py:201: The name tf.log is deprecated. Please use tf.math.log instead.
WARNING:tensorflow:From /Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/instantdl-0.0.1-py3.6.egg/instantdl/segmentation/RCNNmodel.py:609: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version.
Instructions for updating:
box_ind is deprecated, use box_indices instead
WARNING:tensorflow:From /Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/instantdl-0.0.1-py3.6.egg/instantdl/segmentation/RCNNmodel.py:609: calling crop_and_resize_v1 (from tensorflow.python.ops.image_ops_impl) with box_ind is deprecated and will be removed in a future version.
Instructions for updating:
box_ind is deprecated, use box_indices instead
WARNING:tensorflow:From /Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/Keras-2.2.4-py3.6.egg/keras/backend/tensorflow_backend.py:174: The name tf.get_default_session is deprecated. Please use tf.compat.v1.get_default_session instead.
WARNING:tensorflow:From /Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/Keras-2.2.4-py3.6.egg/keras/backend/tensorflow_backend.py:174: The name tf.get_default_session is deprecated. Please use tf.compat.v1.get_default_session instead.
2020-11-02 08:48:02.610094: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
WARNING:tensorflow:From /Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/instantdl-0.0.1-py3.6.egg/instantdl/segmentation/RCNNmodel.py:729: The name tf.sets.set_intersection is deprecated. Please use tf.sets.intersection instead.
WARNING:tensorflow:From /Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/instantdl-0.0.1-py3.6.egg/instantdl/segmentation/RCNNmodel.py:729: The name tf.sets.set_intersection is deprecated. Please use tf.sets.intersection instead.
WARNING:tensorflow:From /Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/instantdl-0.0.1-py3.6.egg/instantdl/segmentation/RCNNmodel.py:781: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.cast` instead.
WARNING:tensorflow:From /Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/instantdl-0.0.1-py3.6.egg/instantdl/segmentation/RCNNmodel.py:781: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.cast` instead.
INFO:root:Running on /Users/pryder/Documents/InstantDL/instantdl-testing
--- Logging error ---
Traceback (most recent call last):
File "/Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/logging/__init__.py", line 994, in emit
msg = self.format(record)
File "/Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/logging/__init__.py", line 840, in format
return fmt.format(record)
File "/Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/logging/__init__.py", line 577, in format
record.message = record.getMessage()
File "/Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/logging/__init__.py", line 338, in getMessage
msg = msg % self.args
TypeError: not all arguments converted during string formatting
Call stack:
File "main.py", line 73, in <module>
start_learning(configs)
File "main.py", line 22, in start_learning
pipeline.run()
File "/Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/instantdl-0.0.1-py3.6.egg/instantdl/segmentation/InstanceSegmentation.py", line 129, in run
detect(model, dataset, testsubset, RESULTS_DIR, test_Val_IMAGE_IDS)
File "/Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/instantdl-0.0.1-py3.6.egg/instantdl/segmentation/RCNNSettings.py", line 463, in detect
dataset.load_dataset(dataset_dir, subset, VAL_IMAGE_IDS)
File "/Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/instantdl-0.0.1-py3.6.egg/instantdl/segmentation/RCNNSettings.py", line 305, in load_dataset
logging.info("Datasetdir", dataset_dir)
Message: 'Datasetdir'
Arguments: ('/Users/pryder/Documents/InstantDL/instantdl-testing/test',)
--- Logging error ---
Traceback (most recent call last):
File "/Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/logging/__init__.py", line 994, in emit
msg = self.format(record)
File "/Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/logging/__init__.py", line 840, in format
return fmt.format(record)
File "/Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/logging/__init__.py", line 577, in format
record.message = record.getMessage()
File "/Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/logging/__init__.py", line 338, in getMessage
msg = msg % self.args
TypeError: not all arguments converted during string formatting
Call stack:
File "main.py", line 73, in <module>
start_learning(configs)
File "main.py", line 22, in start_learning
pipeline.run()
File "/Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/instantdl-0.0.1-py3.6.egg/instantdl/segmentation/InstanceSegmentation.py", line 129, in run
detect(model, dataset, testsubset, RESULTS_DIR, test_Val_IMAGE_IDS)
File "/Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/instantdl-0.0.1-py3.6.egg/instantdl/segmentation/RCNNSettings.py", line 468, in detect
logging.info("Test image id", image_id)
Message: 'Test image id'
Arguments: (0,)
Traceback (most recent call last):
File "main.py", line 73, in <module>
start_learning(configs)
File "main.py", line 22, in start_learning
pipeline.run()
File "/Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/instantdl-0.0.1-py3.6.egg/instantdl/segmentation/InstanceSegmentation.py", line 129, in run
File "/Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/instantdl-0.0.1-py3.6.egg/instantdl/segmentation/RCNNSettings.py", line 470, in detect
File "/Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/instantdl-0.0.1-py3.6.egg/instantdl/segmentation/utilsRCNN.py", line 361, in load_image
File "/Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/instantdl-0.0.1-py3.6.egg/instantdl/data_generator/data_generator.py", line 67, in import_image
File "/Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/skimage/io/_io.py", line 61, in imread
img = call_plugin('imread', fname, plugin=plugin, **plugin_args)
File "/Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/skimage/io/manage_plugins.py", line 210, in call_plugin
return func(*args, **kwargs)
File "/Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/imageio/core/functions.py", line 264, in imread
reader = read(uri, format, "i", **kwargs)
File "/Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/imageio/core/functions.py", line 173, in get_reader
request = Request(uri, "r" + mode, **kwargs)
File "/Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/imageio/core/request.py", line 126, in __init__
self._parse_uri(uri)
File "/Users/pryder/opt/anaconda3/envs/instantdl/lib/python3.6/site-packages/imageio/core/request.py", line 278, in _parse_uri
raise FileNotFoundError("No such file: '%s'" % fn)
FileNotFoundError: No such file: '/Users/pryder/Documents/InstantDL/instantdl-testing/test/images/image/images.png'Dear Pearl,
I’m very happy to hear that!
Of cause you can test InstantDL on your own images, that's what the pipeline is build for!
I don’t know the exact folder names you use, but for InstanceSegmentation the folder setup is not straight-forward as for semanic segmentation, because there instance segmentation masks must be provided.
For example: If you have say three images in your test set named "images.png", "images1.png" and "images2.png" they need to be sorted in the following folders: „/Users/pryder/Documents/InstantDL/instantdl-testing/test/images/image/images.png" „/Users/pryder/Documents/InstantDL/instantdl-testing/test/images1/image/images1.png" „/Users/pryder/Documents/InstantDL/instantdl-testing/test/images2/image/images2.png"
Maybe its easiest to follow the explanation we provide here: https://github.com/marrlab/InstantDL/tree/master/docs/examples/InstanceSegmentation
The first image on this page explains how the folder structure for InstanceSegmentation must be set up.
If you happen to have more many images, you can use this Jupiter-notebook to create the folders and sort the images in the correct folders for you: https://github.com/marrlab/InstantDL/blob/master/docs/preprocessing/PrepareDataforMaskRCNN.ipynb
There you can put all your images into the folder: path_in = "/docs/examples/SemanticSegmentation/" and it will create the desired folder structure and copy the images to: path_out = "/docs/examples/InstanceSegmentation/"
Does this solve your task?
Best Dominik
pearlryder
commented
3 years ago Ahh, I see. I didn't remember that instance segmentation had a different data structure. I was able to restructure according to your example and now it works. Thanks for your help, Dominik! I'm all set.
Hello,
I'm interested in testing out the InstantDL workflow using images of fluorescent nuclei from the 2018 DSB stage 1 test dataset (https://bbbc.broadinstitute.org/BBBC038) before training my own model. I'd love to try out the model that you've trained on this data for your paper on biorxiv (Fig. 3A). Is it possible to do so with the current InstantDL workflow?
Thanks! Looks like a cool resource and I'm excited to check it out. Pearl