search

biigle / maia

:m: BIIGLE module for the Machine Learning Assisted Image Annotation method
GNU General Public License v3.0
2 stars 3 forks source link

Upgrade to TensorFlow 2 #68

Closed mzur closed 3 years ago

mzur commented 3 years ago

There are more and more security vulnerabilities in TensorFlow that are no longer fixed for TF1. Upgrade this module to use TF2. Update the Mask R-CNN implementation where necessary (maybe upstream is already compatible).

mzur commented 3 years ago

Now there are 100 active security vulnerabilities in the TF version we use. We should definitely do the upgrade, as I can't say that we aren't affected by one of them.

dlangenk commented 3 years ago

I don't know about how easy it is to port our code over to TF2, but probably this might be a point scout if pytorch might be a good alternative, but this seems only viable if the TF2 port is not very straight forward.

mzur commented 3 years ago

What advantages would pytorch have? I'd definitely choose the path of least effort here (which is probably sticking to TF).

dlangenk commented 3 years ago

The Mask R-CNN module (detectron2) is from the inventor of the Mask R-CNN and there is a nice model zoo with different backbones and pretrainings. I also think it is a bit more memory efficient but this is just a gut feeling. And it can directly read the image files with the respective csvs exported from Biigle without modifications (with a data loader I wrote).

However, if TF2 is easier probably stick to it.

mzur commented 3 years ago

There are some pull requests to establish TF2 support for the Mask R-CNN implementation. If they work, this is probably easiest. Some people also suggest to use Detectron2, which includes a MRCNN implementation, too. This is more actively maintained and may work better/faster. But we need to compare the performance and would have to do some non-trivial code changes. Maybe a student project?

mzur commented 3 years ago

The downside of Detectron2 is that it uses PyTorch instead of TF. But TF is already included (and required for the novelty detection). So maybe we should stick to TF and upgrade the existing MRCNN implementation. I also found a fork that claims to be compatible with TF2: https://github.com/ahmedfgad/Mask-RCNN-TF2

dlangenk commented 3 years ago

There are some pull requests to establish TF2 support for the Mask R-CNN implementation. If they work, this is probably easiest. Some people also suggest to use Detectron2, which includes a MRCNN implementation, too. This is more actively maintained and may work better/faster. But we need to compare the performance and would have to do some non-trivial code changes. Maybe a student project?

That is actually the implementation I meant with PyTorch MaskRCNN

mzur commented 3 years ago

Sorry, read right over that :smile: