No module named augmentors.augmentorFactory

[PabloDIGITS]

Hi:

I have installed the library with the pip command and it shows when I do pip list. Doing import clodsa in python dont give me errors. But when I'm using a script that begins with: from matplotlib import pyplot as plt from clodsa.augmentors.augmentorFactory import createAugmentor import xml.etree.ElementTree as ET import cv2 And I execute it it show me this error:

Traceback (most recent call last):
  File "clodsa.py", line 2, in <module>
    from clodsa.augmentors.augmentorFactory import createAugmentor
  File "/home/nvidia/CLoDSA/clodsa.py", line 2, in <module>
    from clodsa.augmentors.augmentorFactory import createAugmentor
ImportError: No module named augmentors.augmentorFactory

When I list sys.path (where python looks for packages or modules) it shows a list of paths and I one of them I can find that module. How I can fix this error? PD: I'm working for a company in Spain so we can talk in spanish if you want. EDIT I manage to solve this by adding the root folder of the repository to the sys.path of python 2.7. I did that with the following commands:

python
>>> import sys
>>> sys.path.append('/home/nvidia/CLoDSA')

Now I have another error when executing my script from above:

$ python test.py
Using TensorFlow backend.
[libprotobuf FATAL google/protobuf/stubs/common.cc:61] This program requires version 3.5.0 of the Protocol Buffer runtime library, but the installed version is 2.6.1.  Please update your library.  If you compiled the program yourself, make sure that your headers are from the same version of Protocol Buffers as your link-time library.  (Version verification failed in "external/protobuf_archive/src/google/protobuf/any.pb.cc".)
terminate called after throwing an instance of 'google::protobuf::FatalException'
  what():  This program requires version 3.5.0 of the Protocol Buffer runtime library, but the installed version is 2.6.1.  Please update your library.  If you compiled the program yourself, make sure that your headers are from the same version of Protocol Buffers as your link-time library.  (Version verification failed in "external/protobuf_archive/src/google/protobuf/any.pb.cc".)
Aborted (core dumped)

I dont know how to fix this for now. Thanks for help in advance.

[joheras]

Hi Pablo,

the second problem seems to be related to the installation of tensorflow and not to the CLoDSA library.

About the first problem, I recommend you to use a virtual environment to avoid conflicts the versions of the libraries. There is a nice tutorial about how to define a virtual environment in the following link: https://www.pyimagesearch.com/2016/10/24/ubuntu-16-04-how-to-install-opencv/

Let me know if that helps. You can also write me to joheras@gmail.com and talk there in Spanish.

Best, Jónathan

[PabloDIGITS]

Hi, I manage to solve the last problem. I use this command for uninstalling protobuf in python: sudo pip uninstall protobuf And then I reinstalled it with the version 3.5.1. I use this command: pip install protobuf==3.5.1 And last I modify my script adding the following import in the top of it:

import sys
import tensorflow as tf
sys.path.append('/home/nvidia/CLoDSA')
from matplotlib import pyplot as plt
from clodsa.augmentors.augmentorFactory import createAugmentor
from clodsa.augmentors.generator import Generator
import xml.etree.ElementTree as ET
import cv2

I will comment later when I ensure the library works for me. Thanks for the help.

[joheras]

Great! Let me know if you have any further problem.

[PabloDIGITS]

Hi joheras, I use the library to augment data from 10 images with their label files to 80 images with their modified label files, but I'm not sure if the modified label files are correct. Here is an example with an image flipped vertically: In the original image labeled with the tool Yolo_mark, the label file is: 0 0.535547 0.723611 0.911719 0.508333 In the modified image (CLoDSA library), the modified label file is: 0 0.529411764706 0.0505050505051 0.911764705882 0.104377104377 And if I label with Yolo_mark the modified image to see if it is similar to the above, the label file is: 0 0.528906 0.293750 0.921875 0.559722 If I understand the YOLO format is : c(class) x(relative x coordinate of the center of the rectangle) y(relative y coordinate of the center of the rectangle) width(relative width of the rectangle) height(relative height of the rectangle) In a vertical flip transformation the x, width and height of the rectangle should not change but the y can change. As you can see the y coordinate change but the value is near 0, like if the center of the rectangle is in the top border of the image, and the height is different too. Are this differences normal or I am doing something wrong? Thanks for the help.

[joheras]

Hi, I think that you might be doing something wrong because those values are really strange.

I attach you a Jupyter notebook showing how to augment the dataset of images using YOLO: clodsaYOLO.zip

Let me know if that works for you. Otherwise, you might send me one of your images and the annotation and I can try to see what is wrong.

Best, Jónathan

[PabloDIGITS]

Hi: I'm following a similar example here: https://github.com/ancasag/YOLONotebooks/blob/master/CLODSA_Estomas.ipynb This is the script I'm using:

import sys
import os
import tensorflow as tf
sys.path.append('/home/nvidia/CLoDSA')
from matplotlib import pyplot as plt
from clodsa.augmentors.augmentorFactory import createAugmentor
from clodsa.augmentors.generator import Generator
from clodsa.techniques.techniqueFactory import createTechnique
import xml.etree.ElementTree as ET
import cv2
from imutils import paths

PROBLEM = "detection"
ANNOTATION_MODE = "yolo"
INPUT_PATH = "dataset/"
GENERATION_MODE = "linear"
OUTPUT_MODE = "yolo"
OUTPUT_PATH = "train"

augmentor = createAugmentor(PROBLEM, ANNOTATION_MODE, OUTPUT_MODE, GENERATION_MODE, INPUT_PATH, {"outputPath":OUTPUT_PATH})

vFlip = createTechnique("flip",{"flip":0})
augmentor.addGenerator(Generator(vFlip))

hFlip = createTechnique("flip",{"flip":1})
augmentor.addGenerator(Generator(hFlip))

hvFlip = createTechnique("flip",{"flip":-1})
augmentor.addGenerator(Generator(hvFlip))

rotate = createTechnique("rotate",{"angle":90})
augmentor.addGenerator(Generator(rotate))

rotate = createTechnique("rotate",{"angle":180})
augmentor.addGenerator(Generator(rotate))

rotate = createTechnique("rotate",{"angle":270})
augmentor.addGenerator(Generator(rotate))

avgBlur = createTechnique("average_blurring",{"kernel":5})
augmentor.addGenerator(Generator(avgBlur))

avgNoise = createTechnique("gaussian_noise", {"mean":0, "sigma":10})
augmentor.addGenerator(Generator(avgNoise))

augmentor.applyAugmentation()

print(len(list(paths.list_files("train/", validExts = (".jpg")))))

print ("Number of images in the folder")
os.system('ls -1 dataset/*.jpg | wc -l')

And the original image that I'm checking with the original label file: Label file generated with Yolo_mark: image2.txt 0 0.535547 0.723611 0.911719 0.508333 The flipped image generated by the script: Label file generated by the script: 0_0_image2.txt 0 0.529411764706 0.0505050505051 0.911764705882 0.104377104377 What I'm doing wrong? Is my error in that simple script or maybe in the paths? Thank you for the help.

[joheras]

Hi,

there was a bug in my code, and I was considering the width as the height and viceversa, and it worked in my images because there were squares.

I have uploaded a new version of clodsa with the problem solved, you can install it using pip install clodsa==1.2.8

Now, you should obtain the correct result. Let me know if that works for you.

Thanks a lot for reporting this problem. Best, Jónathan

[PabloDIGITS]

Ok, thanks for the effort. I will check it and comment how its going.

[PabloDIGITS]

Hi, Now the problem with the vertical and horizontal flipped images seems to be fixed. The label coordinates make sense. But checking the 90 grades rotated labels I found this coordinates that I think they are not correct. I show the example: Original image with original labels by Yolo_mark: download1.txt 0 0.535547 0.723611 0.911719 0.508333 Rotated 90 grades clockwise image by the script with rotated labels: 0_3_image2.txt 0 0.161616161616 0.929411764706 0.292929292929 1.59411764706

I label the rotated image with Yolo_mark to have a reference and see how the label coordinates should approximately be: 0 0.301562 0.529167 0.537500 0.916667 I think there is something wrong with the rotation technique and the label transformation.

[joheras]

Sorry, there was another bug computing the size of the generated image and that was the problem. It is fixed now and I hope that there is not any additional bug. You can install it the new version using pip install clodsa==1.2.9

Let me know if that works for you now.

I am just curious about your project, could you send me a private email to joheras@gmail.com? Maybe we could collaborate in some way.

Best, Jónathan

[PabloDIGITS]

Hi Jónathan, I checked the version 1.2.9 and now everything seems to be OK and the generated labels are correct. Thank you for fixing the bugs and your effort.

[joheras]

Great! Let me know if you need further help.