Reproducing the results in Pascal VOC

[TheRevanchist]

Hi,

Is anyone able to reproduce the results posted here in Pascal VOC? I am using the same hyperparams as suggested (so 200 epochs, pretrained VGG backbone, Multi-step scheduler, 24 batch size), but I was unable to reach over 0.63 (which is far worse than 0.77).

It would be very helpful if someone can confirm that s(he) was able to reproduce the results.

[marcelsza]

Hi, I would like to retrain the mb1-ssd model on my own data which are in the PASCAL VOC format. Could you tell me, how you arrranged your dataset directory? Is it neccessary to name the files in a specific way or place them in specificly named subfolders? I have only training and test data (no validation data). For both, training and test data, I have default.tfrecord file and label_map.pbtxt file, but don't really klnow, how I have to arrange them into a directory in order to run the training e.g with:

python train_ssd.py --dataset_type voc_images --datasets --net mb1-ssd --pretrained_ssd models/mobilenet-v1-ssd-mp-0_675.pth --scheduler cosine --lr 0.01 --t_max 100 --validation_epochs 5 --num_epochs 100 --base_net_lr 0.001 --batch_size 5

[Philliec459]

Marcelza, did you ever get your model trained? I followed the gun example and back-engineer into how making the setup for training . I have a marine dataset and am using 'buoys' and 'boats' for my training.

It appears that when you start to train the train_ssd.py only loads sub-train-annotations-bbox.csv and the sub-test-annotations-bbox.csv. The other csv files in the data/open_images in the guns example do not appear to be loaded.

I have used labelImg.py to box in my marine objects, and this creates an xml file for each image file. I have a program downloaded from GitHub (?) called xml_to_csv.py that creates nearly the sub-train-annotations-bbox.csv or sub-test-annotations-bbox.csv files that are needed for training. There are a few difference in what this creates vs. what was used in Gun example. My program was using pixels for xmin and xmax, ymin and ymax, but the gun .csv data appears to be using decimal values and not pixels. I changed the xml_to_csv.py code to export decimal values since LabelImg does export a width and height for each image.

In your sub-train/test.... csv files, I am not quite sure what columns are required. ImageID of course and the min and max values for x/y, and ClassName. I am not sue on using LabelName to compare to the class-descriptions-boxable.csv? I used the same labels as what was used for guns and shotguns and replaced them with boat and buoy with the same LabelName.

I have about 315 training images and 50 test images, and the results are pretty good. I am working on a Jetson NX. The training is quite fast and video fps appears to be good.

It appears that this repository is not being maintained any more so it might be up to us the users of pythorch-SSD to get the most out of this. I hope I didn't offend anyone.

[marcelsza]

Hey Craig, Thanks for the Information. I also Had to modify some of the Scripts a little bit and the retraining succeeded in the end, i.e. it was possible to detect tires on images, as I wanted. BUT on a Live Video feed the model failed and detected randomly false positives :/ So I tried Out the retrainined gun model of the repo in a Live Test and it failed as well. So there might be some discrepancy between detection on images and Videos. Cheers, Marcel

Craig Phillips notifications@github.com schrieb am Mo., 15. Juni 2020, 01:38:

Marcelza, did you ever get your model trained? I followed the gun example and back-engineer into how making the setup for training . I have a marine dataset and am using 'buoys' and 'boats' for my training.

It appears that when you start to train the train_ssd.py only loads sub-train-annotations-bbox.csv and the sub-test-annotations-bbox.csv. The other csv files in the data/open_images in the guns example do not appear to be loaded.

I have used labelImg.py to box in my marine objects, and this creates an xml file for each image file. I have a program downloaded from GitHub (?) called xml_to_csv.py that creates nearly the sub-train-annotations-bbox.csv or sub-test-annotations-bbox.csv files that are needed for training. There are a few difference in what this creates vs. what was used in Gun example. My program was using pixels for xmin and xmax, ymin and ymax, but the gun .csv data appears to be using decimal values and not pixels. I changed the xml_to_csv.py code to export decimal values since LabelImg does export a width and height for each image.

In your sub-train/test.... csv files, I am not quite sure what columns are required. ImageID of course and the min and max values for x/y, and ClassName. I am not sue on using LabelName to compare to the class-descriptions-boxable.csv? I used the same labels as what was used for guns and shotguns and replaced them with boat and buoy with the same LabelName.

I have about 315 training images and 50 test images, and the results are pretty good. I am working on a Jetson NX. The training is quite fast and video fps appears to be good.

It appears that this repository is not being maintained any more so it might be up to us the users of pythorch-SSD to get the most out of this. I hope I didn't offend anyone.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/qfgaohao/pytorch-ssd/issues/101#issuecomment-643838133, or unsubscribe https://github.com/notifications/unsubscribe-auth/APGW6UHTKOYJJWAYFK6RAHDRWVNQTANCNFSM4KYT7YMA .

[Philliec459]

Hi Marcel, I am not quite sure what the problem is. I tried to recreate the gun example to develop my marine model. I started with 50 images or so with labels from labelImg, but that failed. I then downloaded more images, used labelImg to create the boxes and an xml file for each image, ran xml_to_csv.py to create the required csv files for test and training sets, and now the new model works well on both images and in video too (live or mp4).

The size of the object does make a difference. I have a video of a sailboat moving across the water. The Object Detection is best when the sailboat is larger in the image. As the sailboat moves away and becomes smaller, then there is a point when the sailboat is no longer detected.

[marcelsza]

Oh cool that it worked for you! Which settings did you use (#epochs, learning rate, freezing parameters)? And did you use only positive Images, or also Images which do not contain the desired Objects?

Craig Phillips notifications@github.com schrieb am Mo., 15. Juni 2020, 17:42:

Hi Marcel, I am not quite sure what the problem is. I tried to recreate the gun example to develop my marine model. I started with 50 images or so with labels from labelImg, but that failed. I then downloaded more images, used labelImg to create the boxes and an xml file for each image, ran xml_to_csv.py to create the required csv files for test and training sets, and now the new model works well on both images and in video too (live or mp4).

The size of the object does make a difference. I have a video of a sailboat moving across the water. The Object Detection is best when the sailboat is larger in the image. As the sailboat moves away and becomes smaller, then there is a point when the sailboat is no longer detected.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/qfgaohao/pytorch-ssd/issues/101#issuecomment-644212481, or unsubscribe https://github.com/notifications/unsubscribe-auth/APGW6UHUTKG7PMSQ6PPSRGTRWY6N7ANCNFSM4KYT7YMA .

[Philliec459]

This is the script that I have been using for training. It could be dumb luck that it is working for me.

python3 train_ssd.py --dataset_type open_images --datasets ~/data/open_images --net mb1-ssd --pretrained_ssd models/mobilenet-v1-ssd-mp-0_675.pth --validation_dataset ~/data/open_images --scheduler cosine --lr 0.01 --t_max 100 --validation_epochs 5 --num_epochs 100 --base_net_lr 0.001 --batch_size 5

I have only used images that have a clear object in mind and I only used one object type in each image. It would be interesting to add some training images that have both boats and buoys, but for now I wanted to keep it simple. I tried this on jetson-inference classification too, but with the results I felt that I need to at least be able to provide labels to zero in on the object of interest.

This is what I run to view an mp4 video: python3 run_ssd_live_demo.py mb1-ssd models/mb1-ssd-Epoch-99-Loss-1.9556251300705805.pth models/open-images-model-labels.txt ./sail.mp4

[manrayboy]

@Philliec459 Would you happen to have your modified version of xml_to_csv available. I'm getting stuck on a few parts like imageId, labelname, and the xmin/ymin in decimal format

[Philliec459]

@manrayboy. Here is what I have for xml_to_csv.py, but I will need to explain the additional changes made to the csv at the end.

""" Usage:

This script is to give the command lines for using pytorch-ssd

This is for Jetson Object Detection. I moved all jpeg and xml from movie to labelImg folders under train and test to save.

run xml_to_csv.py in Tensorflow/scripts/preprocessing subdirectory to create train data csv label:

python xml_to_csv2.py \ -i /Users/craig/Documents/src/pytorch-ssd-ON-NX/data/open_images/train \ -o /Users/craig/Documents/src/pytorch-ssd-ON-NX/data/open_images/sub-train-annotations-bbox.csv

run xml_to_csv.py in Tensorflow/scripts/preprocessing subdirectory to create test data csv label:

python xml_to_csv2.py \ -i /Users/craig/Documents/src/pytorch-ssd-ON-NX/data/open_images/test \ -o /Users/craig/Documents/src/pytorch-ssd-ON-NX/data/open_images/sub-test-annotations-bbox.csv

run xml_to_csv.py in Tensorflow/scripts/preprocessing subdirectory to create test data csv label:

python xml_to_csv2.py \ -i /Users/craig/Documents/src/pytorch-ssd-ON-NX/data/open_images/validation \ -o /Users/craig/Documents/src/pytorch-ssd-ON-NX/data/open_images/sub-validation-annotations-bbox.csv

"""

import os import glob import pandas as pd import argparse import xml.etree.ElementTree as ET

def xml_to_csv(path): """Iterates through all .xml files (generated by labelImg) in a given directory and combines them in a single Pandas datagrame.

Parameters:
----------
path : {str}
    The path containing the .xml files
Returns
-------
Pandas DataFrame
    The produced dataframe
"""

xml_list = []
for xml_file in glob.glob(path + '/*.xml'):
    tree = ET.parse(xml_file)
    root = tree.getroot()
    for member in root.findall('object'):
        value = (root.find('filename').text,     #ImageID
                int(root.find('size')[0].text),  #Width
                int(root.find('size')[1].text),  #Height
                int(member[4][0].text),          #Xmin
                int(member[4][2].text),          #Xmax
                int(member[4][1].text),          #Ymin
                int(member[4][3].text),          #Ymax
                member[0].text                   #ClassName                    
                )
        xml_list.append(value)
column_name = ['ImageID', 'width', 'height',
               'Xmin', 'Xmax', 'Ymin', 'Ymax' , 'ClassName']
xml_df = pd.DataFrame(xml_list, columns=column_name)
return xml_df

def main():

Initiate argument parser

parser = argparse.ArgumentParser(
    description="Sample TensorFlow XML-to-CSV converter")
parser.add_argument("-i",
                    "--inputDir",
                    help="Path to the folder where the input .xml files are stored",
                    type=str)
parser.add_argument("-o",
                    "--outputFile",
                    help="Name of output .csv file (including path)", type=str)
args = parser.parse_args()

if(args.inputDir is None):
    args.inputDir = os.getcwd()
if(args.outputFile is None):
    args.outputFile = args.inputDir + "/labels.csv"

assert(os.path.isdir(args.inputDir))

xml_df = xml_to_csv(args.inputDir)
xml_df.to_csv(
    args.outputFile, index=None)
print('Successfully converted xml to csv.')

if name == 'main': main()

[Philliec459]

Sorry for the above. I had copied my notes to the top of the file and I am sure this is a bit confusing. Sorry

[manrayboy]

Not confusing at all. Appreciate it tons. I've been trying to figure this out all week

[Philliec459]

In the example you have the data=>open_images and then all the test, train and validate folders. Under the data=>open_images you need to have your sub-train-annotations-bbox.csv type files that look like the following data. I made up the data for the Source column that was also copied to the id column, but I think the source needs to be consistent with the ClassName column. Some of the other columns I just filled in with '0' since I did not capture this with labelimg.py.

I might put my marine version in GitHub to use as another example to document the process.

ImageID	Source	LabelName	Confidence	XMin	XMax	YMin	YMax	IsOccluded	IsTruncated	IsGroupOf	IsDepiction	IsInside	id	ClassName
schooner-wooden-ship-sailboat-vessel-sea-754872.jpg	labelimg	/m/06nrc	1	0.26796875	0.84375	0.10324826	0.79466357	0	0	0	0	0	/m/06nrc	boat
images-148.jpeg	labelimg	/m/06nrc	1	0.22545455	0.8	0.21311475	0.43169399	0	0	0	0	0	/m/06nrc	boat
ship-frachtschiff-freighter-water-shippi-2205504.jpg	labelimg	/m/06nrc	1	0.0265625	0.80078125	0.71056241	0.93141289	0	0	0	0	0	/m/06nrc	boat
ship-sea-tourism-journey-water-2762527.jpg	labelimg	/m/06nrc	1	0.04609375	0.75703125	0.11366006	0.70072993	0	0	0	0	0	/m/06nrc	boat
images-216.jpeg	labelimg	/m/06nrc	1	0.12741313	0.81081081	0.11340206	0.89175258	0	0	0	0	0	/m/06nrc	boat
images-92.jpeg	labelimg	/m/0gxl3	1	0.3	0.65	0.06	0.845	0	0	0	0	0	/m/0gxl3	buoy
images-86.jpeg	labelimg	/m/0gxl3	1	0.26811594	0.36956522	0.29508197	0.64480874	0	0	0	0	0	/m/0gxl3	buoy
sailing-sailboat-windy-lake-water-coast--691802.jpg	labelimg	/m/06nrc	1	0.6578125	0.9109375	0.12075029	0.82766706	0	0	0	0	0	/m/06nrc	boat
images-45.jpeg	labelimg	/m/0gxl3	1	0.24888889	0.79555556	0.08888889	0.86666667	0	0	0	0	0	/m/0gxl3	buoy
images-79.jpeg	labelimg	/m/0gxl3	1	0.43027888	0.58565737	0.12935323	0.76119403	0	0	0	0	0	/m/0gxl3	buoy
images-78.jpeg	labelimg	/m/0gxl3	1	0.23414634	0.78048781	0.16666667	0.84146342	0	0	0	0	0	/m/0gxl3	buoy
images-44.jpeg	labelimg	/m/0gxl3	1	0.33	0.63333333	0.07738095	0.89880952	0	0	0	0	0	/m/0gxl3	buoy
greece-boats-sailboat-sea-ocean-2665945.jpg	labelimg	/m/06nrc	1	0.31216931	0.56261023	0.35257985	0.64004914	0	0	0	0	0	/m/06nrc	boat
images-50.jpeg	labelimg	/m/0gxl3	1	0.35144928	0.54710145	0.08743169	0.65027322	0	0	0	0	0	/m/0gxl3	buoy
cruise-ship-mediterranean-sea-112002.jpg	labelimg	/m/06nrc	1	0.12578125	0.928125	0.13647059	0.75764706	0	0	0	0	0	/m/06nrc	boat
boat-sailing-boat-rest-calm-silent-1992137.jpg	labelimg	/m/06nrc	1	0.22890625	0.45	0.38452521	0.84056272	0	0	0	0	0	/m/06nrc	boat
images-87.jpeg	labelimg	/m/06nrc	1	0.09818182	0.94181818	0.13114754	0.73770492	0	0	0	0	0	/m/06nrc	boat
images-93.jpeg	labelimg	/m/0gxl3	1	0.34156379	0.67901235	0.08695652	0.71980676	0	0	0	0	0	/m/0gxl3	buoy
images-203.jpeg	labelimg	/m/06nrc	1	0.23636364	0.81090909	0.13114754	0.84153006	0	0	0	0	0	/m/06nrc	boat
boat-dawn-dusk-mast-ocean-sailboat-saili-1867174.jpg	labelimg	/m/06nrc	1	0.29559748	0.74685535	0.1625	0.85390625	0	0	0	0	0	/m/06nrc	boat
cruise-ship-cruiser-cruise-ship-1578528.jpg	labelimg	/m/06nrc	1	0.15859375	0.84609375	0.11045828	0.78613396	0	0	0	0	0	/m/06nrc	boat
lake-sailing-vessel-wind-1618382.jpg	labelimg	/m/06nrc	1	0.1890625	0.575	0.06799531	0.87573271	0	0	0	0	0	/m/06nrc	boat
lake-sailing-vessel-wind-1618382.jpg	labelimg	/m/06nrc	1	0.615625	0.71796875	0.14185229	0.35287222	0	0	0	0	0	/m/06nrc	boat
sailing-vessel-sail-sea-3672.jpg	labelimg	/m/06nrc	1	0.04817708	0.90625	0.15234375	0.75878906	0	0	0	0	0	/m/06nrc	boat
colorline-color-fantasy-cruise-ship-1435642.jpg	labelimg	/m/06nrc	1	0.059375	0.75859375	0.30832356	0.69284877	0	0	0	0	0	/m/06nrc	boat
sailboat-ship-sailing-greenland-boat-shi-459794.jpg	labelimg	/m/06nrc	1	0.14140625	0.709375	0.0912365	0.87755102	0	0	0	0	0	/m/06nrc	boat
images-85.jpeg	labelimg	/m/0gxl3	1	0.315	0.695	0.01	0.735	0	0	0	0	0	/m/0gxl3	buoy
images-91.jpeg	labelimg	/m/0gxl3	1	0.05333333	0.93777778	0.35111111	0.95555556	0	0	0	0	0	/m/0gxl3	buoy
sailboat-sea-boats-browse-387638.jpg	labelimg	/m/06nrc	1	0.3359375	0.546875	0.25208333	0.75833333	0	0	0	0	0	/m/06nrc	boat
images-46.jpeg	labelimg	/m/0gxl3	1	0.43666667	0.58333333	0.38690476	0.75595238	0	0	0	0	0	/m/0gxl3	buoy
images-52.jpeg	labelimg	/m/0gxl3	1	0.57407407	0.75555556	0.36363636	0.82352941	0	0	0	0	0	/m/0gxl3	buoy
images-53.jpeg	labelimg	/m/0gxl3	1	0.355	0.655	0.01	0.835	0	0	0	0	0	/m/0gxl3	buoy

[manrayboy]

@Philliec459 Oh man, if you could put your marine version up that would be the best.

I have another question sorry. How did you get the id, or does it not matter. does it just need to be different than the others.

And the other question is how did you setup your folders

[manrayboy]

I'm getting much closer. Now i'm getting: AttributeError: Can't pickle local object 'TrainAugmentation.__init__.<locals>.<lambda>'

[Philliec459]

I used the sources from the guns and instead of gun I used ClassName buoy and instead of shotgun??? I used boat

Best Regards,

---

E. Craig Phillips CEO and Chief Petrophysicist Crested Butte Petrophysical Consultants

459 Cisneros Lane Crested Butte, CO 81224 USA Office: +1 970-343-0730 Mobile: +1 970-343-0730 email: craig@cbpetro.com Website: www.cbpetro.com GitHub: https://github.com/Philliec459

The information contained in this electronic message is confidential, it constitutes a profesional and/or industrial secret in terms of the current legislation, and is intended for its recipient only. If you receive this message by mistake or if you are not the recipient thereof, please notify the sender and destroy it.

La informacion contenida en este mensaje de datos es confidencial, constituye un secreto industrial y/o profesional en terminos de la legislacion vigente y se encuentra dirigida exclusivamente al destinatario indicado en dicho mensaje. Si usted recibe esta informacion por error o si usted no es el destinatario del mensaje, favor de notificar al emisor, y destruyalo.

On Jul 19, 2020, at 11:03 AM, manrayboy notifications@github.com wrote:

@Philliec459 https://github.com/Philliec459 Oh man, if you could put your marine version up that would be the best.

I have another question sorry. How did you get the id, or does it not matter. does it just need to be different than the others.

And the other question is how did you setup your folders

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/qfgaohao/pytorch-ssd/issues/101#issuecomment-660676657, or unsubscribe https://github.com/notifications/unsubscribe-auth/ANSKYPG3W2QQINCIVWRM32TR4MRNNANCNFSM4KYT7YMA.

[Philliec459]

I will need to study what I have on my Jetson NX and Mac to create a well documented GitHub repository.

Originally Dusty at NVIDIA led me to his fork of pytorch-SSD. I had some of the same questions that you have have. I could not raise an Issue on Dusty's pytorch-SSD so I wrote up the following issues on his Jetson-inference repository, but I did not receive an answer.

"Hi Dusty-nv, Per your suggestion I have been working on your pytorch-ssd repository and it is working great. I have used your gun training example and tried to back-engineer into how you setup your training data. I am using 'buoys' and 'boats' for my training. This of course has led to some questions. I would have asked the questions on the pytorch-ssd repository, but see no way to do this.

It appears that when you start to train the train_ssd.py only loads sub-train-annotations-bbox.csv and the sub-test-annotations-bbox.csv. The other csv files in the data/open_images in the guns example do not appear to be loaded. Is this correct?

I have used labelimg.py to box in my marine objects, and this creates an xml file for each image file. I have a program downloaded from GitHub (?) called xml_to_csv.py that creates the sub-train-annotations-bbox.csv or sub-test-annotations-bbox.csv files that are needed for training. There are a few difference in what this creates vs. what you had. My program uses pixels for xmin and xmax, ymin and ymax, but you appear to be using decimal values and not pixels. Correct?

Finally, in your sub-train/test.... csv files, I am wondering what columns are required. ImageID of course and the min and max values for x/y, and ClassName. Are you using LabelName to compare to the class-descriptions-boxable.csv?

I have about 315 training images and 50 test images, and the results look very good, but I want to make sure that I am not missing anything. I am working on a Jetson NX. The training is quite fast and video fps appears to be good."

[manrayboy]

Ahh gotcha, I also came from Dusty's fork, but I couldn't find information on training it from scratch so I came here as well