Two questions when evaluate the result(3DRes18FasterR-CNN.csv and fold 0-9's performances )

[GaoShanwen]

@wentaozhu Exause me,I have two questions when evaluate the result ,,,wish for your helping.

1-- how to create the test result file's in main.py? (3DRes18FasterR-CNN.csv) I find main.py's reult havenot the 3DRes18FasterR-CNN.csv file and would transfrom it from pbb.npy,but the probability FROC curve need is in 0~1,the probability in pbb.npy is over it,how can I solve it.

2-- In the READ.md,there are fold 0-9's performances , but in config_training file, the val_set is only one——subset9,,,,,Are the first 9 sets's prefomances from LIDC-IDRI datasets or other reason?

thank you!

[wentaozhu]

1 After you get the pbb.npy, use DeepLung/evaluationScript/frocwrtdetpepchluna16.py to get the result of csv file 2 You can change the config_training a little bit by changing the val_set to the subset you want. In that way, you can get the performance on the subset you want.

[GaoShanwen]

@wentaozhu My question has been solve after your help.

Thank you very much! After got your reply I find that need to read your project carefully.

[Leebanana]

@wentaozhu To get the 10-fold validation results, do you mean I should train and test the model 10 times? I think it would take much of time?

[wentaozhu]

Yes. You need and it takes some time.

[Leebanana]

Yes. You need and it takes some time.

After I get 10-fold results,it means each fold I have 2csv files in each val*, how can i get the 3DRes18FasterR-CNN.csv?

[Leebanana]

@wentaozhu And I found a problem,the csvfile which we saved as predanno-1.5d3.csv or predanno-1d3.csv, but in the function: def getfroc(detp, eps): maxfroc = 0 maxep = 0 for ep in eps: bboxpath = results_path + str(ep) + '/' predannofnamalist = [] for detpthresh in detp: predannofnamalist.append(bboxpath + 'predanno'+ str(detpthresh) + '.csv') froclist = p.map(getfrocvalue, predannofnamalist) if maxfroc < max(froclist): maxep = ep maxfroc = max(froclist) print froclist for detpthresh in detp:

print len(froclist), int((detpthresh-detp[0])/(detp[1]-detp[0]))

frocarr[(ep-eps[0])/(eps[1]-eps[0]), int((detpthresh-detp[0])/(detp[1]-detp[0]))] = froclist[int((detpthresh-detp[0])/(detp[1]-detp[0]))] print 'ep', ep, 'detp', detpthresh, froclist[int((detpthresh-detp[0])/(detp[1]-detp[0]))] print maxfroc, maxep

the csvfile the function used is named as predanno-1.5.csv or predanno-1.csv, could you give me a advice about that?

[wentaozhu]

You need to concatenate for the final prediction. You can revise the code a little bit to solve the file name related problem.

[Leebanana]

@wentaozhu Thanks a lot! I want to know how do you get the annotation_excluded.csv file, cause I can't find it in the luna16 dataset file,do you generate it by yourself?

[wentaozhu]

There are from the Luna 16 website and the dataset file.

[Leebanana]

@wentaozhu Sorry to bother you again,I am still confused about concatenating the result csvfile,do you mean I should concatenate the 150 csvfiles in the 150 epochs which was one of the 10-fold trained results to get 1-fold performance?Or I should concatenate the 10 csvfiles which are the results of all 10-fold in every val150 file to get the fina performance? I have been confused couples of days. Could you explained it more detailedly?

[wentaozhu]

The later one

[liuliu66]

@wentaozhu Hi, I meet a problem that when I run the frocwrtdetpepchluna16.py file to get the csv file. How could I change the path augments?

annotations_filename = 'annotations/annotations.csv' annotations_excluded_filename = 'annotations/annotations_excluded.csv' seriesuids_filename = 'annotations/seriesuids.csv' results_path = 'detector/results/res18/retrft969/val150' sideinfopath = '/media/data1/wentao/tianchi/luna16/preprocess/lunaall/'#subset'+str(fold)+'/' +str(fold) datapath = 'LUNA16/subset9'

Now I have changed like this, is it right? what is sideinfopath?

[wentaozhu]

This is your preprocessing path. And put all the images in one folder lunaall.