Question about eval - Githubissues

AlbertTJU commented 3 years ago

Snipaste_2020-12-09_18-17-33 Thanks for your code，but I when I eval，this error comes out，I want to know how to fix it，thank you！

xwjabc commented 3 years ago

Hi @AlbertTJU, it seems that there is no eval_bdry_thr.txt file in epoch-39-test/nms-eval folder. You can try to remove the epoch-39-test/nms-eval folder and let the MATLAB code to generate it and see if it can fix the issue.

AlbertTJU commented 3 years ago

Thank you for your suggestion, but I have tried, the same error comes out.

xwjabc commented 3 years ago

Hi @AlbertTJU, could you check if epoch-39-test/nms folder is empty or not?

AlbertTJU commented 3 years ago

Empty.

AlbertTJU commented 3 years ago

And I have another question, before run the model, did you resize all the image size and label size to a specific number? I can't find these codes.

xwjabc commented 3 years ago

Hi @AlbertTJU, could you provide a detailed description of the structure of folder epoch-39-test? It seems the folder is not correctly organized and many files are missing.

Besides, this code repo does not resize the image or the label map. It utilizes the FCN-like architecture which can be applied on image of any size.

AlbertTJU commented 3 years ago

Thank you! I want to know how to realize images of any size. The structure of epoch-39-test is in the picture below.

AlbertTJU commented 3 years ago

My network is used like this: nn.Sequential(*list(vgg16_bn.features.children())[0:6]), and remove all fc, but if the sizes of images are not same, there is error in my code. So I want to konw how to realize any input size. Thank you!

xwjabc commented 3 years ago

@AlbertTJU Could you reply in the GitHub issues (https://github.com/xwjabc/hed/issues/28)? It seems I cannot see your attached picture when you replied using E-mail.

Besides, for your second question, in my implementation, I only use batch_size=1, which does not have such issue. For the case of multiple images in a mini-batch, you may consider adding paddings to make all images have the same size.

AlbertTJU commented 3 years ago

@AlbertTJU Could you reply in the GitHub issues (#28)? It seems I cannot see your attached picture when you replied using E-mail.

Besides, for your second question, in my implementation, I only use batch_size=1, which does not have such issue. For the case of multiple images in a mini-batch, you may consider adding paddings to make all images have the same size.

Snipaste_2020-12-12_10-41-35

AlbertTJU commented 3 years ago

Thank you for your suggestion! I fix the issue. And I want to know, your eval code can be applied on any network? Or just hed and rcf? What should I do can be applied on my network. Thank you!

xwjabc commented 3 years ago

@AlbertTJU Given your picture, it seems only mat and png folders are in epoch-39-test. You may need to check if mat folder is empty or not. If it has the generated results, you may need to debug the MATLAB evaluation code and see why the NMS results are not generated.

For your another question, if applied to other networks, my training code may need to modified according since HED extracts multiple feature maps and has several side supervisions, which may not be compatible with all the networks. My evaluation code only requires images and predicted edge maps, which should work on any network.

AlbertTJU commented 3 years ago

How to plot the curves?

AlbertTJU commented 3 years ago

    Thanks a lot !!!! The issues above are all fixed! And I have another question, that is, if I use your eval code on other datasets, I just modify the path will be ok?

xwjabc commented 3 years ago

@AlbertTJU For the evaluation on other datasets, you need to modify both the path of generated predicted edge maps and the path of ground-truth (i.e. ../data/BSDS500/BSDS500/data/groundTruth/test in hed/eval/eval_edge.m)

When you run the eval_edge.m, it should automatically draw precision-recall curves and pop up a MATLAB figure window after calculating the statistics.

AlbertTJU commented 3 years ago

Thank you!

HelloJill commented 3 years ago

My output edge seems good,but i test and plot the p-r curves not good and AP=0.01,what is the problem? I put the predict edge in path .\output\epoch-x-test\mat\ and put the groundtruth in path ./data/BSDS500/BSDS500/data/groundTruth/test, and the nms result is seems good enough.Thank you! @AlbertTJU @xwjabc

xwjabc commented 3 years ago

Since your NMS result is okay, it means the evaluation code is running correctly to process your generated edges. Perhaps the issue is, it cannot correctly load the ground-truth. You may need to debug the ground-truth loading part of the MATLAB script and see if that is the issue.

HelloJill commented 3 years ago

Thank you for your reply.It seems my ground-truth loading part hsas bug,but I don't know whats' the problem.my ground-truth edge is 255，and non-edge is 0,ist hat right？And the _ev1.txt is like this: 0.01 5411 7637.75 5411 5464 0.02 5411 7637.75 5411 5464 0.03 5411 7637.75 5411 5464 0.04 5411 7637.75 5411 5464 0.05 5411 7637.75 5411 5464 0.06 5411 7637.75 5411 5464 0.07 5409 7637.75 5409 5462 0.08 5409 7637.75 5409 5462 0.09 5409 7637.75 5409 5462 0.1 5409 7637.75 5409 5462 0.11 5409 7637.75 5409 5462 0.12 5409 7637.75 5409 5462 0.13 5402 7637.75 5402 5455 0.14 5402 7637.75 5402 5455 0.15 5402 7637.75 5402 5455 0.16 5402 7637.75 5402 5455 0.17 5402 7637.75 5402 5455 0.18 5402 7637.75 5402 5455 0.19 5400 7637.75 5400 5452 0.2 5400 7637.75 5400 5452 0.21 5400 7637.75 5400 5452

@xwjabc Thank you!

AlbertTJU commented 3 years ago

@AlbertTJU For the evaluation on other datasets, you need to modify both the path of generated predicted edge maps and the path of ground-truth (i.e. ../data/BSDS500/BSDS500/data/groundTruth/test in hed/eval/eval_edge.m)

When you run the eval_edge.m, it should automatically draw precision-recall curves and pop up a MATLAB figure window after calculating the statistics.

Hi, I have another question, I try to reimplement your loss function and I noticed your code for dataset have this snippet, I want to know the two red lines are required? I confused about these codes and I need your suggestions. Thank you! Snipaste_2021-03-13_14-57-18

xwjabc commented 3 years ago

Hi @AlbertTJU, these two lines are used to convert the pixel values (range from 0 to 255) from edge map into binary values (0: background, 1: edge).

AlbertTJU commented 3 years ago

font{
    line-height: 1.6;
}
ul,ol{
    padding-left: 20px;
    list-style-position: inside;
}

Ok, thanks！

                            395494013

                                ***@***.***

    签名由
    网易邮箱大师
    定制

On 3/13/2021 15:13，Weijian ***@***.***> wrote：

Hi @AlbertTJU, these two lines are used to convert the pixel values (range from 0 to 255) from edge map into binary values (0: background, 1: edge).

—You are receiving this because you were mentioned.Reply to this email directly, view it on GitHub, or unsubscribe.

AlbertTJU commented 3 years ago

font{
    line-height: 1.6;
}
ul,ol{
    padding-left: 20px;
    list-style-position: inside;
}

Hi, I have another question about mean value. I want to know if I changed datasets such as NYUD, these three number should be changed? Thanks for your reply !

                            395494013

                                ***@***.***

    签名由
    网易邮箱大师
    定制

On 3/13/2021 15:13，Weijian ***@***.***> wrote：

Hi @AlbertTJU, these two lines are used to convert the pixel values (range from 0 to 255) from edge map into binary values (0: background, 1: edge).

—You are receiving this because you were mentioned.Reply to this email directly, view it on GitHub, or unsubscribe.

xwjabc commented 3 years ago

I think there is no need to change the values. However, since I have not trained on the NYUD dataset before, I am not fully sure if all the settings should remain the same or not.

AlbertTJU commented 3 years ago

Thank you for your reply.It seems my ground-truth loading part hsas bug,but I don't know whats' the problem.my ground-truth edge is 255，and non-edge is 0,ist hat right？And the _ev1.txt is like this: 0.01 5411 7637.75 5411 5464 0.02 5411 7637.75 5411 5464 0.03 5411 7637.75 5411 5464 0.04 5411 7637.75 5411 5464 0.05 5411 7637.75 5411 5464 0.06 5411 7637.75 5411 5464 0.07 5409 7637.75 5409 5462 0.08 5409 7637.75 5409 5462 0.09 5409 7637.75 5409 5462 0.1 5409 7637.75 5409 5462 0.11 5409 7637.75 5409 5462 0.12 5409 7637.75 5409 5462 0.13 5402 7637.75 5402 5455 0.14 5402 7637.75 5402 5455 0.15 5402 7637.75 5402 5455 0.16 5402 7637.75 5402 5455 0.17 5402 7637.75 5402 5455 0.18 5402 7637.75 5402 5455 0.19 5400 7637.75 5400 5452 0.2 5400 7637.75 5400 5452 0.21 5400 7637.75 5400 5452

@xwjabc Thank you!

Has the problem been solved? I just watch your _ev1.txt, did you change the dataset? I don't think your .txt file is the result of BSDS500. If you changed the dataset, you need to change the maximun tolerance in eval code at the same time.

AlbertTJU commented 3 years ago

I think there is no need to change the values. However, since I have not trained on the NYUD dataset before, I am not fully sure if all the settings should remain the same or not.

Hi, I have another question. I want to collect my own dataset, so I want to know how to create groundtruth pictures? Which software can be used? Thank you!

xwjabc commented 3 years ago

Hi @AlbertTJU, I think as long as you follow the annotation format in the provided BSDS dataset, it should be fine. Unfortunately, I am not familiar with generating new dataset.

AlbertTJU commented 2 years ago

I think there is no need to change the values. However, since I have not trained on the NYUD dataset before, I am not fully sure if all the settings should remain the same or not.

Hi, I have another question. I want to collect my own dataset, so I want to know how to create groundtruth pictures? Which software can be used? Thank you!

Hi,你现在能生成groundtrue了嘛？

可以了

AlbertTJU commented 2 years ago

font{
    line-height: 1.6;
}
ul,ol{
    padding-left: 20px;
    list-style-position: inside;
}

五个的话，是需要有五个人来标注啊

                            395494013

                                ***@***.***

    签名由
    网易邮箱大师
    定制

On 10/13/2021 14:32，Clerk ***@***.***> wrote：

I think there is no need to change the values. However, since I have not trained on the NYUD dataset before, I am not fully sure if all the settings should remain the same or not.

Hi, I have another question. I want to collect my own dataset, so I want to know how to create groundtruth pictures? Which software can be used? Thank you!

Hi,你现在能生成groundtrue了嘛？

可以了

你是怎么生成那种五个矩阵的groundtrue的啊，能不能指导一下啊

—You are receiving this because you were mentioned.Reply to this email directly, view it on GitHub, or unsubscribe.

AlbertTJU commented 2 years ago

font{
    line-height: 1.6;
}
ul,ol{
    padding-left: 20px;
    list-style-position: inside;
}

那就只有一个了

                            395494013

                                ***@***.***

    签名由
    网易邮箱大师
    定制

On 10/13/2021 14:42，Clerk ***@***.***> wrote： 

font{

    line-height: 1.6;

}

ul,ol{

    padding-left: 20px;

    list-style-position: inside;

}

哦哦，所以你用的数据集就是有五个人标注的吗，可是那个DeepCrack那篇文章用的数据集就一个矩阵啊，该怎么弄啊

                ***@***.***

    签名由网易邮箱大师定制

On 10/13/2021 ***@***.***> wrote： 

font{

    line-height: 1.6;

}

ul,ol{

    padding-left: 20px;

    list-style-position: inside;

}

五个的话，是需要有五个人来标注啊

                            395494013

                                ***@***.***

    签名由

    网易邮箱大师

    定制

On 10/13/2021 14:32，Clerk ***@***.***> wrote：

I think there is no need to change the values. However, since I have not trained on the NYUD dataset before, I am not fully sure if all the settings should remain the same or not.

Hi, I have another question. I want to collect my own dataset, so I want to know how to create groundtruth pictures? Which software can be used? Thank you!

Hi,你现在能生成groundtrue了嘛？

可以了

你是怎么生成那种五个矩阵的groundtrue的啊，能不能指导一下啊

—You are receiving this because you were mentioned.Reply to this email directly, view it on GitHub, or unsubscribe.

—You are receiving this because you commented.Reply to this email directly, view it on GitHub, or unsubscribe.Triage notifications on the go with GitHub Mobile for iOS or Android.

—You are receiving this because you were mentioned.Reply to this email directly, view it on GitHub, or unsubscribe.

AlbertTJU commented 2 years ago

font{
    line-height: 1.6;
}
ul,ol{
    padding-left: 20px;
    list-style-position: inside;
}

需要修改eval的代码，去掉循环，变成单独的就行了

                            395494013

                                ***@***.***

    签名由
    网易邮箱大师
    定制

On 10/13/2021 14:50，Clerk ***@***.***> wrote： 

font{

    line-height: 1.6;

}

ul,ol{

    padding-left: 20px;

    list-style-position: inside;

}

可是那个matlab的程序需要五个的那种，怎么变过去呢？就是评估DeepCrack文章的效果         

                ***@***.***

    签名由网易邮箱大师定制

On 10/13/2021 ***@***.***> wrote： 

font{

    line-height: 1.6;

}

ul,ol{

    padding-left: 20px;

    list-style-position: inside;

}

那就只有一个了

                            395494013

                                ***@***.***

    签名由

    网易邮箱大师

    定制

On 10/13/2021 14:42，Clerk ***@***.***> wrote： 

font{

    line-height: 1.6;

}

ul,ol{

    padding-left: 20px;

    list-style-position: inside;

}

哦哦，所以你用的数据集就是有五个人标注的吗，可是那个DeepCrack那篇文章用的数据集就一个矩阵啊，该怎么弄啊

                ***@***.***

    签名由网易邮箱大师定制

On 10/13/2021 ***@***.***> wrote： 

font{

    line-height: 1.6;

}

ul,ol{

    padding-left: 20px;

    list-style-position: inside;

}

五个的话，是需要有五个人来标注啊

                            395494013

                                ***@***.***

    签名由

    网易邮箱大师

    定制

On 10/13/2021 14:32，Clerk ***@***.***> wrote：

I think there is no need to change the values. However, since I have not trained on the NYUD dataset before, I am not fully sure if all the settings should remain the same or not.

Hi, I have another question. I want to collect my own dataset, so I want to know how to create groundtruth pictures? Which software can be used? Thank you!

Hi,你现在能生成groundtrue了嘛？

可以了

你是怎么生成那种五个矩阵的groundtrue的啊，能不能指导一下啊

—You are receiving this because you were mentioned.Reply to this email directly, view it on GitHub, or unsubscribe.

—You are receiving this because you commented.Reply to this email directly, view it on GitHub, or unsubscribe.Triage notifications on the go with GitHub Mobile for iOS or Android.

—You are receiving this because you were mentioned.Reply to this email directly, view it on GitHub, or unsubscribe.

—You are receiving this because you commented.Reply to this email directly, view it on GitHub, or unsubscribe.Triage notifications on the go with GitHub Mobile for iOS or Android.

—You are receiving this because you were mentioned.Reply to this email directly, view it on GitHub, or unsubscribe.

AlbertTJU commented 2 years ago

font{
    line-height: 1.6;
}
ul,ol{
    padding-left: 20px;
    list-style-position: inside;
}

出现了负值是不是没有用sigmoid激活函数啊

                            395494013

                                ***@***.***

    签名由
    网易邮箱大师
    定制

On 10/13/2021 15:20，Clerk ***@***.***> wrote： 

font{

    line-height: 1.6;

}

ul,ol{

    padding-left: 20px;

    list-style-position: inside;

}

    好的！万分感谢啊！还有一个问题，就是mat矩阵里面按理应该是0-1的概率值，但是我的         DeepCrack预测保存的mat文件里面出现了负值，您有出现过这样的情况吗

On 10/13/2021 ***@***.***> wrote： 

font{

    line-height: 1.6;

}

ul,ol{

    padding-left: 20px;

    list-style-position: inside;

}

需要修改eval的代码，去掉循环，变成单独的就行了

                            395494013

                                ***@***.***

    签名由

    网易邮箱大师

    定制

On 10/13/2021 14:50，Clerk ***@***.***> wrote： 

font{

    line-height: 1.6;

}

ul,ol{

    padding-left: 20px;

    list-style-position: inside;

}

可是那个matlab的程序需要五个的那种，怎么变过去呢？就是评估DeepCrack文章的效果         

                ***@***.***

    签名由网易邮箱大师定制

On 10/13/2021 ***@***.***> wrote： 

font{

    line-height: 1.6;

}

ul,ol{

    padding-left: 20px;

    list-style-position: inside;

}

那就只有一个了

                            395494013

                                ***@***.***

    签名由

    网易邮箱大师

    定制

On 10/13/2021 14:42，Clerk ***@***.***> wrote： 

font{

    line-height: 1.6;

}

ul,ol{

    padding-left: 20px;

    list-style-position: inside;

}

哦哦，所以你用的数据集就是有五个人标注的吗，可是那个DeepCrack那篇文章用的数据集就一个矩阵啊，该怎么弄啊

                ***@***.***

    签名由网易邮箱大师定制

On 10/13/2021 ***@***.***> wrote： 

font{

    line-height: 1.6;

}

ul,ol{

    padding-left: 20px;

    list-style-position: inside;

}

五个的话，是需要有五个人来标注啊

                            395494013

                                ***@***.***

    签名由

    网易邮箱大师

    定制

On 10/13/2021 14:32，Clerk ***@***.***> wrote：

I think there is no need to change the values. However, since I have not trained on the NYUD dataset before, I am not fully sure if all the settings should remain the same or not.

Hi, I have another question. I want to collect my own dataset, so I want to know how to create groundtruth pictures? Which software can be used? Thank you!

Hi,你现在能生成groundtrue了嘛？

可以了

你是怎么生成那种五个矩阵的groundtrue的啊，能不能指导一下啊

—You are receiving this because you were mentioned.Reply to this email directly, view it on GitHub, or unsubscribe.

—You are receiving this because you commented.Reply to this email directly, view it on GitHub, or unsubscribe.Triage notifications on the go with GitHub Mobile for iOS or Android.

—You are receiving this because you were mentioned.Reply to this email directly, view it on GitHub, or unsubscribe.

—You are receiving this because you commented.Reply to this email directly, view it on GitHub, or unsubscribe.Triage notifications on the go with GitHub Mobile for iOS or Android.

—You are receiving this because you were mentioned.Reply to this email directly, view it on GitHub, or unsubscribe.

—You are receiving this because you commented.Reply to this email directly, view it on GitHub, or unsubscribe.Triage notifications on the go with GitHub Mobile for iOS or Android.

—You are receiving this because you were mentioned.Reply to this email directly, view it on GitHub, or unsubscribe.

AlbertTJU commented 2 years ago

font{
    line-height: 1.6;
}
ul,ol{
    padding-left: 20px;
    list-style-position: inside;
}

DeepCrack不就是hed+fpn嘛，还有修改了损失函数，模型搭建好了，换数据集就可以了啊

                            395494013

                                ***@***.***

    签名由
    网易邮箱大师
    定制

On 10/13/2021 15:34，Clerk ***@***.***> wrote： 

font{

    line-height: 1.6;

}

ul,ol{

    padding-left: 20px;

    list-style-position: inside;

}

    哇，大佬，太谢谢啦，你能复现出来deepCrack文章的ap、ois、ods吗

On 10/13/2021 ***@***.***> wrote： 

font{

    line-height: 1.6;

}

ul,ol{

    padding-left: 20px;

    list-style-position: inside;

}

出现了负值是不是没有用sigmoid激活函数啊

                            395494013

                                ***@***.***

    签名由

    网易邮箱大师

    定制

On 10/13/2021 15:20，Clerk ***@***.***> wrote： 

font{

    line-height: 1.6;

}

ul,ol{

    padding-left: 20px;

    list-style-position: inside;

}

    好的！万分感谢啊！还有一个问题，就是mat矩阵里面按理应该是0-1的概率值，但是我的         DeepCrack预测保存的mat文件里面出现了负值，您有出现过这样的情况吗

On 10/13/2021 ***@***.***> wrote： 

font{

    line-height: 1.6;

}

ul,ol{

    padding-left: 20px;

    list-style-position: inside;

}

需要修改eval的代码，去掉循环，变成单独的就行了

                            395494013

                                ***@***.***

    签名由

    网易邮箱大师

    定制

On 10/13/2021 14:50，Clerk ***@***.***> wrote： 

font{

    line-height: 1.6;

}

ul,ol{

    padding-left: 20px;

    list-style-position: inside;

}

可是那个matlab的程序需要五个的那种，怎么变过去呢？就是评估DeepCrack文章的效果         

                ***@***.***

    签名由网易邮箱大师定制

On 10/13/2021 ***@***.***> wrote： 

font{

    line-height: 1.6;

}

ul,ol{

    padding-left: 20px;

    list-style-position: inside;

}

那就只有一个了

                            395494013

                                ***@***.***

    签名由

    网易邮箱大师

    定制

On 10/13/2021 14:42，Clerk ***@***.***> wrote： 

font{

    line-height: 1.6;

}

ul,ol{

    padding-left: 20px;

    list-style-position: inside;

}

哦哦，所以你用的数据集就是有五个人标注的吗，可是那个DeepCrack那篇文章用的数据集就一个矩阵啊，该怎么弄啊

                ***@***.***

    签名由网易邮箱大师定制

On 10/13/2021 ***@***.***> wrote： 

font{

    line-height: 1.6;

}

ul,ol{

    padding-left: 20px;

    list-style-position: inside;

}

五个的话，是需要有五个人来标注啊

                            395494013

                                ***@***.***

    签名由

    网易邮箱大师

    定制

On 10/13/2021 14:32，Clerk ***@***.***> wrote：