[MFR21] ICCV21 masked face recognition challenge discussion

nttstar commented 3 years ago

How to participate: https://github.com/deepinsight/insightface/tree/master/challenges/iccv21-mfr

Submission server: http://iccv21-mfr.com/

Workshop homepage: https://ibug.doc.ic.ac.uk/resources/masked-face-recognition-challenge-workshop-iccv-21/

Any issue can be left in this thread.

ygtxr1997 commented 2 years ago

Could you provide the testing dataset?

guijuzhejiang commented 2 years ago

Can anyone share some tips and ideas?

jacqueline-weng commented 2 years ago

My model has passed the check_onnx.py in local environmnet with cuda10.2, onnx1.6.0, ort1.8.0 but failed the test. Could you give more clues about what is going on? Thank you in advance.

v-trungdt21 commented 2 years ago

Can anybody update the link for the non-China user? I can't access it and submit via the iccv-mfr website is desperately slow for me in this region :(

beingjoey commented 1 year ago

I can not sign up MFR ongo. Is there any other server?

nttstar commented 1 year ago

@beingjoey We're upgrading the whole system, please wait for 1-2 days.

JackLin-Authme commented 1 year ago

I have some problems on testing my model with onnx_helper.py It's seems that self.providers = None would be broken when using onnxruntime-gpu>1.9.0

I think this issue would lead to the load onnx error

nttstar commented 1 year ago

@JesseEisen No such problem in our system.

jovialio commented 1 year ago

Hi i would like to benchmark my model against the MFR dataset and have registered via http://iccv21-mfr.com/#/login. Is the sign up still valid? Have not receive replies nor confirmation email.

nttstar commented 1 year ago

@jovialio what is your username?

jovialio commented 1 year ago

@jovialio what is your username?

dennis. Thanks it's been approved!

HaoLiuHust commented 1 year ago

减均值除方差的操作是由模型做还是测试时会做？

nttstar commented 1 year ago

如果onnx模型前几层有Sub Div类似的操作会被自动监测到，这样测试代码不会做额外mean std处理。否则会采用默认的mean=127.5, std=127.5。 @HaoLiuHust

hyperf0cus commented 7 months ago

@nttstar Trained with arcface_torch MobileFaceNet MFN model, converted via torch2onnx.py, then converted onnx model weights can't pass onnx_helper.py, and obviously can't pass submission on MFR challenge... evaluation tool MFR failed: [ONNXRuntimeError] : 1 : FAIL : Non-zero status code returned while running MatMul node. Name:'MatMul_95' Status Message: matmul_helper.h:57 Compute MatMul dimension mismatch I didn't change architecture, structure, layers inside backbones mobilefacenet, but error happens...

abdu-khamd commented 7 months ago

@nttstar I am also having problems with uploading my model. It passed onnx_helper.py. However, when submitting to the server I get this error evaluation tool MFR failed: load onnx failed.

nttstar commented 7 months ago

@hyperf0cus @abdu-khamd Make sure the onnxruntime-gpu version in your test env matches our guide.

thNam98 commented 6 months ago

Hi @nttstar , i would like to benchmark my model against the MFR dataset and have registered user. But always received "user unconfirmed after register". Can you check again for me, thanks!

Cevikalp commented 5 months ago

Hello, I would like to benchmark my model against the MFR dataset and have registered via http://iccv21-mfr.com/#/login. Is the sign up still valid? I did not receive any confirmation email.

amoonfana commented 5 months ago

Hi, it seems new registering via http://iccv21-mfr.com/#/login is not available anymore. Is there any way if we want to test our face recognition method on MFR?

nttstar commented 5 months ago

I have approved all new registrations.

Cevikalp commented 5 months ago

Hello, We uploaded our model to the server but it is written that "evaluation tool MIV failed: max feat dim exceed, given 12500". Our method returns 12500 dimensional features and there is no way to decrease it. We submitted a paper to IEEE Transactions on Neural networks and one reviewer insisted on testing our method on this challenge. Could you test our method and share the results with us please? You do not need to report our results on the MFR site. Best regards, Prof. Hakan Cevikalp

On Wed, Mar 27, 2024 at 12:35 PM Jia Guo @.***> wrote:

I have approved all new registrations.

— Reply to this email directly, view it on GitHub https://github.com/deepinsight/insightface/issues/1564#issuecomment-2022313208, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQFHF22EGFQAIJV3ZS5LRB3Y2KHFZAVCNFSM46CTAFN2U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TEMBSGIZTCMZSGA4A . You are receiving this because you commented.Message ID: @.***>

nttstar commented 5 months ago

@Cevikalp Sorry we don't support the model evaluation with such high feature dimension.

Cevikalp commented 5 months ago

Hi Hakan，

Let me check the evaluation server status and get back to you shortly.

Best regards, Jiankang

From: Hakan Cevikalp @.> Sent: 30 March 2024 13:45 To: deepinsight/insightface @.>; @. @.>; Deng, Jiankang @.>; @. @.>; Jiwen Lu @.>; @. @.>; @. @.> Cc: deepinsight/insightface @.>; Comment @.> Subject: Re: [deepinsight/insightface] [MFR21] ICCV21 masked face recognition challenge discussion (#1564)

This email from @.*** originates from outside Imperial. Do not click on links and attachments unless you recognise the sender. If you trust the sender, add them to your safe senders listhttps://spam.ic.ac.uk/SpamConsole/Senders.aspx to disable email stamping for this address.

Hello, We uploaded our model to the server but it is written that "evaluation tool MIV failed: max feat dim exceed, given 12500". Our method returns 12500 dimensional features and there is no way to decrease it. We submitted a paper to IEEE Transactions on Neural networks and one reviewer insisted on testing our method on this challenge. Could you test our method and share the results with us please? You do not need to report our results on the MFR site. Best regards, Prof. Hakan Cevikalp

On Wed, Mar 27, 2024 at 12:35 PM Jia Guo @.**@.>> wrote:

I have approved all new registrations.

— Reply to this email directly, view it on GitHubhttps://github.com/deepinsight/insightface/issues/1564#issuecomment-2022313208, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AQFHF22EGFQAIJV3ZS5LRB3Y2KHFZAVCNFSM46CTAFN2U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TEMBSGIZTCMZSGA4A. You are receiving this because you commented.Message ID: @.***>

Cevikalp commented 5 months ago

Thank you very much.

31 Mar 2024 Pzr 00:55 tarihinde Deng, Jiankang @.***> şunu yazdı:

Hi Hakan，

Let me check the evaluation server status and get back to you shortly.

Best regards, Jiankang

From: Hakan Cevikalp @.> Sent: 30 March 2024 13:45 To: deepinsight/insightface < @.>; @. @.>; Deng, Jiankang < @.>; @. < @.>; Jiwen Lu @.>; @. @.>; @. @.> Cc: deepinsight/insightface @.>; Comment < @.> Subject: Re: [deepinsight/insightface] [MFR21] ICCV21 masked face recognition challenge discussion (#1564)

This email from @.*** originates from outside Imperial. Do not click on links and attachments unless you recognise the sender. If you trust the sender, add them to your safe senders list https://spam.ic.ac.uk/SpamConsole/Senders.aspx to disable email stamping for this address.

Hello, We uploaded our model to the server but it is written that "evaluation tool MIV failed: max feat dim exceed, given 12500". Our method returns 12500 dimensional features and there is no way to decrease it. We submitted a paper to IEEE Transactions on Neural networks and one reviewer insisted on testing our method on this challenge. Could you test our method and share the results with us please? You do not need to report our results on the MFR site. Best regards, Prof. Hakan Cevikalp

On Wed, Mar 27, 2024 at 12:35 PM Jia Guo @.***> wrote:

I have approved all new registrations.

— Reply to this email directly, view it on GitHub https://github.com/deepinsight/insightface/issues/1564#issuecomment-2022313208, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQFHF22EGFQAIJV3ZS5LRB3Y2KHFZAVCNFSM46CTAFN2U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TEMBSGIZTCMZSGA4A . You are receiving this because you commented.Message ID: @.***>