Evaluation Metric -- Style Score

[zhaoshengDS]

Hello,

Thanks for sharing this great work with the community.

In the published paper, you mention you guys utilize a metric called Style Score. Since that metric requires a trained classifier model, I am wondering which specific classifier are you guys using? Also, how do you train and test that classifier? Are you going to release the codebase for that classifier in this repo?

Thank you.

[dailenson]

Thanks for your attention. We adopt the writer identifier in DeepImitator[61] as our style classifier. We train and test the classifier in the handwritings belonging to the test writers, with a random split of 80%:20%. More details can be seen in Appendix A.1.1.

[zhaoshengDS]

Thanks for your comprehensive response. I just read through the Appendix section of your published paper. For the Appendix section, I have one more question to ask:

• In Table 5 the style score for the Japanese dataset is 99.64%
• In Table 9 the style score of your model for the Japanese dataset is 41.85%

Why there is a drastic change in terms of the style score metric?

Thank you.

[dailenson]

Thanks. We first clarify the difference between Table 5 and Table 9. Table 5 provides the performance of the writer identifier trained on the real handwritings. After obtaining a good style classifier (i.e., ACC 99.64%) on the Japanese dataset, we use it to evaluate the generation quality of various methods. As shown in Table 9, our SDT achieves a better imitation performance than the second-best (41.85% vs. 32.88% ) regarding the style metric. In the future, we will further improve our SDT in the Japanese dataset.

[zhaoshengDS]

Thank you so much for your timely response. Let's stick to Table 5 in the Appendix section this time. When you are testing the accuracy of the style classifier, the accuracy is beyond 99% for both Chinese and Japanese.

So my question has something to do with the batch size. When you test the style classifier,

• How many writers are included in each batch?
• In each batch, what is the number of samples / characters for each writer?

Thank you.

[Gdnaiteab]

Thank you for your patient answers. May I ask if you are going to release the content recognizer and writer identifier in this repository?

[yuyukuy534]

Hello,I don't understand the codes for these two scores，May I ask if you are going to release the content recognizer and writer identifier in this repository? looking forward to your reply ，thank you

[yuyukuy534]

Hello,

Thanks for sharing this great work with the community.

In the published paper, you mention you guys utilize a metric called Style Score. Since that metric requires a trained classifier model, I am wondering which specific classifier are you guys using? Also, how do you train and test that classifier? Are you going to release the codebase for that classifier in this repo?

Thank you.

Hello Have you found the codebase for that classifier in this repo? I didn't find it, could you please send me the code? Thank you very much!

[yuluo-kuaiyirensheng]

Hello, I'm sorry to bother you. Could you provide the pretrained weights of the model for style score and content score? I find it takes a long time to train this model on my own computer. Thank you very much.

[yuyukuy534]

Hello, I haven't found a way to train the style score and the content score, if you are willing to provide the code to train these two indicators, I am willing to train it and send you the weight of the pre-trained model when it is completed, I have a A100GPU server here, and I will finish training it soon. Looking forward to receiving your reply. Thank you!

------------------ 原始邮件 ------------------ 发件人: "dailenson/SDT" @.>; 发送时间: 2023年11月24日(星期五) 下午5:54 @.>; @.**@.>; 主题: Re: [dailenson/SDT] Evaluation Metric -- Style Score (Issue #6)

Hello, I'm sorry to bother you. Could you provide the pretrained weights of the model for style score and content score? I find it takes a long time to train this model on my own computer. Thank you very much.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.***>

[yuyukuy534]

Hello, I'm sorry to bother you. Could you provide the pretrained weights of the model for style score and content score? I find it takes a long time to train this model on my own computer. Thank you very much.

Hello, I haven't found a way to train the style score and the content score, if you are willing to provide the code to train these two indicators, I am willing to train it and send you the weight of the pre-trained model when it is completed, I have a A100GPU server here, and I will finish training it soon. Looking forward to receiving your reply. Thank you!

[yuluo-kuaiyirensheng]

Hello, May I ask how you obtained the style score and content score in your paper if you didn't have a model for style score and content score? In the source code, I have only found the DTW evaluation metrics. I wish to receive from you soon. Thank you very much!

------------------ 原始邮件 ------------------ 发件人: "dailenson/SDT" @.>; 发送时间: 2023年11月25日(星期六) 上午10:41 @.>; @.**@.>; 主题: Re: [dailenson/SDT] Evaluation Metric -- Style Score (Issue #6)

Hello, I'm sorry to bother you. Could you provide the pretrained weights of the model for style score and content score? I find it takes a long time to train this model on my own computer. Thank you very much.

Hello, I haven't found a way to train the style score and the content score, if you are willing to provide the code to train these two indicators, I am willing to train it and send you the weight of the pre-trained model when it is completed, I have a A100GPU server here, and I will finish training it soon. Looking forward to receiving your reply. Thank you!

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.***>

[yuyukuy534]

Hello, May I ask how you obtained the style score and content score in your paper if you didn't have a model for style score and content score? In the source code, I have only found the DTW evaluation metrics. I wish to receive from you soon. Thank you very much! … ------------------ 原始邮件 ------------------ 发件人: "dailenson/SDT" @.>; 发送时间: 2023年11月25日(星期六) 上午10:41 @.>; @.**@.>; 主题: Re: [dailenson/SDT] Evaluation Metric -- Style Score (Issue #6) Hello, I'm sorry to bother you. Could you provide the pretrained weights of the model for style score and content score? I find it takes a long time to train this model on my own computer. Thank you very much. Hello, I haven't found a way to train the style score and the content score, if you are willing to provide the code to train these two indicators, I am willing to train it and send you the weight of the pre-trained model when it is completed, I have a A100GPU server here, and I will finish training it soon. Looking forward to receiving your reply. Thank you! — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.***>

Hi. Have you found a model for style score and content score, I still haven't found it