search

kakaobrain / coyo-dataset

COYO-700M: Large-scale Image-Text Pair Dataset
https://kakaobrain.com/contents?contentId=7eca73e3-3089-43cb-b701-332e8a1743fd
1.16k stars 38 forks source link

Watermark detection model #2

Open robvanvolt opened 2 years ago

robvanvolt commented 2 years ago

Hey there!

It's Robert from LAION! Congratulations on this really interesting dataset release!

I was just wondering if it was possible for you to release details on your internal watermark detection model, or even the model itself: "watermark_score | float | The watermark probability of the image by our internal model"

We released our model here and would appreciate potential improvements for our model or incorporations of interesting techniques from your side!:)

Best,

Robert

beomheepark commented 2 years ago

Hello Robert!

Thanks for your interest in the COYO dataset. I'm Brook, a member of the COYO team.

First, we needed a model that can detect watermarks not only in the COYO dataset but also in the images generated by our image generation model (to be released soon). We tried to utilize the public model with good performance, but there were hardly any. As far as I know, the model you released was the only one. However, after evaluating internally, we decided that even this model was not suitable for us. (Because, as you know, the criteria for "watermark" are very vague.) Therefore, we needed to train a new model, and the details of the training are summarized below.

The following three types of datasets were used for training and evaluation; 1) Public watermark dataset (you released), 2) dataset collected from our image generation model, 3) dataset composed of watermark images from stock image sites (e.g. shutterstock) and non-watermark images from OpenImages. For the classification model, RegNetY 16GF pre-trained on ImageNet-21K was selected and trained since it was suitable in terms of the trade-off between accuracy and speed. The evaluation was performed on each data set separately, and there was a performance improvement of about 10% for all data sets compared to your published model. (the performance gap is thought to be simply caused by the increase in data and model size.)

It seems that it would be difficult to release the dataset used for training due to the license issue. However, I think that the model and code for evaluation can be sufficiently released, and I would like to discuss it with the team and release it. I am very impressed by the many contents that LAION has released. Thanks again for your hard work and interest. If you have any additional questions or comments, please feel free to reply.

Best, Brook

robvanvolt commented 2 years ago

Thank you for the detailed answer - it would be really amazing if you discussed that matter with your team and came to a positive result and releasing the code and architecture of your watermark detection model!

Looking forward to hearing from you again!

Best,

Robert