SDXL one of the ten reference images' caption generates a disturbing image

mlcommons / inference

Reference implementations of MLPerf™ inference benchmarks

https://mlcommons.org/en/groups/inference

Apache License 2.0

1.19k stars 519 forks source link

Open nvyihengz opened 2 months ago

nvyihengz commented 2 months ago

Hi, caption id 1303 generates an image that looks a bit disturbing. Since in 4.0, we disclosed the images to the public, e.g. https://github.com/mlcommons/inference_results_v4.0/tree/main/closed/NVIDIA/results/DGX-H100_H100-SXM-80GBx8_TRT/stable-diffusion-xl/Offline/accuracy/images, I would like to ask if we can replace 1303 with a different caption or just avoid showing the images in the results directory? We can remove the images after the review period if the latter is preferred. Thanks!

nvyihengz commented 2 months ago

Also I think the taskforce agreed to not disclose the compliance images to the public.

pgmpablo157321 commented 2 months ago

@pgmpablo157321

Add option in submission checker to remove all the stable diffusion images folder, and skip the checks done for the images folder
Remove the images from the inference_results_v4.0 repository