AUTOMATIC1111 / stable-diffusion-webui

Stable Diffusion web UI
GNU Affero General Public License v3.0
138.31k stars 26.28k forks source link

[Feature]Provide a new page to delete the information of png info #2374

Open huige233 opened 1 year ago

huige233 commented 1 year ago

If I want to erase the contents of Png info from some shared pictures, but do not want to modify the settings in setting so that locally generated pictures are not written to the header of png info, I need this function to erase the ne of png info

kybercore commented 1 year ago

Use ExifCleaner or one of the many other portable open source tools for that. Surely there even is a webapp.

https://github.com/szTheory/exifcleaner

huige233 commented 1 year ago

Why not build in such a function

kybercore commented 1 year ago

Because this is digital forensics, you should rely on a battle-tested project for that. Integrating something like this is only necessary for very few niche scenarios because almost every social media and image hosting site removes Metadata anyways. It would take away ressources from development of more critical features is what I'm saying, which can't be easily replaced with external tools.

huige233 commented 1 year ago

Please tell me what form this pnginfo is stored in. You can't read the content directly with exiread

kybercore commented 1 year ago

Have you tried running it through ExifCleaner? It tells you what properties it removes

huige233 commented 1 year ago

I tried to run exifcleaner and knew that I needed to delete the parameters, but I could not read the content through exifread

huige233 commented 1 year ago

image

hunterfrerich commented 1 year ago

It's a common misconception that the PNG data is "classic metadata" or "exif". It is not, but you can see it (and delete it) by opening the PNG in notepad.. it's stored in the beginning.

Use PIL PngInfo to manipulate the data correctly:

from PIL import Image
from PIL.PngImagePlugin import PngInfo

targetImage = Image.open("pathToImage.png")

png_txt = PngInfo()
png_txt.add_text("MyNewString", "A string")
png_txt.add_text("MyNewInt", str(1234))

targetImage.save("NewPath.png", pnginfo=png_txt)
targetImage = Image.open("NewPath.png")

print(targetImage.text)

>>> {'MyNewString': 'A string', 'MyNewInt': '1234'}
huige233 commented 1 year ago

It's a common misconception that the PNG data is "classic metadata" or "exif". It is not, but you can see it (and delete it) by opening the PNG in notepad.. it's stored in the beginning.

Use PIL PngInfo to manipulate the data correctly:

from PIL import Image
from PIL.PngImagePlugin import PngInfo

targetImage = Image.open("pathToImage.png")

png_txt = PngInfo()
png_txt.add_text("MyNewString", "A string")
png_txt.add_text("MyNewInt", str(1234))

targetImage.save("NewPath.png", pnginfo=png_txt)
targetImage = Image.open("NewPath.png")

print(targetImage.text)

>>> {'MyNewString': 'A string', 'MyNewInt': '1234'}

Thanks for your explanation. Then I just need to extract the data and delete the parameters

hunterfrerich commented 1 year ago

Thanks for your explanation. Then I just need to extract the data and delete the parameters

I just gave generic write/read code. You can overwrite it with blank or nonsense data (as shown), but it's worth testing to be sure.

EliEron commented 1 year ago

It's a common misconception that the PNG data is "classic metadata" or "exif". It is not, but you can see it (and delete it) by opening the PNG in notepad.. it's stored in the beginning.

To be technical, the PNG DATA is stored using text chunks, it is an official PNG feature, though a somewhat obscure one. I had never encountered this way of storing metadata before I saw it used by Automatic.

Since it is pretty obscure there aren't any standardized tools I know of to clean it, and it's not really common for third party sites to sanitize it like they do with Exif. Though a re-encode will erase the metadata, and most large image hosters do re-encode PNGs.

huige233 commented 1 year ago

It's a common misconception that the PNG data is "classic metadata" or "exif". It is not, but you can see it (and delete it) by opening the PNG in notepad.. it's stored in the beginning.

To be technical, the PNG DATA is stored using text chunks, it is an official PNG feature, though a somewhat obscure one. I had never encountered this way of storing metadata before I saw it used by Automatic.

Since it is pretty obscure there aren't any standardized tools I know of to clean it, and it's not really common for third party sites to sanitize it like they do with Exif. Though a re-encode will erase the metadata, and most large image hosters do re-encode PNGs.

In many Chinese articles, they also try to make exif for the data in the png, and have software such as magicexif directly read and modify it. Subjectively, they think that png also uses IIF information such as exif to mark additional n

grayau commented 1 year ago

Why not build in such a function

you can add the feature yourself, it's open source. Such inane features would only hinder more important developments. Also mandatory why are you wanting to hide your prompts.jpg, you should want to share your prompts

cgwers commented 10 months ago

@grayu - thank you stating your feelings on the subject. However, the ask was valid. You might not see the need, but you're not qualified to speak for all users performing AI image generation.

PII issues, possible IP concerns if someone wants to sell their work, whatever. There are many reasons for stripping all non-image data from the files even if they aren't applicable to you.

If someone sees a need for a tool, you should accept that it's a valid request for their situation and simply not comment if you can't help and especially not tell someone to do it themselves. That's a dick move. The vast majority of people playing with AI are no longer Python gurus, they're average computer tinkerers who might be able to write bash/bat for minor tasks and aren't programmers.

cgwers-2 commented 10 months ago

@huige233

This python code strips the metadata from InvokeAI and SD images. It saves two files: One with the extracted text and the other the image file without metadata. The image file has NoDat appended to the filename. It does not modify the original image.

Take care, stay safe.


Example usage: D:\1 tools>"D:\1 tools\stripImgGen.py" "D:\1 tools\00005-3510580121.png" Metadata saved to: D:\1 tools\00005-3510580121_metadata.json Image without metadata saved to: D:\1 tools\00005-3510580121_NoDat.png


import json
import os
import argparse

def extract_metadata_and_save(file_path):
    with open(file_path, 'rb') as f:
        data = f.read()

        # Check for InvokeAI metadata
        if b'tEXtinvokeai_metadata' in data:
            start_index = data.find(b'tEXtinvokeai_metadata') + len('tEXtinvokeai_metadata') + 1
            end_index = data.find(b'\x00', start_index)
            json_data = data[start_index:end_index].decode('utf-8', errors='replace')
            # Remove Yٝ from the end
            json_data = json_data.replace("\u059D\u0002", "")
        # Check for SD Web metadata
        elif b'tEXtparameters' in data:
            start_index = data.find(b'tEXtparameters') + len('tEXtparameters') + 1
            end_index = data.find(b'\x00', start_index)
            json_data = data[start_index:end_index].decode('utf-8', errors='replace')
        else:
            print(f"No metadata found in {file_path}")
            return

        # Remove non-JSON characters from the end
        while json_data and (json_data[-1] not in ['}', ']', '"', '0', '1', '2', '3', '4', '5', '6', '7', '8', '9']):
            json_data = json_data[:-1]

    # Write the metadata to a JSON file
    output_file_path = os.path.splitext(file_path)[0] + "_metadata.json"
    with open(output_file_path, 'w', encoding='utf-8') as json_file:
        json_file.write(json_data)
    print(f"Metadata saved to: {output_file_path}")

    # Save a copy of the image without the metadata
    data_without_metadata = data.replace(data[start_index - len('tEXtparameters') - 1:end_index], b'')
    output_image_path = os.path.splitext(file_path)[0] + "_NoDat.png"
    with open(output_image_path, 'wb') as img_file:
        img_file.write(data_without_metadata)
    print(f"Image without metadata saved to: {output_image_path}")

if __name__ == "__main__":
    parser = argparse.ArgumentParser(description="Extract metadata from PNGs generated by InvokeAI and SD Web.")
    parser.add_argument('files', metavar='FILE', type=str, nargs='+',
                        help="Path to the PNG file(s) to process.")

    args = parser.parse_args()

    for file_path in args.files:
        extract_metadata_and_save(file_path)
NullEqualsZero commented 2 months ago

The code didn't work for me so with the help of chat GPT I made this if anyone has problems with the above solution feel free to use it!

import json
import os
import argparse
from PIL import PngImagePlugin, Image

def extract_metadata_and_save(file_path):
    with Image.open(file_path) as img:
        # Check for InvokeAI metadata
        if "invokeai_metadata" in img.info:
            json_data = img.info["invokeai_metadata"]
        # Check for SD Web metadata
        elif "parameters" in img.info:
            json_data = img.info["parameters"]
        else:
            print(f"No metadata found in {file_path}")
            return

        # Save metadata to a JSON file
        output_file_path = os.path.splitext(file_path)[0] + "_metadata.json"
        with open(output_file_path, "w", encoding="utf-8") as json_file:
            json_file.write(json_data)
        print(f"Metadata saved to: {output_file_path}")

        # Remove metadata and save a copy of the image
        img_no_metadata = img.copy()
        img_no_metadata.info.pop("invokeai_metadata", None)
        img_no_metadata.info.pop("parameters", None)
        output_image_path = os.path.splitext(file_path)[0] + "_NoDat.png"
        img_no_metadata.save(output_image_path)
        print(f"Image without metadata saved to: {output_image_path}")

if __name__ == "__main__":
    parser = argparse.ArgumentParser(
        description="Extract metadata from PNGs generated by InvokeAI and SD Web."
    )
    parser.add_argument(
        "files",
        metavar="FILE",
        type=str,
        nargs="+",
        help="Path to the PNG file(s) to process.",
    )

    args = parser.parse_args()

    for file_path in args.files:
        extract_metadata_and_save(file_path)

I plan to make an extension for it!