Open huige233 opened 1 year ago
Use ExifCleaner or one of the many other portable open source tools for that. Surely there even is a webapp.
Why not build in such a function
Because this is digital forensics, you should rely on a battle-tested project for that. Integrating something like this is only necessary for very few niche scenarios because almost every social media and image hosting site removes Metadata anyways. It would take away ressources from development of more critical features is what I'm saying, which can't be easily replaced with external tools.
Please tell me what form this pnginfo is stored in. You can't read the content directly with exiread
Have you tried running it through ExifCleaner? It tells you what properties it removes
I tried to run exifcleaner and knew that I needed to delete the parameters, but I could not read the content through exifread
It's a common misconception that the PNG data is "classic metadata" or "exif". It is not, but you can see it (and delete it) by opening the PNG in notepad.. it's stored in the beginning.
Use PIL PngInfo to manipulate the data correctly:
from PIL import Image
from PIL.PngImagePlugin import PngInfo
targetImage = Image.open("pathToImage.png")
png_txt = PngInfo()
png_txt.add_text("MyNewString", "A string")
png_txt.add_text("MyNewInt", str(1234))
targetImage.save("NewPath.png", pnginfo=png_txt)
targetImage = Image.open("NewPath.png")
print(targetImage.text)
>>> {'MyNewString': 'A string', 'MyNewInt': '1234'}
It's a common misconception that the PNG data is "classic metadata" or "exif". It is not, but you can see it (and delete it) by opening the PNG in notepad.. it's stored in the beginning.
Use PIL PngInfo to manipulate the data correctly:
from PIL import Image from PIL.PngImagePlugin import PngInfo targetImage = Image.open("pathToImage.png") png_txt = PngInfo() png_txt.add_text("MyNewString", "A string") png_txt.add_text("MyNewInt", str(1234)) targetImage.save("NewPath.png", pnginfo=png_txt) targetImage = Image.open("NewPath.png") print(targetImage.text) >>> {'MyNewString': 'A string', 'MyNewInt': '1234'}
Thanks for your explanation. Then I just need to extract the data and delete the parameters
Thanks for your explanation. Then I just need to extract the data and delete the parameters
I just gave generic write/read code. You can overwrite it with blank or nonsense data (as shown), but it's worth testing to be sure.
It's a common misconception that the PNG data is "classic metadata" or "exif". It is not, but you can see it (and delete it) by opening the PNG in notepad.. it's stored in the beginning.
To be technical, the PNG DATA is stored using text chunks, it is an official PNG feature, though a somewhat obscure one. I had never encountered this way of storing metadata before I saw it used by Automatic.
Since it is pretty obscure there aren't any standardized tools I know of to clean it, and it's not really common for third party sites to sanitize it like they do with Exif. Though a re-encode will erase the metadata, and most large image hosters do re-encode PNGs.
It's a common misconception that the PNG data is "classic metadata" or "exif". It is not, but you can see it (and delete it) by opening the PNG in notepad.. it's stored in the beginning.
To be technical, the PNG DATA is stored using text chunks, it is an official PNG feature, though a somewhat obscure one. I had never encountered this way of storing metadata before I saw it used by Automatic.
Since it is pretty obscure there aren't any standardized tools I know of to clean it, and it's not really common for third party sites to sanitize it like they do with Exif. Though a re-encode will erase the metadata, and most large image hosters do re-encode PNGs.
In many Chinese articles, they also try to make exif for the data in the png, and have software such as magicexif directly read and modify it. Subjectively, they think that png also uses IIF information such as exif to mark additional n
Why not build in such a function
you can add the feature yourself, it's open source. Such inane features would only hinder more important developments. Also mandatory why are you wanting to hide your prompts.jpg, you should want to share your prompts
@grayu - thank you stating your feelings on the subject. However, the ask was valid. You might not see the need, but you're not qualified to speak for all users performing AI image generation.
PII issues, possible IP concerns if someone wants to sell their work, whatever. There are many reasons for stripping all non-image data from the files even if they aren't applicable to you.
If someone sees a need for a tool, you should accept that it's a valid request for their situation and simply not comment if you can't help and especially not tell someone to do it themselves. That's a dick move. The vast majority of people playing with AI are no longer Python gurus, they're average computer tinkerers who might be able to write bash/bat for minor tasks and aren't programmers.
@huige233
This python code strips the metadata from InvokeAI and SD images. It saves two files: One with the extracted text and the other the image file without metadata. The image file has NoDat appended to the filename. It does not modify the original image.
Take care, stay safe.
Example usage: D:\1 tools>"D:\1 tools\stripImgGen.py" "D:\1 tools\00005-3510580121.png" Metadata saved to: D:\1 tools\00005-3510580121_metadata.json Image without metadata saved to: D:\1 tools\00005-3510580121_NoDat.png
import json
import os
import argparse
def extract_metadata_and_save(file_path):
with open(file_path, 'rb') as f:
data = f.read()
# Check for InvokeAI metadata
if b'tEXtinvokeai_metadata' in data:
start_index = data.find(b'tEXtinvokeai_metadata') + len('tEXtinvokeai_metadata') + 1
end_index = data.find(b'\x00', start_index)
json_data = data[start_index:end_index].decode('utf-8', errors='replace')
# Remove Yٝ from the end
json_data = json_data.replace("\u059D\u0002", "")
# Check for SD Web metadata
elif b'tEXtparameters' in data:
start_index = data.find(b'tEXtparameters') + len('tEXtparameters') + 1
end_index = data.find(b'\x00', start_index)
json_data = data[start_index:end_index].decode('utf-8', errors='replace')
else:
print(f"No metadata found in {file_path}")
return
# Remove non-JSON characters from the end
while json_data and (json_data[-1] not in ['}', ']', '"', '0', '1', '2', '3', '4', '5', '6', '7', '8', '9']):
json_data = json_data[:-1]
# Write the metadata to a JSON file
output_file_path = os.path.splitext(file_path)[0] + "_metadata.json"
with open(output_file_path, 'w', encoding='utf-8') as json_file:
json_file.write(json_data)
print(f"Metadata saved to: {output_file_path}")
# Save a copy of the image without the metadata
data_without_metadata = data.replace(data[start_index - len('tEXtparameters') - 1:end_index], b'')
output_image_path = os.path.splitext(file_path)[0] + "_NoDat.png"
with open(output_image_path, 'wb') as img_file:
img_file.write(data_without_metadata)
print(f"Image without metadata saved to: {output_image_path}")
if __name__ == "__main__":
parser = argparse.ArgumentParser(description="Extract metadata from PNGs generated by InvokeAI and SD Web.")
parser.add_argument('files', metavar='FILE', type=str, nargs='+',
help="Path to the PNG file(s) to process.")
args = parser.parse_args()
for file_path in args.files:
extract_metadata_and_save(file_path)
The code didn't work for me so with the help of chat GPT I made this if anyone has problems with the above solution feel free to use it!
import json
import os
import argparse
from PIL import PngImagePlugin, Image
def extract_metadata_and_save(file_path):
with Image.open(file_path) as img:
# Check for InvokeAI metadata
if "invokeai_metadata" in img.info:
json_data = img.info["invokeai_metadata"]
# Check for SD Web metadata
elif "parameters" in img.info:
json_data = img.info["parameters"]
else:
print(f"No metadata found in {file_path}")
return
# Save metadata to a JSON file
output_file_path = os.path.splitext(file_path)[0] + "_metadata.json"
with open(output_file_path, "w", encoding="utf-8") as json_file:
json_file.write(json_data)
print(f"Metadata saved to: {output_file_path}")
# Remove metadata and save a copy of the image
img_no_metadata = img.copy()
img_no_metadata.info.pop("invokeai_metadata", None)
img_no_metadata.info.pop("parameters", None)
output_image_path = os.path.splitext(file_path)[0] + "_NoDat.png"
img_no_metadata.save(output_image_path)
print(f"Image without metadata saved to: {output_image_path}")
if __name__ == "__main__":
parser = argparse.ArgumentParser(
description="Extract metadata from PNGs generated by InvokeAI and SD Web."
)
parser.add_argument(
"files",
metavar="FILE",
type=str,
nargs="+",
help="Path to the PNG file(s) to process.",
)
args = parser.parse_args()
for file_path in args.files:
extract_metadata_and_save(file_path)
I plan to make an extension for it!
If I want to erase the contents of Png info from some shared pictures, but do not want to modify the settings in setting so that locally generated pictures are not written to the header of png info, I need this function to erase the ne of png info