Metadata structure improvements

[stassius]

Hello! I'm developing a metadata parsing software and images created with dynamic-prompts extension are hard to parse.

Key problems:

1. No commas before the keywords.
2. Commas inside the templates themselves.

Here is an example: CFG scale minimum: 3.5 Template: __topnotch-gpt3.5__, [ __1980s-technology__|__1990s-technology__|__render-engine__|__render__]

My proposal is:

1. Put commas before "Template" and "Negative template" keywords
2. Enclose the template text into quotes.

This way extension metadata could be parsed along with standard A1111 parameters and data from other extensions.

[adieyal]

I don't understand your request. Are you looking at parsing metadata embedded in the png file? If so, the easiest solution is to re-purpose the same parser that it used to generate prompts in the first place which you can find here: https://github.com/adieyal/dynamicprompts

Writing your own parser will be a lot of work and will almost certainly be brittle.

[stassius]

The request is: use the common A1111 rules when saving the metadata. Parsers split key-value pairs by comma, and split the key from the value by colon. In your case, there are no commas before the keys.

The solution is simple:

1. Put commas before "Template" and "Negative template" keywords. You can leave the carriage return as is, just add commas.
2. Enclose the template texts into quotes as they can contain commas as well

That's all. This way it won't break a metadata parsing process and won't corrupt all the metadata written by other extensions after the dynamic-prompts.

[adieyal]

Here's is an example from a random file I just generated:

A green Neapolitan Mastiff
Negative prompt: text, signature, watermark, username, blurry, artist name
Steps: 20, Sampler: Euler a, CFG scale: 7, Seed: 1645088782, Size: 512x512, Model hash: e6415c4892, Model: realisticVisionV2.0
Template: A {red|green|blue} __animals__
Negative Template: __negatives/marks__

How how your proposal change it? Something like this:

A green Neapolitan Mastiff
Negative prompt: text, signature, watermark, username, blurry, artist name
Steps: 20, Sampler: Euler a, CFG scale: 7, Seed: 1645088782, Size: 512x512, Model hash: e6415c4892, Model: realisticVisionV2.0
,Template: "A {red|green|blue} __animals__"
,Negative Template: "__negatives/marks__"

[stassius]

Yes, that's right. You can move the commas before the "\n" for the esthetic reason.

[akx]

@stassius Your parser could make use of the fact that Template: and Negative Template: are always preceded by a newline character. (Note how the stock A1111 data also has \nNegative prompt: in the same vein.)

This is somewhat related to #343 - AIUI, we could store SDDP metadata in a different field in the PNG file. (I haven't looked further into how the A1111 metadata is structured in the PNG file.)

[stassius]

@akx It's possible that other extensions will write metadata after the Dynamic prompts, and if they will use the common scheme (commas-colons), it would be impossible to tell where the Dynamic prompts part is ending. The standard comma-colon approach will eliminate the hassle here.

[akx]

@stassius Indeed it is. Comments from e.g. hijack hook modules (e.g. Used embeddings: easynegative [119b]) are already double-newline-separated anyway (see https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/22bcc7be428c94e9408f589966c2040187245d81/modules/processing.py#L754).

Additionally: how would your proposed scheme handle quotes within the prompts/templates?

That aside, there's a huge corpus of SDDP-generated metadata already out there with the current format, so your parser should support it anyway...

[akx]

@stassius Your project may be interested to know infotext structure changes re quoting in sd-webui 1.3.0: https://github.com/AUTOMATIC1111/stable-diffusion-webui/commit/ff0e17174f8d93a71fdd5a4a80a4629bbf97f822

[catboxanon]

This is somewhat related to https://github.com/adieyal/sd-dynamic-prompts/issues/343 - AIUI, we could store SDDP metadata in a different field in the PNG file. (I haven't looked further into how the A1111 metadata is structured in the PNG file.)

@akx AFAIK all that would need to be changed would be this section, and then tests adapted accordingly. Since I don't think this extension reads from these I would assume it's not a concern to store them under a different key. https://github.com/adieyal/sd-dynamic-prompts/blob/525ee62a9a1ce308a8ae932883be0f4f264282c4/sd_dynamic_prompts/callbacks.py#L24-L34

Semi-related: shouldn't these not be stored anyway if no wildcards are used? Apologies if this has already been mentioned previously, I can't find anything on it.