Conditional skip

[valconius]

Around line 168 under the file in scripts/, an error is raised if prompt replace did not replace something.

if Script.VALIDATE_REPLACE and match not in p.prompt and match not in p.negative_prompt:
        raise RuntimeError(f"Invalid prompt replace, '{match}' is not in prompt '{p.prompt}' nor negative prompt '{p.negative_prompt}'")

It is convenient, as mentioned in other issues here and there, to have a dreambooth model specify its trained activation string with a prompt replace. If a placeholder string is "style" and the activation string is "drawing" but we have an axis that instead uses "painting" and "watercolor", then presently I have two placeholder strings "style_dreambooth" and "style_other" and perform prompt replace twice in each value of the axis. (This means non-dreambooth's results have a dummy "style_dreambooth" string left over.)

To decrease the number of images generated, an axis could perhaps skip generating some value when its prompt replace fails, assuming the dreambooth's axis already replaced "style_other" with an empty string.

However, more in line with the spirit of this extension, the regular style prompt replace can be left alone. In this case, the style axis should delete the "style_dreambooth" string only if it exists. If the dreambooth model already replaced that string, then the error should not be raised.

So this weird hack could be used to make better grids in the later case, but it would also permit fractionally decreasing the dimensions of a grid by saying, "we have many axes including 3 models and 2 styles, don't make images only for styles when the single dreambooth model is selected".

I don't know how to reference other github issues, but the former case is like having the model axis' dreambooth value set a variable, "don't define a style because I already set it," and then having the prompt replace that occur later on either not generate an image under this condition or else perform an alternative prompt replace under this condition, but now this is starting to resemble things expected of a full programming language and goes beyond the confines of yaml or the prompt replace mini-dsl where you merely separate params with an equals sign.

[mcmonkey4eva]

This is already an option, it's a checkbox in the grid generation WebUI

[valconius]

Ah, you're right. And here I was in the middle of modifying the stuff I randomly scratched-together earlier today! Thanks for pointing that out, though the option is named rather strangely. However, I was thinking about my problem just now, and I realized I really wanted a more conditional promptreplace failure. Let's say we had…

grid.params.prompt: cat style
axes:
  model.values:
    m:
      model: m
    db:
      model: m_dreambooth
      promptreplace: style = activation_string
  styles.values:
    dog without style:
      promptreplace: cat = dog
      prompt replace: style =
    s0.promptreplace: style = watercolor
    s1.promptreplace: style = graphite
    without style: style =

…, then we'll get 2×4 pictures by permitting promptreplace to be invalid, …

• m: "dog" & "cat watercolor" & "cat graphite" & "cat"
• db: "dog activation_string" & "cat activation_string" & "cat activation_string" & "cat activation_string" …, where the same picture is made twice. This is a silly example where the failure does not matter much, but:
  1. It can be confusing to see three identical pictures in a grid. It might not be obvious to the viewer that only one of those images is relevant;
  2. In a more practical grid of mine, this results in most images being redundant.

So, would you say adding a simple domain-specific language almost identical to promptreplace's would be not too crazy? Or, see s1 in the following for something simpler without a DSL and with different logic overall. For example, I would like to rewrite the above like so:

grid.params.prompt: cat style
axes:
  model.values:
    m:
      model: m
    db:
      model: db
      promptreplace: style = activation_string
  styles.values:
    dog without style:
      promptreplace: cat = dog
      prompt replace: style =
    s0.promptreplace: skip this image generation when model is 'm_dreambooth', style = watercolor
    s1:
      skip this image generation when model is: m_dreambooth
      promptreplace: style = graphite
    without style:
      promptreplace: style = 

The resulting grid would then have holes in it like so:

• m: "dog" & "cat watercolor" & "cat graphite" & "cat"
• db: "dog activation_string" & None & None & "cat activation_string"

Alternatively, more involved logic could somehow be integrated and extra holes could be made:

• m: "dog" & "cat watercolor" & "cat graphite" & None
• db: "dog activation_string" & None & None & "cat activation_string" (Since dog's style and m_dreambooth's style are basically cross-referencing each other, even more involved logic would be needed to turn "dog activation string" into simply "dog", but such a situation is rare.)

EDIT Ah, something else occurred to me, I couldn't set an axis to be "model" and another to be "embed" when mixing SD1.x models with SD2.x models. A way around this is to define an axis "model + embed", but simply defining an axis' value to be one promptreplace for one model and a different promptreplace for a different model would be ideal.

[mcmonkey4eva]

... man, AI software dev really is pushing speed boundaries. Normal software dev usually gets at least a few years of feature creep before something turns into a DSL.

[mcmonkey4eva]

Also delayed response to your edit, but you can do something like this:

grid:
    title: for issue 18
    author: a
    description: a
    format: jpg
    params:
        prompt: a cat (embed_sd1) (embed_sd2)
axes:
    model:
        title: model
        values:
            a:
                title: 1.5
                params:
                    model: sd-v1-5
                    prompt replace: (embed_sd2)=
            b:
                title: 2.1
                params:
                    model: sd-v2-1_768
                    promptreplace: (embed_sd1)=
    embed:
        title: embed
        values:
            a:
                title: embed
                params:
                    promptreplace: (embed_sd1)=wolf
                    prompt replace: (embed_sd2)=dog

ie, have multiple slots in the base prompt, and have model params remove the incompatible slots, and then embed replaces slots that haven't been removed with real values.

Not the same as your full request but does achieve the need in your edit at the bottom about embeds with the current system.

[Mistborn-First-Era]

I wasn't able to solve my problem with embeds since I am not using different models to change to a 'base prompt'. I even tried to use the unused replacement prompts with a strength of zero like '(animal:0)' but it led to artifacts in longer prompts. I also tried to comment out one prompt and its associated replacements, generate a grid. Then comment out the other prompt and its associations and continue generating on the same grid, which didn't work. Really wish prompt replace would just skip the generation if no replacement was found. Allowing sub axes for replacements within prompt axes would probably work too. +1 this feature.

axes:
  model:
    title: Models
    values:
      Lyriel_v15:
        title: Lyriel_v15
        params:
          model: lyriel_v15
  prompt:
    title: Prompts
    values:
      Animals in styles in landscapes:
        title: Animals
        params:
          prompt: style, animal, natural_landscape
      Buildings in styles:
        title: Buildings
        params:
          prompt: style, buildings_and_interiors
  prompt replacement 1:
    title: Style Replacements
    values:
      Cubism: PromptReplace= style = Cubism, Geometric shapes, Multiple perspectives, Fragmentation, Analytical cubism, Collage elements
      PopArt: PromptReplace= style = Pop Art, Popular culture references, Consumerism critique, Bold and vibrant colors, Comic book style
  prompt replacement 2:
    title: Animal Replacements
    values:
      Dog: PromptReplace= animal = Dog, Wagging tail, Expressive eyes, Playful demeanor, Fluffy coats, Varied colors, Floppy/perky ears
      Cat: PromptReplace= animal = Cat, Graceful, Sleek, Mesmerizing eyes, Various fur textures, Diverse colors/patterns, Agile bodies
  prompt replacement 3:
    title: Natural Landscapes
    values:
      Jungle: PromptReplace= natural_landscape = Jungle, Lush vegetation, Canopy layers, Vibrant colors, Dappled sunlight
      Above Water:  PromptReplace= natural_landscape = Above water, Expansive horizon, Reflections, Serene atmosphere, Sunlit ripples
  prompt replacement 4:
    title: Building Interiors
    values:
      Skyscrapers: PromptReplace= buildings_and_interiors = Skyscrapers, Towering heights, Modern architecture, Glass facades, Urban landmarks
      Villa: PromptReplace= buildings_and_interiors = Villa, Elegant and spacious, Luxury residence, Beautiful gardens, Mediterranean charm