Discussion on the call process for training individual portraits Lora in AIGC

[alexsunxl]

Destination

AIOS integrates AI portrait process. Includes personal ID photos, artistic photos, pictures of hairstyle changes, clothing changes, sense changs, etc

Basic process

• Connect to a personal stable diffusion.
• Verify the model and extension of the stable diffusion.
• Encapsulate the return information of different states of stable diffusion, allowing users to perceive the steps.
• AIOS kernel load the local directory, read 5 to 10 individual photos, and initiate training for the Lora model.
• The LORA training process may take 10 to 20 minutes, depending on the GPU.
• Once training is complete, notify the user and await a new input message.
• The llm parses the message to a prompt and sends a request to the AIGC model. The AIGC can then generate ID photos, artistic photos, pictures of hairstyle changes, clothing changes, etc.

Other

More consideration should be given to using LLM to parse requirements and invoke commands, reducing the commands users need to input in the process, so that users only need to input their requirements.

@waterflier @lurenpluto

[waterflier]

Good idea~

Using an Agent to replace complex SD operations is a good idea. I thought about the process, is it like this: users can first pass a template (usually seen on social networks) to the Agent, and then the Agent guides the user to enter enough information to generate prompts through dialogue interaction, and automatically completes the SD model download, generation, and selection based on the existing LoRA model (this may rely on the GPT-V API).

[alexsunxl]

This part of individual portraits is not easy to handle in terms of model detection and initialization. I'm still on hold.

On the other hand, regarding AGI or AIGC， GPT4's API has many new features, including dall-e3, vision, and text to speech. You can consider integrating it first. Is it better to place these function calls in the agent or in the function?

@waterflier

[waterflier]

I am focus on New Agent working cycle right now~ If you could add these new features to the openai_node, I would be very pleased, but make sure to follow the pattern we currently use for enhancing AI capabilities.

You wourld create some new issue about that features~~ Thank you!