I'm using MAC M1 chip, and a python3.8.10 venv (created by python3.8 -m venv venv_py3810 instead of conda).
When using CPU (python visual_chatgpt.py --load ImageCaptioning_cpu,Text2Image_cpu) it works fine
Then I tried to use MPS by python visual_chatgpt.py --load "Text2Box_mps:0,Segmenting_mps:0, Inpainting_mps:0,ImageCaptioning_mps:0, Text2Image_mps:1,Image2Canny_cpu,CannyText2Image_mps:1, Image2Depth_cpu,DepthText2Image_mps:1,VisualQuestionAnswering_mps:2, InstructPix2Pix_mps:2,Image2Scribble_cpu,ScribbleText2Image_mps:2, SegText2Image_mps:2,Image2Pose_cpu,PoseText2Image_mps:2, Image2Hed_cpu,HedText2Image_mps:3,Image2Normal_cpu, NormalText2Image_mps:3,Image2Line_cpu,LineText2Image_mps:3" and it starts to give nonsense captions.
e.g.:
Output Text: a a a a a a a a a a a a a a a a a a a
Output Text: four four four four four four four four four four four four four four four four four four four
Any ideas why this might be the case?
Many thanks
Terminal Output Details
> (venv_py3810) TaskMatrix % python visual_chatgpt.py --load "Text2Box_mps:0,Segmenting_mps:0,
> Inpainting_mps:0,ImageCaptioning_mps:0,
> Text2Image_mps:1,Image2Canny_cpu,CannyText2Image_mps:1,
> Image2Depth_cpu,DepthText2Image_mps:1,VisualQuestionAnswering_mps:2,
> InstructPix2Pix_mps:2,Image2Scribble_cpu,ScribbleText2Image_mps:2,
> SegText2Image_mps:2,Image2Pose_cpu,PoseText2Image_mps:2,
> Image2Hed_cpu,HedText2Image_mps:3,Image2Normal_cpu,
> NormalText2Image_mps:3,Image2Line_cpu,LineText2Image_mps:3"
> /path_to_dir/TaskMatrix/venv_py3810/lib/python3.8/site-packages/groundingdino/models/GroundingDINO/ms_deform_attn.py:31: UserWarning: Failed to load custom C++ ops. Running on CPU mode Only!
> warnings.warn("Failed to load custom C++ ops. Running on CPU mode Only!")
> Initializing VisualChatGPT, load_dict={'Text2Box': 'mps:0', 'Segmenting': 'mps:0', 'Inpainting': 'mps:0', 'ImageCaptioning': 'mps:0', 'Text2Image': 'mps:1', 'Image2Canny': 'cpu', 'CannyText2Image': 'mps:1', 'Image2Depth': 'cpu', 'DepthText2Image': 'mps:1', 'VisualQuestionAnswering': 'mps:2', 'InstructPix2Pix': 'mps:2', 'Image2Scribble': 'cpu', 'ScribbleText2Image': 'mps:2', 'SegText2Image': 'mps:2', 'Image2Pose': 'cpu', 'PoseText2Image': 'mps:2', 'Image2Hed': 'cpu', 'HedText2Image': 'mps:3', 'Image2Normal': 'cpu', 'NormalText2Image': 'mps:3', 'Image2Line': 'cpu', 'LineText2Image': 'mps:3'}
> Loading Text2Box ( ) to mps:0
> Initializing ObjectDetection to mps:0
> /path_to_dir/TaskMatrix/venv_py3810/lib/python3.8/site-packages/torch/functional.py:504: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at /Users/runner/work/pytorch/pytorch/pytorch/aten/src/ATen/native/TensorShape.cpp:3191.)
> return _VF.meshgrid(tensors, **kwargs) # type: ignore[attr-defined]
> final text_encoder_type: bert-base-uncased
> Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertModel: ['cls.seq_relationship.weight', 'cls.seq_relationship.bias', 'cls.predictions.transform.dense.bias', 'cls.predictions.bias', 'cls.predictions.transform.LayerNorm.bias', 'cls.predictions.transform.dense.weight', 'cls.predictions.transform.LayerNorm.weight']
> \- This IS expected if you are initializing BertModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
> \- This IS NOT expected if you are initializing BertModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
> _IncompatibleKeys(missing_keys=[], unexpected_keys=['label_enc.weight'])
> Loading Segmenting ( ) to mps:0
> Inintializing Segmentation to mps:0
> Loading Inpainting ( ) to mps:0
> text_encoder/model.safetensors not found
> /path_to_dir/TaskMatrix/venv_py3810/lib/python3.8/site-packages/transformers/models/clip/feature_extraction_clip.py:28: FutureWarning: The class CLIPFeatureExtractor is deprecated and will be removed in version 5 of Transformers. Please use CLIPImageProcessor instead.
> warnings.warn(
> `text_config_dict` is provided which will be used to initialize `CLIPTextConfig`. The value `text_config["id2label"]` will be overriden.
> Loading ImageCaptioning ( ) to mps:0
> Initializing ImageCaptioning to mps:0
> Loading Text2Image ( ) to mps:1
> Initializing Text2Image to mps:1
> `text_config_dict` is provided which will be used to initialize `CLIPTextConfig`. The value `text_config["id2label"]` will be overriden.
> Loading Image2Canny ( ) to cpu
> Initializing Image2Canny
> Loading CannyText2Image ( ) to mps:1
> Initializing CannyText2Image to mps:1
> You have disabled the safety checker for by passing `safety_checker=None`. Ensure that you abide to the conditions of the Stable Diffusion license and do not expose unfiltered results in services or applications open to the public. Both the diffusers team and Hugging Face strongly recommend to keep the safety filter enabled in all public facing circumstances, disabling it only for use-cases that involve analyzing network behavior or auditing its results. For more information, please have a look at https://github.com/huggingface/diffusers/pull/254 .
> Loading Image2Depth ( ) to cpu
> Initializing Image2Depth
> No model was supplied, defaulted to Intel/dpt-large and revision e93beec (https://huggingface.co/Intel/dpt-large).
> Using a pipeline without specifying a model name and revision in production is not recommended.
> Some weights of DPTForDepthEstimation were not initialized from the model checkpoint at Intel/dpt-large and are newly initialized: ['neck.fusion_stage.layers.0.residual_layer1.convolution2.weight', 'neck.fusion_stage.layers.0.residual_layer1.convolution2.bias', 'neck.fusion_stage.layers.0.residual_layer1.convolution1.bias', 'neck.fusion_stage.layers.0.residual_layer1.convolution1.weight']
> You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
> Could not find image processor class in the image processor config or the model config. Loading based on pattern matching with the model's feature extractor configuration.
> Loading DepthText2Image ( ) to mps:1
> Initializing DepthText2Image to mps:1
> You have disabled the safety checker for by passing `safety_checker=None`. Ensure that you abide to the conditions of the Stable Diffusion license and do not expose unfiltered results in services or applications open to the public. Both the diffusers team and Hugging Face strongly recommend to keep the safety filter enabled in all public facing circumstances, disabling it only for use-cases that involve analyzing network behavior or auditing its results. For more information, please have a look at https://github.com/huggingface/diffusers/pull/254 .
> Loading VisualQuestionAnswering ( ) to mps:2
> Initializing VisualQuestionAnswering to mps:2
> Loading InstructPix2Pix ( ) to mps:2
> Initializing InstructPix2Pix to mps:2
> Loading Image2Scribble ( ) to cpu
> Initializing Image2Scribble
> Loading ScribbleText2Image ( ) to mps:2
> Initializing ScribbleText2Image to mps:2
> You have disabled the safety checker for by passing `safety_checker=None`. Ensure that you abide to the conditions of the Stable Diffusion license and do not expose unfiltered results in services or applications open to the public. Both the diffusers team and Hugging Face strongly recommend to keep the safety filter enabled in all public facing circumstances, disabling it only for use-cases that involve analyzing network behavior or auditing its results. For more information, please have a look at https://github.com/huggingface/diffusers/pull/254 .
> Loading SegText2Image ( ) to mps:2
> Initializing SegText2Image to mps:2
> You have disabled the safety checker for by passing `safety_checker=None`. Ensure that you abide to the conditions of the Stable Diffusion license and do not expose unfiltered results in services or applications open to the public. Both the diffusers team and Hugging Face strongly recommend to keep the safety filter enabled in all public facing circumstances, disabling it only for use-cases that involve analyzing network behavior or auditing its results. For more information, please have a look at https://github.com/huggingface/diffusers/pull/254 .
> Loading Image2Pose ( ) to cpu
> Initializing Image2Pose
> Loading PoseText2Image ( ) to mps:2
> Initializing PoseText2Image to mps:2
> You have disabled the safety checker for by passing `safety_checker=None`. Ensure that you abide to the conditions of the Stable Diffusion license and do not expose unfiltered results in services or applications open to the public. Both the diffusers team and Hugging Face strongly recommend to keep the safety filter enabled in all public facing circumstances, disabling it only for use-cases that involve analyzing network behavior or auditing its results. For more information, please have a look at https://github.com/huggingface/diffusers/pull/254 .
> Loading Image2Hed ( ) to cpu
> Initializing Image2Hed
> Loading HedText2Image ( ) to mps:3
> Initializing HedText2Image to mps:3
> You have disabled the safety checker for by passing `safety_checker=None`. Ensure that you abide to the conditions of the Stable Diffusion license and do not expose unfiltered results in services or applications open to the public. Both the diffusers team and Hugging Face strongly recommend to keep the safety filter enabled in all public facing circumstances, disabling it only for use-cases that involve analyzing network behavior or auditing its results. For more information, please have a look at https://github.com/huggingface/diffusers/pull/254 .
> Loading Image2Normal ( ) to cpu
> Initializing Image2Normal
> Loading NormalText2Image ( ) to mps:3
> Initializing NormalText2Image to mps:3
> You have disabled the safety checker for by passing `safety_checker=None`. Ensure that you abide to the conditions of the Stable Diffusion license and do not expose unfiltered results in services or applications open to the public. Both the diffusers team and Hugging Face strongly recommend to keep the safety filter enabled in all public facing circumstances, disabling it only for use-cases that involve analyzing network behavior or auditing its results. For more information, please have a look at https://github.com/huggingface/diffusers/pull/254 .
> Loading Image2Line ( ) to cpu
> Initializing Image2Line
> Loading LineText2Image ( ) to mps:3
> Initializing LineText2Image to mps:3
> You have disabled the safety checker for by passing `safety_checker=None`. Ensure that you abide to the conditions of the Stable Diffusion license and do not expose unfiltered results in services or applications open to the public. Both the diffusers team and Hugging Face strongly recommend to keep the safety filter enabled in all public facing circumstances, disabling it only for use-cases that involve analyzing network behavior or auditing its results. For more information, please have a look at https://github.com/huggingface/diffusers/pull/254 .
> Initializing ImageEditing
> All the Available Functions: {'Text2Box': <__main__.Text2Box object at 0x2967d28b0>, 'Segmenting': <__main__.Segmenting object at 0x2967d2820>, 'Inpainting': <__main__.Inpainting object at 0x2967d28e0>, 'ImageCaptioning': <__main__.ImageCaptioning object at 0x299effa90>, 'Text2Image': <__main__.Text2Image object at 0x299a00b50>, 'Image2Canny': <__main__.Image2Canny object at 0x16e49dfd0>, 'CannyText2Image': <__main__.CannyText2Image object at 0x16e49dee0>, 'Image2Depth': <__main__.Image2Depth object at 0x299a00ee0>, 'DepthText2Image': <__main__.DepthText2Image object at 0x2f3c2ab20>, 'VisualQuestionAnswering': <__main__.VisualQuestionAnswering object at 0x2c2931e80>, 'InstructPix2Pix': <__main__.InstructPix2Pix object at 0x2c33edf10>, 'Image2Scribble': <__main__.Image2Scribble object at 0x2c67e4d60>, 'ScribbleText2Image': <__main__.ScribbleText2Image object at 0x2c33cd0a0>, 'SegText2Image': <__main__.SegText2Image object at 0x2c33dd460>, 'Image2Pose': <__main__.Image2Pose object at 0x2c33cd0d0>, 'PoseText2Image': <__main__.PoseText2Image object at 0x5e7f16f70>, 'Image2Hed': <__main__.Image2Hed object at 0x4af128430>, 'HedText2Image': <__main__.HedText2Image object at 0x767db6f70>, 'Image2Normal': <__main__.Image2Normal object at 0x4a0cb0fa0>, 'NormalText2Image': <__main__.NormalText2Image object at 0x76e9dacd0>, 'Image2Line': <__main__.Image2Line object at 0x910c438b0>, 'LineText2Image': <__main__.LineText2Image object at 0x93f4dbf40>, 'InfinityOutPainting': <__main__.InfinityOutPainting object at 0x910ca7760>, 'ObjectSegmenting': <__main__.ObjectSegmenting object at 0xad59c8d00>, 'ImageEditing': <__main__.ImageEditing object at 0xad59c8c10>}
> Running on local URL: http://0.0.0.0:7860
>
> To create a public link, set `share=True` in `launch()`.
>
> \> Entering new AgentExecutor chain...
> Action: Get Photo Description
> Action Input: image/94b2a3ff.pngopening image/94b2a3ff.png
> /path_to_dir/TaskMatrix/venv_py3810/lib/python3.8/site-packages/transformers/generation/utils.py:1313: UserWarning: Using `max_length`'s default (20) to control the generation length. This behaviour is deprecated and will be removed from the config in v5 of Transformers -- we recommend using `max_new_tokens` to control the maximum length of the generation.
> warnings.warn(
>
> Processed ImageCaptioning, Input Image: image/94b2a3ff.png, Output Text: a a a a a a a a a a a a a a a a a a a
>
> Observation: a a a a a a a a a a a a a a a a a a a
> Thought:AI: The image you provided is a a a a a a a a a a a a a a a a a a a.
>
> \> Finished chain.
>
> Processed run_text, Input text: image/94b2a3ff.png
> Current state: [('image/94b2a3ff.png', 'The image you provided is a a a a a a a a a a a a a a a a a a a.')]
> Current Memory:
> Human: image/94b2a3ff.png
> AI: The image you provided is a a a a a a a a a a a a a a a a a a a.
> history_memory:
> Human: image/94b2a3ff.png
> AI: The image you provided is a a a a a a a a a a a a a a a a a a a., n_tokens: 27
>
>
>
> \> Entering new AgentExecutor chain...
> Action: Get Photo Description
> Action Input: /Users/Downloads/1girl_solo_blonde_hair_long_hair_dress_jewelry_necklace.pngopening /Users/Downloads/1girl_solo_blonde_hair_long_hair_dress_jewelry_necklace.png
>
> Processed ImageCaptioning, Input Image: /Users/Downloads/1girl_solo_blonde_hair_long_hair_dress_jewelry_necklace.png, Output Text: four four four four four four four four four four four four four four four four four four four
>
> Observation: four four four four four four four four four four four four four four four four four four four
> Thought:AI: The image you provided is of a woman with long blonde hair wearing a dress and jewelry.
>
> \> Finished chain.
>
> Processed run_text, Input text: /Users/Downloads/1girl_solo_blonde_hair_long_hair_dress_jewelry_necklace.png
> Current state: [('image/94b2a3ff.png', 'The image you provided is a a a a a a a a a a a a a a a a a a a.'), ('/Users/Downloads/1girl_solo_blonde_hair_long_hair_dress_jewelry_necklace.png', 'The image you provided is of a woman with long blonde hair wearing a dress and jewelry.')]
> Current Memory:
> Human: image/94b2a3ff.png
> AI: The image you provided is a a a a a a a a a a a a a a a a a a a.
> Human: /Users/Downloads/1girl_solo_blonde_hair_long_hair_dress_jewelry_necklace.png
> AI: The image you provided is of a woman with long blonde hair wearing a dress and jewelry.
> history_memory:
> Human: image/94b2a3ff.png
> AI: The image you provided is a a a a a a a a a a a a a a a a a a a.
> Human: /Users/Downloads/1girl_solo_blonde_hair_long_hair_dress_jewelry_necklace.png
> AI: The image you provided is of a woman with long blonde hair wearing a dress and jewelry., n_tokens: 47
>
>
> \> Entering new AgentExecutor chain...
> Action: Get Photo Description
> Action Input: /Users/Downloads/data/girl/02.jpgopening /Users/Downloads/data/girl/02.jpg
>
> Processed ImageCaptioning, Input Image: /Users/Downloads/data/girl/02.jpg, Output Text: a a a a a a a a a a a a a a a a a a a
>
> Observation: a a a a a a a a a a a a a a a a a a a
> Thought:AI: The image you provided is of a woman with long brown hair wearing a white dress and jewelry.
>
> \> Finished chain.
huluobohua
commented
1 year ago
I'm using MAC M1 chip, and a python3.8.10 venv (created by
python3.8 -m venv venv_py3810instead of conda).When using CPU (
python visual_chatgpt.py --load ImageCaptioning_cpu,Text2Image_cpu) it works fineThen I tried to use MPS by
python visual_chatgpt.py --load "Text2Box_mps:0,Segmenting_mps:0, Inpainting_mps:0,ImageCaptioning_mps:0, Text2Image_mps:1,Image2Canny_cpu,CannyText2Image_mps:1, Image2Depth_cpu,DepthText2Image_mps:1,VisualQuestionAnswering_mps:2, InstructPix2Pix_mps:2,Image2Scribble_cpu,ScribbleText2Image_mps:2, SegText2Image_mps:2,Image2Pose_cpu,PoseText2Image_mps:2, Image2Hed_cpu,HedText2Image_mps:3,Image2Normal_cpu, NormalText2Image_mps:3,Image2Line_cpu,LineText2Image_mps:3"and it starts to give nonsense captions. e.g.:Any ideas why this might be the case?
Many thanks
Terminal Output Details
> (venv_py3810) TaskMatrix % python visual_chatgpt.py --load "Text2Box_mps:0,Segmenting_mps:0, > Inpainting_mps:0,ImageCaptioning_mps:0, > Text2Image_mps:1,Image2Canny_cpu,CannyText2Image_mps:1, > Image2Depth_cpu,DepthText2Image_mps:1,VisualQuestionAnswering_mps:2, > InstructPix2Pix_mps:2,Image2Scribble_cpu,ScribbleText2Image_mps:2, > SegText2Image_mps:2,Image2Pose_cpu,PoseText2Image_mps:2, > Image2Hed_cpu,HedText2Image_mps:3,Image2Normal_cpu, > NormalText2Image_mps:3,Image2Line_cpu,LineText2Image_mps:3" > /path_to_dir/TaskMatrix/venv_py3810/lib/python3.8/site-packages/groundingdino/models/GroundingDINO/ms_deform_attn.py:31: UserWarning: Failed to load custom C++ ops. Running on CPU mode Only! > warnings.warn("Failed to load custom C++ ops. Running on CPU mode Only!") > Initializing VisualChatGPT, load_dict={'Text2Box': 'mps:0', 'Segmenting': 'mps:0', 'Inpainting': 'mps:0', 'ImageCaptioning': 'mps:0', 'Text2Image': 'mps:1', 'Image2Canny': 'cpu', 'CannyText2Image': 'mps:1', 'Image2Depth': 'cpu', 'DepthText2Image': 'mps:1', 'VisualQuestionAnswering': 'mps:2', 'InstructPix2Pix': 'mps:2', 'Image2Scribble': 'cpu', 'ScribbleText2Image': 'mps:2', 'SegText2Image': 'mps:2', 'Image2Pose': 'cpu', 'PoseText2Image': 'mps:2', 'Image2Hed': 'cpu', 'HedText2Image': 'mps:3', 'Image2Normal': 'cpu', 'NormalText2Image': 'mps:3', 'Image2Line': 'cpu', 'LineText2Image': 'mps:3'} > Loading Text2Box () to mps:0
> Initializing ObjectDetection to mps:0
> /path_to_dir/TaskMatrix/venv_py3810/lib/python3.8/site-packages/torch/functional.py:504: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at /Users/runner/work/pytorch/pytorch/pytorch/aten/src/ATen/native/TensorShape.cpp:3191.)
> return _VF.meshgrid(tensors, **kwargs) # type: ignore[attr-defined]
> final text_encoder_type: bert-base-uncased
> Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertModel: ['cls.seq_relationship.weight', 'cls.seq_relationship.bias', 'cls.predictions.transform.dense.bias', 'cls.predictions.bias', 'cls.predictions.transform.LayerNorm.bias', 'cls.predictions.transform.dense.weight', 'cls.predictions.transform.LayerNorm.weight']
> \- This IS expected if you are initializing BertModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
> \- This IS NOT expected if you are initializing BertModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
> _IncompatibleKeys(missing_keys=[], unexpected_keys=['label_enc.weight'])
> Loading Segmenting ( ) to mps:0
> Inintializing Segmentation to mps:0
> Loading Inpainting ( ) to mps:0
> text_encoder/model.safetensors not found
> /path_to_dir/TaskMatrix/venv_py3810/lib/python3.8/site-packages/transformers/models/clip/feature_extraction_clip.py:28: FutureWarning: The class CLIPFeatureExtractor is deprecated and will be removed in version 5 of Transformers. Please use CLIPImageProcessor instead.
> warnings.warn(
> `text_config_dict` is provided which will be used to initialize `CLIPTextConfig`. The value `text_config["id2label"]` will be overriden.
> Loading ImageCaptioning ( ) to mps:0
> Initializing ImageCaptioning to mps:0
> Loading Text2Image ( ) to mps:1
> Initializing Text2Image to mps:1
> `text_config_dict` is provided which will be used to initialize `CLIPTextConfig`. The value `text_config["id2label"]` will be overriden.
> Loading Image2Canny ( ) to cpu
> Initializing Image2Canny
> Loading CannyText2Image ( ) to mps:1
> Initializing CannyText2Image to mps:1
> You have disabled the safety checker for by passing `safety_checker=None`. Ensure that you abide to the conditions of the Stable Diffusion license and do not expose unfiltered results in services or applications open to the public. Both the diffusers team and Hugging Face strongly recommend to keep the safety filter enabled in all public facing circumstances, disabling it only for use-cases that involve analyzing network behavior or auditing its results. For more information, please have a look at https://github.com/huggingface/diffusers/pull/254 .
> Loading Image2Depth ( ) to cpu
> Initializing Image2Depth
> No model was supplied, defaulted to Intel/dpt-large and revision e93beec (https://huggingface.co/Intel/dpt-large).
> Using a pipeline without specifying a model name and revision in production is not recommended.
> Some weights of DPTForDepthEstimation were not initialized from the model checkpoint at Intel/dpt-large and are newly initialized: ['neck.fusion_stage.layers.0.residual_layer1.convolution2.weight', 'neck.fusion_stage.layers.0.residual_layer1.convolution2.bias', 'neck.fusion_stage.layers.0.residual_layer1.convolution1.bias', 'neck.fusion_stage.layers.0.residual_layer1.convolution1.weight']
> You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
> Could not find image processor class in the image processor config or the model config. Loading based on pattern matching with the model's feature extractor configuration.
> Loading DepthText2Image ( ) to mps:1
> Initializing DepthText2Image to mps:1
> You have disabled the safety checker for by passing `safety_checker=None`. Ensure that you abide to the conditions of the Stable Diffusion license and do not expose unfiltered results in services or applications open to the public. Both the diffusers team and Hugging Face strongly recommend to keep the safety filter enabled in all public facing circumstances, disabling it only for use-cases that involve analyzing network behavior or auditing its results. For more information, please have a look at https://github.com/huggingface/diffusers/pull/254 .
> Loading VisualQuestionAnswering ( ) to mps:2
> Initializing VisualQuestionAnswering to mps:2
> Loading InstructPix2Pix ( ) to mps:2
> Initializing InstructPix2Pix to mps:2
> Loading Image2Scribble ( ) to cpu
> Initializing Image2Scribble
> Loading ScribbleText2Image ( ) to mps:2
> Initializing ScribbleText2Image to mps:2
> You have disabled the safety checker for by passing `safety_checker=None`. Ensure that you abide to the conditions of the Stable Diffusion license and do not expose unfiltered results in services or applications open to the public. Both the diffusers team and Hugging Face strongly recommend to keep the safety filter enabled in all public facing circumstances, disabling it only for use-cases that involve analyzing network behavior or auditing its results. For more information, please have a look at https://github.com/huggingface/diffusers/pull/254 .
> Loading SegText2Image ( ) to mps:2
> Initializing SegText2Image to mps:2
> You have disabled the safety checker for by passing `safety_checker=None`. Ensure that you abide to the conditions of the Stable Diffusion license and do not expose unfiltered results in services or applications open to the public. Both the diffusers team and Hugging Face strongly recommend to keep the safety filter enabled in all public facing circumstances, disabling it only for use-cases that involve analyzing network behavior or auditing its results. For more information, please have a look at https://github.com/huggingface/diffusers/pull/254 .
> Loading Image2Pose ( ) to cpu
> Initializing Image2Pose
> Loading PoseText2Image ( ) to mps:2
> Initializing PoseText2Image to mps:2
> You have disabled the safety checker for by passing `safety_checker=None`. Ensure that you abide to the conditions of the Stable Diffusion license and do not expose unfiltered results in services or applications open to the public. Both the diffusers team and Hugging Face strongly recommend to keep the safety filter enabled in all public facing circumstances, disabling it only for use-cases that involve analyzing network behavior or auditing its results. For more information, please have a look at https://github.com/huggingface/diffusers/pull/254 .
> Loading Image2Hed ( ) to cpu
> Initializing Image2Hed
> Loading HedText2Image ( ) to mps:3
> Initializing HedText2Image to mps:3
> You have disabled the safety checker for by passing `safety_checker=None`. Ensure that you abide to the conditions of the Stable Diffusion license and do not expose unfiltered results in services or applications open to the public. Both the diffusers team and Hugging Face strongly recommend to keep the safety filter enabled in all public facing circumstances, disabling it only for use-cases that involve analyzing network behavior or auditing its results. For more information, please have a look at https://github.com/huggingface/diffusers/pull/254 .
> Loading Image2Normal ( ) to cpu
> Initializing Image2Normal
> Loading NormalText2Image ( ) to mps:3
> Initializing NormalText2Image to mps:3
> You have disabled the safety checker for by passing `safety_checker=None`. Ensure that you abide to the conditions of the Stable Diffusion license and do not expose unfiltered results in services or applications open to the public. Both the diffusers team and Hugging Face strongly recommend to keep the safety filter enabled in all public facing circumstances, disabling it only for use-cases that involve analyzing network behavior or auditing its results. For more information, please have a look at https://github.com/huggingface/diffusers/pull/254 .
> Loading Image2Line ( ) to cpu
> Initializing Image2Line
> Loading LineText2Image ( ) to mps:3
> Initializing LineText2Image to mps:3
> You have disabled the safety checker for by passing `safety_checker=None`. Ensure that you abide to the conditions of the Stable Diffusion license and do not expose unfiltered results in services or applications open to the public. Both the diffusers team and Hugging Face strongly recommend to keep the safety filter enabled in all public facing circumstances, disabling it only for use-cases that involve analyzing network behavior or auditing its results. For more information, please have a look at https://github.com/huggingface/diffusers/pull/254 .
> Initializing ImageEditing
> All the Available Functions: {'Text2Box': <__main__.Text2Box object at 0x2967d28b0>, 'Segmenting': <__main__.Segmenting object at 0x2967d2820>, 'Inpainting': <__main__.Inpainting object at 0x2967d28e0>, 'ImageCaptioning': <__main__.ImageCaptioning object at 0x299effa90>, 'Text2Image': <__main__.Text2Image object at 0x299a00b50>, 'Image2Canny': <__main__.Image2Canny object at 0x16e49dfd0>, 'CannyText2Image': <__main__.CannyText2Image object at 0x16e49dee0>, 'Image2Depth': <__main__.Image2Depth object at 0x299a00ee0>, 'DepthText2Image': <__main__.DepthText2Image object at 0x2f3c2ab20>, 'VisualQuestionAnswering': <__main__.VisualQuestionAnswering object at 0x2c2931e80>, 'InstructPix2Pix': <__main__.InstructPix2Pix object at 0x2c33edf10>, 'Image2Scribble': <__main__.Image2Scribble object at 0x2c67e4d60>, 'ScribbleText2Image': <__main__.ScribbleText2Image object at 0x2c33cd0a0>, 'SegText2Image': <__main__.SegText2Image object at 0x2c33dd460>, 'Image2Pose': <__main__.Image2Pose object at 0x2c33cd0d0>, 'PoseText2Image': <__main__.PoseText2Image object at 0x5e7f16f70>, 'Image2Hed': <__main__.Image2Hed object at 0x4af128430>, 'HedText2Image': <__main__.HedText2Image object at 0x767db6f70>, 'Image2Normal': <__main__.Image2Normal object at 0x4a0cb0fa0>, 'NormalText2Image': <__main__.NormalText2Image object at 0x76e9dacd0>, 'Image2Line': <__main__.Image2Line object at 0x910c438b0>, 'LineText2Image': <__main__.LineText2Image object at 0x93f4dbf40>, 'InfinityOutPainting': <__main__.InfinityOutPainting object at 0x910ca7760>, 'ObjectSegmenting': <__main__.ObjectSegmenting object at 0xad59c8d00>, 'ImageEditing': <__main__.ImageEditing object at 0xad59c8c10>}
> Running on local URL: http://0.0.0.0:7860
>
> To create a public link, set `share=True` in `launch()`.
>
> \> Entering new AgentExecutor chain...
> Action: Get Photo Description
> Action Input: image/94b2a3ff.pngopening image/94b2a3ff.png
> /path_to_dir/TaskMatrix/venv_py3810/lib/python3.8/site-packages/transformers/generation/utils.py:1313: UserWarning: Using `max_length`'s default (20) to control the generation length. This behaviour is deprecated and will be removed from the config in v5 of Transformers -- we recommend using `max_new_tokens` to control the maximum length of the generation.
> warnings.warn(
>
> Processed ImageCaptioning, Input Image: image/94b2a3ff.png, Output Text: a a a a a a a a a a a a a a a a a a a
>
> Observation: a a a a a a a a a a a a a a a a a a a
> Thought:AI: The image you provided is a a a a a a a a a a a a a a a a a a a.
>
> \> Finished chain.
>
> Processed run_text, Input text: image/94b2a3ff.png
> Current state: [('image/94b2a3ff.png', 'The image you provided is a a a a a a a a a a a a a a a a a a a.')]
> Current Memory:
> Human: image/94b2a3ff.png
> AI: The image you provided is a a a a a a a a a a a a a a a a a a a.
> history_memory:
> Human: image/94b2a3ff.png
> AI: The image you provided is a a a a a a a a a a a a a a a a a a a., n_tokens: 27
>
>
>
> \> Entering new AgentExecutor chain...
> Action: Get Photo Description
> Action Input: /Users/Downloads/1girl_solo_blonde_hair_long_hair_dress_jewelry_necklace.pngopening /Users/Downloads/1girl_solo_blonde_hair_long_hair_dress_jewelry_necklace.png
>
> Processed ImageCaptioning, Input Image: /Users/Downloads/1girl_solo_blonde_hair_long_hair_dress_jewelry_necklace.png, Output Text: four four four four four four four four four four four four four four four four four four four
>
> Observation: four four four four four four four four four four four four four four four four four four four
> Thought:AI: The image you provided is of a woman with long blonde hair wearing a dress and jewelry.
>
> \> Finished chain.
>
> Processed run_text, Input text: /Users/Downloads/1girl_solo_blonde_hair_long_hair_dress_jewelry_necklace.png
> Current state: [('image/94b2a3ff.png', 'The image you provided is a a a a a a a a a a a a a a a a a a a.'), ('/Users/Downloads/1girl_solo_blonde_hair_long_hair_dress_jewelry_necklace.png', 'The image you provided is of a woman with long blonde hair wearing a dress and jewelry.')]
> Current Memory:
> Human: image/94b2a3ff.png
> AI: The image you provided is a a a a a a a a a a a a a a a a a a a.
> Human: /Users/Downloads/1girl_solo_blonde_hair_long_hair_dress_jewelry_necklace.png
> AI: The image you provided is of a woman with long blonde hair wearing a dress and jewelry.
> history_memory:
> Human: image/94b2a3ff.png
> AI: The image you provided is a a a a a a a a a a a a a a a a a a a.
> Human: /Users/Downloads/1girl_solo_blonde_hair_long_hair_dress_jewelry_necklace.png
> AI: The image you provided is of a woman with long blonde hair wearing a dress and jewelry., n_tokens: 47
>
>
> \> Entering new AgentExecutor chain...
> Action: Get Photo Description
> Action Input: /Users/Downloads/data/girl/02.jpgopening /Users/Downloads/data/girl/02.jpg
>
> Processed ImageCaptioning, Input Image: /Users/Downloads/data/girl/02.jpg, Output Text: a a a a a a a a a a a a a a a a a a a
>
> Observation: a a a a a a a a a a a a a a a a a a a
> Thought:AI: The image you provided is of a woman with long brown hair wearing a white dress and jewelry.
>
> \> Finished chain.