search

mudler / LocalAI

:robot: The free, Open Source alternative to OpenAI, Claude and others. Self-hosted and local-first. Drop-in replacement for OpenAI, running on consumer-grade hardware. No GPU required. Runs gguf, transformers, diffusers and many more models architectures. Features: Generate Text, Audio, Video, Images, Voice Cloning, Distributed, P2P inference
https://localai.io
MIT License
26.44k stars 1.98k forks source link

txt2vid requires opencv + memory attn. #2107

Open jtwolfe opened 7 months ago

jtwolfe commented 7 months ago

LocalAI version: v2.12.4

Environment, CPU architecture, OS, and Version: Container, AMD 1600X (aka x64) host on Ubuntu LTS 22.04.4

Describe the bug When attempting to execute video generation as documented there appears an error indicating that the opencv-python package needs to be available for the diffusers environment, there are also some warnings about memory attention not configured but this is just a warning that i think would be good to get rid of

To Reproduce I rebuild the v2.12.4-ffmpeg-hipblas image for my RadVII with the following model configuration

name: txt2vid
parameters:
  model: damo-vilab/text-to-video-ms-1.7b
backend: diffusers
step: 25
# Force CPU usage
f16: true
cuda: true
diffusers:
  pipeline_type: VideoDiffusionPipeline
  cuda: true

I then execute the following 

(echo -n '{"prompt": "spiderman surfing","size": "512x512","model":"txt2vid"}') |
curl -H "Content-Type: application/json" -X POST -d @- https://OHNONONO/v1/images/generations

and i get the error

Expected behavior Model is loaded with model, GPU is loaded with work, failure occurs and an empty file is produced in the /tmp/generated/images directory

Logs

6:09AM DBG Request received: {"model":"txt2vid","language":"","n":0,"top_p":null,"top_k":null,"temperature":null,"max_tokens":null,"echo":false,"batch":0,"ignore_eos":false,"repeat_penalty":0,"n_keep":0,"frequency_penalty":0,"presence_penalty":0,"tfz":null,"typical_p":null,"seed":null,"negative_prompt":"","rope_freq_base":0,"rope_freq_scale":0,"negative_prompt_scale":0,"use_fast_tokenizer":false,"clip_skip":0,"tokenizer":"","file":"","response_format":{},"size":"512x512","prompt":"spiderman surfing","instruction":"","input":null,"stop":null,"messages":null,"functions":null,"function_call":null,"stream":false,"mode":0,"step":0,"grammar":"","grammar_json_functions":null,"backend":"","model_base_name":""}
6:09AM DBG Loading model: txt2vid
6:09AM DBG Parameter Config: &{PredictionOptions:{Model:damo-vilab/text-to-video-ms-1.7b Language: N:0 TopP:0xc0001f5478 TopK:0xc0001f5480 Temperature:0xc0001f5488 Maxtokens:0xc0001f5490 Echo:false Batch:0 IgnoreEOS:false RepeatPenalty:0 Keep:0 FrequencyPenalty:0 PresencePenalty:0 TFZ:0xc0001f54b8 TypicalP:0xc0001f54b0 Seed:0xc0001f54d0 NegativePrompt: RopeFreqBase:0 RopeFreqScale:0 NegativePromptScale:0 UseFastTokenizer:false ClipSkip:0 Tokenizer:} Name:txt2vid F16:0xc0001f5448 Threads:0xc0001f5468 Debug:0xc00080ddb8 Roles:map[] Embeddings:false Backend:diffusers TemplateConfig:{Chat: ChatMessage: Completion: Edit: Functions:} PromptStrings:[spiderman surfing] InputStrings:[] InputToken:[] functionCallString: functionCallNameString: FunctionsConfig:{DisableNoAction:false NoActionFunctionName: NoActionDescriptionName: ParallelCalls:false} FeatureFlag:map[] LLMConfig:{SystemPrompt: TensorSplit: MainGPU: RMSNormEps:0 NGQA:0 PromptCachePath: PromptCacheAll:false PromptCacheRO:false MirostatETA:0xc0001f54a8 MirostatTAU:0xc0001f54a0 Mirostat:0xc0001f5498 NGPULayers:0xc0001f54c0 MMap:0xc0001f54c8 MMlock:0xc0001f54c9 LowVRAM:0xc0001f54c9 Grammar: StopWords:[] Cutstrings:[] TrimSpace:[] TrimSuffix:[] ContextSize:0xc0001f5460 NUMA:false LoraAdapter: LoraBase: LoraScale:0 NoMulMatQ:false DraftModel: NDraft:0 Quantization: GPUMemoryUtilization:0 TrustRemoteCode:false EnforceEager:false SwapSpace:0 MaxModelLen:0 MMProj: RopeScaling: ModelType: YarnExtFactor:0 YarnAttnFactor:0 YarnBetaFast:0 YarnBetaSlow:0} AutoGPTQ:{ModelBaseName: Device: Triton:false UseFastTokenizer:false} Diffusers:{CUDA:true PipelineType:VideoDiffusionPipeline SchedulerType: EnableParameters: CFGScale:0 IMG2IMG:false ClipSkip:0 ClipModel: ClipSubFolder: ControlNet:} Step:25 GRPC:{Attempts:0 AttemptsSleepTime:0} VallE:{AudioPath:} CUDA:true DownloadFiles:[] Description: Usage:}
2024-04-23T16:09:46.613312062+10:00 6:09AM INF Loading model 'damo-vilab/text-to-video-ms-1.7b' with backend
 diffusers
6:09AM DBG Loading model in memory from file: /models/damo-vilab/text-to-video-ms-1.7b
2024-04-23T16:09:46.613359401+10:00 6:09AM DBG Loading Model damo-vilab/text-to-video-ms-1.7b with gRPC (file: /models/damo-vilab/text-to-video-ms-1.7b) (backend: diffusers): {backendString:diffusers model:damo-vilab/text-to-video-ms-1.7b threads:8 assetDir:/tmp/localai/backend_data context:{emptyCtx:{}} gRPCOptions:0xc0001f7400 externalBackends:map[autogptq:/build/backend/python/autogptq/run.sh bark:/build/backend/python/bark/run.sh coqui:/build/backend/python/coqui/run.sh diffusers:/build/backend/python/diffusers/run.sh exllama:/build/backend/python/exllama/run.sh exllama2:/build/backend/python/exllama2/run.sh huggingface-embeddings:/build/backend/python/sentencetransformers/run.sh mamba:/build/backend/python/mamba/run.sh petals:/build/backend/python/petals/run.sh sentencetransformers:/build/backend/python/sentencetransformers/run.sh transformers:/build/backend/python/transformers/run.sh transformers-musicgen:/build/backend/python/transformers-musicgen/run.sh vall-e-x:/build/backend/python/vall-e-x/run.sh vllm:/build/backend/python/vllm/run.sh] grpcAttempts:20 grpcAttemptsDelay:2 singleActiveBackend:false parallelRequests:false}
6:09AM DBG Loading external backend: /build/backend/python/diffusers/run.sh
6:09AM DBG Loading GRPC Process: /build/backend/python/diffusers/run.sh
6:09AM DBG GRPC Service for damo-vilab/text-to-video-ms-1.7b will be running at: '127.0.0.1:43347'
2024-04-23T16:09:46.618991978+10:00 6:09AM DBG GRPC Service state dir: /tmp/go-processmanager2915698736
6:09AM DBG GRPC Service Started
6:09AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:43347): stderr /opt/conda/envs/diffusers/lib/python3.11/site-packages/transformers/utils/hub.py:124: FutureWarning: Using `TRANSFORMERS_CACHE` is deprecated and will be removed in v5 of Transformers. Use `HF_HOME` instead.
2024-04-23T16:09:48.598796321+10:00 6:09AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:43347): stderr   warnings.warn(
2024-04-23T16:09:49.914693119+10:00 6:09AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:43347): stderr /opt/conda/envs/diffusers/lib/python3.11/site-packages/diffusers/utils/outputs.py:63: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
6:09AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:43347): stderr   torch.utils._pytree._register_pytree_node(
2024-04-23T16:09:50.016093518+10:00 6:09AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:43347): stderr Server started. Listening on: 127.0.0.1:43347
2024-04-23T16:09:50.623880269+10:00 6:09AM DBG GRPC Service Ready
2024-04-23T16:09:50.623979678+10:00 6:09AM DBG GRPC: Loading model with options: {state:{NoUnkeyedLiterals:{} DoNotCompare:[] DoNotCopy:[] atomicMessageInfo:<nil>} sizeCache:0 unknownFields:[] Model:damo-vilab/text-to-video-ms-1.7b ContextSize:4096 Seed:1060164781 NBatch:512 F16Memory:true MLock:false MMap:true VocabOnly:false LowVRAM:false Embeddings:false NUMA:false NGPULayers:99999999 MainGPU: TensorSplit: Threads:8 LibrarySearchPath: RopeFreqBase:0 RopeFreqScale:0 RMSNormEps:0 NGQA:0 ModelFile:/models/damo-vilab/text-to-video-ms-1.7b Device: UseTriton:false ModelBaseName: UseFastTokenizer:false PipelineType:VideoDiffusionPipeline SchedulerType: CUDA:true CFGScale:0 IMG2IMG:false CLIPModel: CLIPSubfolder: CLIPSkip:0 ControlNet: Tokenizer: LoraBase: LoraAdapter: LoraScale:0 NoMulMatQ:false DraftModel: AudioPath: Quantization: GPUMemoryUtilization:0 TrustRemoteCode:false EnforceEager:false SwapSpace:0 MaxModelLen:0 MMProj: RopeScaling: YarnExtFactor:0 YarnAttnFactor:0 YarnBetaFast:0 YarnBetaSlow:0 Type:}
6:09AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:43347): stderr Loading model damo-vilab/text-to-video-ms-1.7b...
6:09AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:43347): stderr Request Model: "damo-vilab/text-to-video-ms-1.7b"
2024-04-23T16:09:50.625426524+10:00 6:09AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:43347): stderr ContextSize: 4096
6:09AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:43347): stderr Seed: 1060164781
2024-04-23T16:09:50.625527503+10:00 6:09AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:43347): stderr NBatch: 512
2024-04-23T16:09:50.625554593+10:00 6:09AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:43347): stderr F16Memory: true
2024-04-23T16:09:50.625618932+10:00 6:09AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:43347): stderr MMap: true
6:09AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:43347): stderr NGPULayers: 99999999
2024-04-23T16:09:50.625681221+10:00 6:09AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:43347): stderr Threads: 8
6:09AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:43347): stderr ModelFile: "/models/damo-vilab/text-to-video-ms-1.7b"
2024-04-23T16:09:50.625855500+10:00 6:09AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:43347): stderr PipelineType: "VideoDiffusionPipeline"
6:09AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:43347): stderr CUDA: true
2024-04-23T16:09:50.625963519+10:00 6:09AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:43347): stderr 
6:09AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:43347): stderr 
Loading pipeline components...:   0%|          | 0/5 [00:00<?, ?it/s]
Loading pipeline components...:  60%|██████    | 3/5 [00:00<00:00,  6.41it/s]
Loading pipeline components...:  80%|████████  | 4/5 [00:01<00:00,  2.17it/s]
Loading pipeline components...: 100%|██████████| 5/5 [00:01<00:00,  2.87it/s]
Loading pipeline components...: 100%|██████████| 5/5 [00:01<00:00,  2.99it/s]
2024-04-23T16:10:00.559009373+10:00 6:10AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:43347): stderr 
  0%|          | 0/25 [00:00<?, ?it/s]/opt/conda/envs/diffusers/lib/python3.11/site-packages/diffusers/models/attention_processor.py:1231: UserWarning: 1Torch was not compiled with memory efficient attention. (Triggered internally at ../aten/src/ATen/native/transformers/hip/sdp_utils.cpp:505.)
6:10AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:43347): stderr   hidden_states = F.scaled_dot_product_attention(
6:11AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:43347): stderr 
  4%|▍         | 1/25 [00:30<12:07, 30.33s/it]
  8%|▊         | 2/25 [00:33<05:31, 14.41s/it]
 12%|█▏        | 3/25 [00:36<03:24,  9.32s/it]
 16%|█▌        | 4/25 [00:40<02:25,  6.93s/it]
 20%|██        | 5/25 [00:43<01:52,  5.60s/it]
 24%|██▍       | 6/25 [00:46<01:31,  4.81s/it]
 28%|██▊       | 7/25 [00:49<01:17,  4.30s/it]
 32%|███▏      | 8/25 [00:53<01:07,  3.97s/it]
 36%|███▌      | 9/25 [00:56<01:00,  3.75s/it]
 40%|████      | 10/25 [00:59<00:54,  3.60s/it]
 44%|████▍     | 11/25 [01:02<00:49,  3.50s/it]
 48%|████▊     | 12/25 [01:06<00:44,  3.43s/it]
 52%|█████▏    | 13/25 [01:09<00:40,  3.38s/it]
 56%|█████▌    | 14/25 [01:12<00:36,  3.35s/it]
 60%|██████    | 15/25 [01:16<00:33,  3.32s/it]
 64%|██████▍   | 16/25 [01:19<00:29,  3.31s/it]
 68%|██████▊   | 17/25 [01:22<00:26,  3.30s/it]
 72%|███████▏  | 18/25 [01:25<00:23,  3.29s/it]
 76%|███████▌  | 19/25 [01:29<00:19,  3.28s/it]
 80%|████████  | 20/25 [01:32<00:16,  3.28s/it]
 84%|████████▍ | 21/25 [01:35<00:13,  3.28s/it]
 88%|████████▊ | 22/25 [01:38<00:09,  3.28s/it]
 92%|█████████▏| 23/25 [01:42<00:06,  3.27s/it]
 96%|█████████▌| 24/25 [01:45<00:03,  3.27s/it]
100%|██████████| 25/25 [01:48<00:00,  3.27s/it]
100%|██████████| 25/25 [01:48<00:00,  4.35s/it]
6:12AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:43347): stderr Traceback (most recent call last):
2024-04-23T16:12:32.676102521+10:00 6:12AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:43347): stderr   File "/opt/conda/envs/diffusers/lib/python3.11/site-packages/grpc/_server.py", line 552, in _call_behavior
2024-04-23T16:12:32.676123291+10:00 6:12AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:43347): stderr     response_or_iterator = behavior(argument, context)
6:12AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:43347): stderr                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-04-23T16:12:32.676227430+10:00 6:12AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:43347): stderr   File "/build/backend/python/diffusers/backend_diffusers.py", line 396, in GenerateImage
6:12AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:43347): stderr     export_to_video(video_frames, request.dst)
2024-04-23T16:12:32.676304489+10:00 6:12AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:43347): stderr   File "/opt/conda/envs/diffusers/lib/python3.11/site-packages/diffusers/utils/export_utils.py", line 124, in export_to_video
[10.42.182.221]:47204 500 - POST /v1/images/generations
6:12AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:43347): stderr     raise ImportError(BACKENDS_MAPPING["opencv"][1].format("export_to_video"))
6:12AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:43347): stderr ImportError: 
6:12AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:43347): stderr export_to_video requires the OpenCV library but it was not found in your environment. You can install it with pip: `pip
6:12AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:43347): stderr install opencv-python`
6:12AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:43347): stderr

Additional context

localai-bot commented 7 months ago

It seems that the OpenCV library is missing in your environment. The error message suggests installing it with pip using the command: pip install opencv-python. Once you have installed OpenCV, you should be able to run the text-to-video model without encountering the "export_to_video requires the OpenCV library but it was not found" error. If the issue continues after installing OpenCV, you may want to verify that the installation was successful or check if there are any conflicts with other installed packages.

jtwolfe commented 7 months ago

no way thanks bot that was super helpful \ sarcasm

jtwolfe commented 7 months ago

I'm prepping a fix for this here https://github.com/jtwolfe/LocalAI/tree/incl-ocv-pkg-for-diffsusers-utils I'm unsure about version compatibility and while i would like to run a test build im not super familiar with GHActions So we'll see how much I can do to sort this out. :|

jtwolfe commented 7 months ago 
Collecting opencv-python
  Obtaining dependency information for opencv-python from https://files.pythonhosted.org/packages/d9/64/7fdfb9386511cd6805451e012c537073a79a958a58795c4e602e538c388c/opencv_python-4.9.0.80-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
  Downloading opencv_python-4.9.0.80-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (20 kB)
Requirement already satisfied: numpy>=1.21.2 in /opt/conda/envs/diffusers/lib/python3.11/site-packages (from opencv-python) (1.26.0)
Downloading opencv_python-4.9.0.80-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (62.2 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 62.2/62.2 MB 10.1 MB/s eta 0:00:00
Installing collected packages: opencv-python
Successfully installed opencv-python-4.9.0.80

The in-container install into the (diffusers) env

jtwolfe commented 7 months ago

ok so i had an issue where even after installing opencv-python into the diffusers backend it was not working, this was due to installing the package AFTER the backend had been loaded, reset the container and tried

$ /opt/conda/bin/conda init
$ bash
(base) $ conda activate diffusers
(diffusers) $ pip install opencv-python

now on attempting the txt2vid example there is no opencv error but....

2:19AM DBG Loading GRPC Process: /build/backend/python/diffusers/run.sh
2:19AM DBG GRPC Service for damo-vilab/text-to-video-ms-1.7b will be running at: '127.0.0.1:37463'
2024-04-24T12:19:17.829356664+10:00 2:19AM DBG GRPC Service state dir: /tmp/go-processmanager3289495431
2:19AM DBG GRPC Service Started
2:19AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr /opt/conda/envs/diffusers/lib/python3.11/site-packages/transformers/utils/hub.py:124: FutureWarning: Using `TRANSFORMERS_CACHE` is deprecated and will be removed in v5 of Transformers. Use `HF_HOME` instead.
2:19AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr   warnings.warn(
2024-04-24T12:19:20.980289979+10:00 2:19AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr /opt/conda/envs/diffusers/lib/python3.11/site-packages/diffusers/utils/outputs.py:63: UserWarning: torch.utils._pytree._register_pytree_node is deprecated. Please use torch.utils._pytree.register_pytree_node instead.
2:19AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr   torch.utils._pytree._register_pytree_node(
2024-04-24T12:19:21.066571107+10:00 2:19AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr Server started. Listening on: 127.0.0.1:37463
2024-04-24T12:19:21.834329072+10:00 2:19AM DBG GRPC Service Ready
2:19AM DBG GRPC: Loading model with options: {state:{NoUnkeyedLiterals:{} DoNotCompare:[] DoNotCopy:[] atomicMessageInfo:<nil>} sizeCache:0 unknownFields:[] Model:damo-vilab/text-to-video-ms-1.7b ContextSize:4096 Seed:565091323 NBatch:512 F16Memory:true MLock:false MMap:true VocabOnly:false LowVRAM:false Embeddings:false NUMA:false NGPULayers:99999999 MainGPU: TensorSplit: Threads:8 LibrarySearchPath: RopeFreqBase:0 RopeFreqScale:0 RMSNormEps:0 NGQA:0 ModelFile:/models/damo-vilab/text-to-video-ms-1.7b Device: UseTriton:false ModelBaseName: UseFastTokenizer:false PipelineType:VideoDiffusionPipeline SchedulerType: CUDA:true CFGScale:0 IMG2IMG:false CLIPModel: CLIPSubfolder: CLIPSkip:0 ControlNet: Tokenizer: LoraBase: LoraAdapter: LoraScale:0 NoMulMatQ:false DraftModel: AudioPath: Quantization: GPUMemoryUtilization:0 TrustRemoteCode:false EnforceEager:false SwapSpace:0 MaxModelLen:0 MMProj: RopeScaling: YarnExtFactor:0 YarnAttnFactor:0 YarnBetaFast:0 YarnBetaSlow:0 Type:}
2:19AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr Loading model damo-vilab/text-to-video-ms-1.7b...
2024-04-24T12:19:21.835596421+10:00 2:19AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr Request Model: "damo-vilab/text-to-video-ms-1.7b"
2:19AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr ContextSize: 4096
2024-04-24T12:19:21.835621131+10:00 2:19AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr Seed: 565091323
2:19AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr NBatch: 512
2024-04-24T12:19:21.835641811+10:00 2:19AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr F16Memory: true
2:19AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr MMap: true
2024-04-24T12:19:21.835695890+10:00 2:19AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr NGPULayers: 99999999
2:19AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr Threads: 8
2024-04-24T12:19:21.835786349+10:00 2:19AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr ModelFile: "/models/damo-vilab/text-to-video-ms-1.7b"
2:19AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr PipelineType: "VideoDiffusionPipeline"
2:19AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr CUDA: true
2:19AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr 
2024-04-24T12:19:24.850735141+10:00 2:19AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr 
Loading pipeline components...:   0%|          | 0/5 [00:00<?, ?it/s]
Loading pipeline components...:  40%|████      | 2/5 [00:01<00:01,  1.62it/s]
Loading pipeline components...:  60%|██████    | 3/5 [00:01<00:00,  2.47it/s]
Loading pipeline components...:  80%|████████  | 4/5 [00:01<00:00,  2.57it/s]
Loading pipeline components...: 100%|██████████| 5/5 [00:01<00:00,  2.93it/s]
2:19AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr 
  0%|          | 0/25 [00:00<?, ?it/s]/opt/conda/envs/diffusers/lib/python3.11/site-packages/diffusers/models/attention_processor.py:1231: UserWarning: 1Torch was not compiled with memory efficient attention. (Triggered internally at ../aten/src/ATen/native/transformers/hip/sdp_utils.cpp:505.)
2:19AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr   hidden_states = F.scaled_dot_product_attention(
2:21AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr 
  4%|▍         | 1/25 [00:30<12:08, 30.37s/it]
  8%|▊         | 2/25 [00:33<05:31, 14.43s/it]
 12%|█▏        | 3/25 [00:36<03:25,  9.33s/it]
 16%|█▌        | 4/25 [00:40<02:25,  6.93s/it]
 20%|██        | 5/25 [00:43<01:52,  5.61s/it]
 24%|██▍       | 6/25 [00:46<01:31,  4.81s/it]
 28%|██▊       | 7/25 [00:49<01:17,  4.31s/it]
 32%|███▏      | 8/25 [00:53<01:07,  3.98s/it]
 36%|███▌      | 9/25 [00:56<01:00,  3.76s/it]
 40%|████      | 10/25 [00:59<00:54,  3.61s/it]
 44%|████▍     | 11/25 [01:03<00:49,  3.50s/it]
 48%|████▊     | 12/25 [01:06<00:44,  3.43s/it]
 52%|█████▏    | 13/25 [01:09<00:40,  3.38s/it]
 56%|█████▌    | 14/25 [01:12<00:36,  3.35s/it]
 60%|██████    | 15/25 [01:16<00:33,  3.33s/it]
 64%|██████▍   | 16/25 [01:19<00:29,  3.31s/it]
 68%|██████▊   | 17/25 [01:22<00:26,  3.30s/it]
 72%|███████▏  | 18/25 [01:25<00:23,  3.29s/it]
 76%|███████▌  | 19/25 [01:29<00:19,  3.29s/it]
 80%|████████  | 20/25 [01:32<00:16,  3.28s/it]
 84%|████████▍ | 21/25 [01:35<00:13,  3.28s/it]
 88%|████████▊ | 22/25 [01:39<00:09,  3.28s/it]
 92%|█████████▏| 23/25 [01:42<00:06,  3.28s/it]
 96%|█████████▌| 24/25 [01:45<00:03,  3.28s/it]
100%|██████████| 25/25 [01:48<00:00,  3.28s/it]
100%|██████████| 25/25 [01:48<00:00,  4.36s/it]
2024-04-24T12:22:04.025640140+10:00 2:22AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr OpenCV: FFMPEG: tag 0x7634706d/'mp4v' is not supported with codec id 12 and format 'image2 / image2 sequence'
2:22AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr [image2 @ 0x71920e7c5700] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
2:22AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr [image2 @ 0x71920e7c5700] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
2024-04-24T12:22:04.031053502+10:00 2:22AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr [image2 @ 0x71920e7c5700] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
2024-04-24T12:22:04.032093613+10:00 2:22AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr [image2 @ 0x71920e7c5700] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
2024-04-24T12:22:04.033116524+10:00 2:22AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr [image2 @ 0x71920e7c5700] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
2:22AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr [image2 @ 0x71920e7c5700] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
2:22AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr [image2 @ 0x71920e7c5700] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
2024-04-24T12:22:04.036202567+10:00 2:22AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr [image2 @ 0x71920e7c5700] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
2:22AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr [image2 @ 0x71920e7c5700] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
2:22AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr [image2 @ 0x71920e7c5700] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
2:22AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr [image2 @ 0x71920e7c5700] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
2:22AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr [image2 @ 0x71920e7c5700] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
2024-04-24T12:22:04.041174054+10:00 2:22AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr [image2 @ 0x71920e7c5700] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
2024-04-24T12:22:04.042167565+10:00 2:22AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr [image2 @ 0x71920e7c5700] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
2:22AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr [image2 @ 0x71920e7c5700] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
2024-04-24T12:22:04.044204687+10:00 2:22AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr [image2 @ 0x71920e7c5700] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
2:22AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr [image2 @ 0x71920e7c5700] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
2:22AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr [image2 @ 0x71920e7c5700] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
2:22AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr [image2 @ 0x71920e7c5700] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
2:22AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr [image2 @ 0x71920e7c5700] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
2024-04-24T12:22:04.049558320+10:00 2:22AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr [image2 @ 0x71920e7c5700] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
2:22AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr [image2 @ 0x71920e7c5700] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
2024-04-24T12:22:04.051651632+10:00 2:22AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr [image2 @ 0x71920e7c5700] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
2:22AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr [image2 @ 0x71920e7c5700] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
2:22AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr [image2 @ 0x71920e7c5700] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
2:22AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr [image2 @ 0x71920e7c5700] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
2024-04-24T12:22:04.055589208+10:00 2:22AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr [image2 @ 0x71920e7c5700] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
2024-04-24T12:22:04.056704888+10:00 2:22AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr [image2 @ 0x71920e7c5700] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
2:22AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr [image2 @ 0x71920e7c5700] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
2:22AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr [image2 @ 0x71920e7c5700] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
2024-04-24T12:22:04.059794662+10:00 2:22AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr [image2 @ 0x71920e7c5700] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
2:22AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr [image2 @ 0x71920e7c5700] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
2:22AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr [image2 @ 0x71920e7c5700] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
2024-04-24T12:22:04.062824305+10:00 2:22AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr [image2 @ 0x71920e7c5700] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
2:22AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr [image2 @ 0x71920e7c5700] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
2:22AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr [image2 @ 0x71920e7c5700] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
2:22AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr [image2 @ 0x71920e7c5700] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
2:22AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr [image2 @ 0x71920e7c5700] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
2024-04-24T12:22:04.067876051+10:00 2:22AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr [image2 @ 0x71920e7c5700] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
2:22AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr [image2 @ 0x71920e7c5700] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
2:22AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr [image2 @ 0x71920e7c5700] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
2:22AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr [image2 @ 0x71920e7c5700] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
2:22AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr [image2 @ 0x71920e7c5700] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
2:22AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr [image2 @ 0x71920e7c5700] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
2:22AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr [image2 @ 0x71920e7c5700] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
2024-04-24T12:22:04.076252358+10:00 2:22AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr [image2 @ 0x71920e7c5700] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
2:22AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr [image2 @ 0x71920e7c5700] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
2:22AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr [image2 @ 0x71920e7c5700] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
2024-04-24T12:22:04.079378420+10:00 2:22AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr [image2 @ 0x71920e7c5700] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
2:22AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr [image2 @ 0x71920e7c5700] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
2024-04-24T12:22:04.081510162+10:00 2:22AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr [image2 @ 0x71920e7c5700] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
2:22AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr [image2 @ 0x71920e7c5700] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
2024-04-24T12:22:04.083611773+10:00 2:22AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr [image2 @ 0x71920e7c5700] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
2024-04-24T12:22:04.084636124+10:00 2:22AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr [image2 @ 0x71920e7c5700] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
2:22AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr [image2 @ 0x71920e7c5700] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
2:22AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr [image2 @ 0x71920e7c5700] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
2024-04-24T12:22:04.087681838+10:00 2:22AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr [image2 @ 0x71920e7c5700] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
2:22AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr [image2 @ 0x71920e7c5700] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
2:22AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr [image2 @ 0x71920e7c5700] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
2:22AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr [image2 @ 0x71920e7c5700] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
2024-04-24T12:22:04.091816442+10:00 2:22AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr [image2 @ 0x71920e7c5700] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
2024-04-24T12:22:04.092768183+10:00 2:22AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr [image2 @ 0x71920e7c5700] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
2:22AM DBG GRPC(damo-vilab/text-to-video-ms-1.7b-127.0.0.1:37463): stderr [image2 @ 0x71920e7c5700] Cannot write more than one file with the same name. Are you missing the -update option or a sequence pattern?
2:22AM DBG Response: {"created":1713925324,"id":"0c99cdef-01ed-4851-8395-b948b2bf58d6","data":[{"embedding":null,"index":0,"url":"http://ai-api.enphi.net/generated-images/b641723442895.png"}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
2024-04-24T12:22:04.094315900+10:00 [10.42.182.221]:55894 200 - POST /v1/images/generations

So now it appears that there is an issue with the the pipeline itself :|

jtwolfe commented 7 months ago

image it does seem that spiderman is surfing but in png or mp4 the file encounters MIME errors