Lines in the temporal_style & overall_consistency metrics file (link1, link2) will return an error for custom prompts longer than 77, and the variable text obtained by these lines will not be used in subsequent code, so the codes is commented to avoid errors and speed up metric calculation.
Error logs that may occur before modification:
```
Traceback (most recent call last):
File "/data/VBench/evaluate.py", line 159, in
main()
File "/data/VBench/evaluate.py", line 145, in main
result_name = my_VBench.evaluate(
File "/data/VBench/vbench/__init__.py", line 187, in evaluate
results = evaluate_func(cur_full_info_path, self.device, submodules_list, **kwargs)
File "/data/VBench/vbench/temporal_style.py", line 61, in compute_temporal_style
all_results, video_results = temporal_style(viclip, video_dict, tokenizer, device)
File "/data/VBench/vbench/temporal_style.py", line 39, in temporal_style
text = clip.tokenize([query]).to(device)
File "/data/anaconda3/envs/pixart/lib/python3.9/site-packages/clip/clip.py", line 234, in tokenize
raise RuntimeError(f"Input {texts[i]} is too long for context length {context_length}")
RuntimeError: Input Drone view of waves crashing against the rugged cliffs along Big Sur’s garay point beach. The crashing blue waters create white-tipped waves, while the golden light of the setting sun illuminates the rocky shore. A small island with a lighthouse sits in the distance, and green shrubbery covers the cliff’s edge. The steep drop from the road down to the beach is a dramatic feat, with the cliff’s edges jutting out over the sea. This is a view that captures the raw beauty of the coast and the rugged landscape of the Pacific Coast Highway., masterpiece, best quality, highly detailed, good composition, extremely delicate and beautiful, 4k, 8k is too long for context length 77
```
makedirs issue in VBench.init()
The original code will cause an error when performing multiprocess processing.
For example, when two processes are running to calculate different metrics, process 1 will report an error when process 2 creates the self.output_path folder after process 1 successfully executes if. The revised code avoids this problem.
Clip issues for custom prompt
Lines in the temporal_style & overall_consistency metrics file (link1, link2) will return an error for custom prompts longer than 77, and the variable
textobtained by these lines will not be used in subsequent code, so the codes is commented to avoid errors and speed up metric calculation.Error logs that may occur before modification:
``` Traceback (most recent call last): File "/data/VBench/evaluate.py", line 159, inmakedirs issue in VBench.init()
The original code will cause an error when performing multiprocess processing.
For example, when two processes are running to calculate different metrics, process 1 will report an error when process 2 creates the
self.output_pathfolder after process 1 successfully executesif. The revised code avoids this problem.