search

baaivision / Emu

Emu Series: Generative Multimodal Models from BAAI
https://baaivision.github.io/emu2/
Apache License 2.0
1.66k stars 86 forks source link

Issue with EMU2-Gen #75

Closed Woodyet closed 10 months ago

Woodyet commented 10 months ago

Hi when running the following

# For the first time of using,
# you need to download the huggingface repo "BAAI/Emu2-GEN" to local first
path = "path to local BAAI/Emu2-GEN"

multimodal_encoder = AutoModelForCausalLM.from_pretrained(
    f"{path}/multimodal_encoder",
    trust_remote_code=True,
    torch_dtype=torch.bfloat16,
    use_safetensors=True,
    variant="bf16"
)
tokenizer = AutoTokenizer.from_pretrained(f"{path}/tokenizer")

pipe = DiffusionPipeline.from_pretrained(
    path,
    custom_pipeline="pipeline_emu2_gen",
    torch_dtype=torch.bfloat16,
    use_safetensors=True,
    variant="bf16",
    multimodal_encoder=multimodal_encoder,
    tokenizer=tokenizer,
)

I get this error

File "/data/home/eex869/EMU2/Emu/Emu2/test_b.py", line 32, in pipe = DiffusionPipeline.from_pretrained( File "/data/home/eex869/.conda/envs/EMU_2/lib/python3.9/site-packages/diffusers/pipelines/pipeline_utils.py", line 1141, in from_pretrained pipeline_class = _get_pipeline_class( File "/data/home/eex869/.conda/envs/EMU_2/lib/python3.9/site-packages/diffusers/pipelines/pipeline_utils.py", line 374, in _get_pipeline_class return get_class_from_dynamic_module( File "/data/home/eex869/.conda/envs/EMU_2/lib/python3.9/site-packages/diffusers/utils/dynamic_modules_utils.py", line 456, in get_class_from_dynamic_module return get_class_in_module(class_name, final_module.replace(".py", "")) File "/data/home/eex869/.conda/envs/EMU_2/lib/python3.9/site-packages/diffusers/utils/dynamic_modules_utils.py", line 163, in get_class_in_module module = importlib.import_module(module_path) File "/data/home/eex869/.conda/envs/EMU_2/lib/python3.9/importlib/init.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "", line 1030, in _gcd_import File "", line 1007, in _find_and_load File "", line 986, in _find_and_load_unlocked File "", line 680, in _load_unlocked File "", line 850, in exec_module File "", line 228, in _call_with_frames_removed File "/data/home/eex869/.cache/huggingface/modules/diffusers_modules/local/pipeline_emu2_gen.py", line 45, in class EmuVisualGenerationPipeline(DiffusionPipeline): File "/data/home/eex869/.cache/huggingface/modules/diffusers_modules/local/pipeline_emu2_gen.py", line 90, in EmuVisualGenerationPipeline inputs: List[Image.Image | str] | str | Image.Image, TypeError: unsupported operand type(s) for |: 'type' and 'type'

Woodyet commented 10 months ago

re-installed enviorment with python 3.11 fixed the issue