search

BAAI-DCAI / Bunny

A family of lightweight multimodal models.
Apache License 2.0
865 stars 65 forks source link

A few issues running the README #6

Closed carlini closed 6 months ago

carlini commented 6 months ago

I'm trying to spin up the server so I can run this for inference as described in the README and I've hit a few issues.

First: demo_3.png and demo_4.png don't exist. This is easy to fix here:

https://github.com/BAAI-DCAI/Bunny/blob/516437ebf250b99e3967b0fb1ad5cb6bc8b43953/bunny/serve/gradio_web_server.py#L348-L349

should be edited to example_1.png and example_2.png.

Second: (and why this isn't just a PR) I can't get the service to start with the model to load phi-2. I'm just trying to get the inference demo working.

If I run the model_worker service with --model-type phi-2 then I get a crash KeyError: 'BunnyPhiConfig' when it tries to load the tokenizer. It looks like you try to configure this config somewhere but it doesn't get added to the huggingface transformers list of known configs for some reason.

Are there other steps required (e.g., modifying the huggingface code)?

Third: I don't understand what set of model paths I should be passing to run the service if I don't want to fine-tune anything. Could you give an example for what model-path should be? I've downloaded bunny-phi-2-siglip-lora and I'm passing this as the path, but I can't test this because of the prior crash.

I'm pretty sure I have the correct versions of everything installed. Have you tried following the readme on a clean machine install to verify it runs as expected?

Isaachhh commented 6 months ago

Thanks for your attention!

For the first part, thank you for finding the typo and we will fix it.

For the second part, BunnyPhiConfig is defined here. If you install this project as INSTALL in README, it shouldn't be a KeyError. Could you please share more information?

For the third part,

python -m bunny.serve.model_worker \
    --host 0.0.0.0 \
    --controller http://localhost:10000 \
    --port 40000 \
    --worker http://localhost:40000 \
    --model-path  /path/to/bunny-phi-2-siglip-lora \
    --model-base /path/to/microsoft/phi-2 \
    --model-type phi-2
carlini commented 6 months ago

Ah okay the third point fixes both the second and third.

If you don't pass --model-base to the phi-2 path then you get the error, and if you do pass it then it works correctly. Thanks for the help.