search

BAAI-DCAI / SpatialBot

The official repo for "SpatialBot: Precise Spatial Understanding with Vision Language Models.
MIT License
157 stars 10 forks source link

Batched Inference for SpatialBot #5

Closed Yuxin916 closed 3 months ago

Yuxin916 commented 3 months ago

Hi! I am wondering is there a way for the model.generate or model() forward pass to handle batched images. For example, now the image_tensor input is in dimension of 2(RGB and Depth) x 3(Channel) x 384(H) x 384(W), which is batch_size=1. What if i have multiple pair of RGBD image to do forward pass: such as batch x 2(RGB and Depth) x 3(Channel) x 384(H) x 384(W). How should i modify the model forward pass, and how should i modify the input_ids? Thank you!

RussRobin commented 3 months ago

Hi @Yuxin916

Sorry but I haven’t implemented batch inference on hf version of spatialbot.

You may want to refer to Bunny implementation of batch inference here

Hope it helps.

Yuxin916 commented 3 months ago

Thank you for your quick reply. I will take a look at it.

Cheers.