chaoyi-wu / RadFM

The official code for "Towards Generalist Foundation Model for Radiology by Leveraging Web-scale 2D&3D Medical Data".
315 stars 32 forks source link

how to pass multiple images #26

Open anishmenon opened 5 months ago

anishmenon commented 5 months ago

Can anyone revert a demo code to to send multiple images to prompt. I can only see 1 image and questions

Screenshot 2024-02-21 at 8 46 09 PM

Gender: Male Perianal purulent discharge and tenderness. Please caption this scan with finding and impression.

chaoyi-wu commented 5 months ago

You can check the test demo line https://github.com/chaoyi-wu/RadFM/blob/edf73149f09f5328b6530a073be04be1fe0d303d/Quick_demo/test.py#L86, as follow:

image =[ { 'img_path': './view1_frontal.jpg', 'position': 0, #indicate where to put the images in the text string, range from [0,len(question)-1] }, # can add abitrary number of imgs ]

Simply, you can just appand the list and the `position' key can indicate where to insert the image, indexing following string length. For example 0 for the begginning and len(question)-1 to for the end. For multiple images, the indexes can be the same, the code will insert image one-by-one following list order at the same postition. For the text part, you can just give the plain text and no need for any image placeholder.

Hope this can help you.