search

chaoyi-wu / RadFM

The official code for "Towards Generalist Foundation Model for Radiology by Leveraging Web-scale 2D&3D Medical Data".
332 stars 33 forks source link

It looks like transformers==4.28.1 will introduce weird output #27

Closed Ashespt closed 6 months ago

Ashespt commented 7 months ago

Thanks for your great job! Compared to some works I reproduced, this repo's result looks convinced. But, I met some problems during training with transformers==4.28.1. The situation is that the correct result is produced in transformers==4.28.0.dev0-py3.9.egg while 4.28.1 will come out weird output. for 4.28.0.dev0-py3.9.egg image for 4.28.1 815c519ba73259568a95a42cbad533f However, if I used 4.28.0.dev0-py3.9.egg , it will throw errors like

ImportError: cannot import name 'strtobool' from 'transformers.utils'

Do you have any idea about this weird output or dependence? Thanks.

chaoyi-wu commented 6 months ago

The issue seems about the language model, i.e. the language model PMC-LLaMA is trained under the 4.28.0.dev0-py3.9.egg version. You'd better check the prompt files to pick a perfect prompt to perform the report generation task, as listed in https://github.com/chaoyi-wu/RadFM/blob/main/src/Dataset/dataset/report_prompt.json.

We are also under checking why the transformers version changes will make the LM so sensitivity to prompts.

Hope this can help you.

Ashespt commented 6 months ago

Thanks for your reply. With 4.28.0.dev0-py3.9.egg and modifying some code, the project was ran successfully.