search

flowersteam / Grounding_LLMs_with_online_RL

We perform functional grounding of LLMs' knowledge in BabyAI-Text
MIT License
209 stars 24 forks source link

Can I verify the FPS? #25

Open ArchyZheng opened 2 months ago

ArchyZheng commented 2 months ago

Hello, Everyone.

I am just curious that what fps you have when you run T5-small and T5-large at the default task setting?

I just run at 12 FPS at T5-small, using 4 3090 GPU under default task setting(env number 34, horizon length is 34 * 40). I just wanna know what's yours.

ClementRomac commented 2 months ago

Hey @ArchyZheng,

Thanks for your question. I don't have any numbers to provide on my side (we did not keep track of the whole clock time in the paper and we are now using slightly different setups with LoRA), but I'm interested to hear anyone else's FPS.

Properly calibrating the minibatch_size is usually what matters the most to increase speed :)