search

vietai / ViT5

MIT License
59 stars 9 forks source link

viT5 base issue #2

Closed hieudx149 closed 2 years ago

hieudx149 commented 2 years ago

Hi @heraclex12, As far as I understand, T5 is trained using the span-corruption objective, I tried to create a sample similar to the training data, and the result is still just a random sequence. Is there any way (or task) to try vi-T5 base without fine-tune step to see how effective T5 is ? This problem only occurs in viT5-base version, large version seems to work well. (I create new issue make sure you guys can see it, because issue 1 was closed) Span-corruption objective image My code test viT5-base image Result test on large version image

Originally posted by @r1ckC139 in https://github.com/vietai/ViT5/issues/1#issuecomment-1154989427

heraclex12 commented 2 years ago

Hi @r1ckC139,

Thank you for your feedback. We have recently realized some problems with the base config and quickly fixed them. So from now, you can try to load the base model to make some predictions.

If you face any problems with our models, feel free to let us know.

Thank you!