search

ggjy / DeLVM

109 stars 8 forks source link

The performance on more tasks #1

Open zhuqiang02 opened 9 months ago

zhuqiang02 commented 9 months ago

Your profile photo are just like you! Niubility! I have been waiting LVM release code longlong time.

This work has a great performance on segment&pose&deraining. And did you test on more tasks? (Especially the 3D tasks such as
depth estimation, which the original LVM performance is good） In other words, I'm very curious about the multi-task capability of LVM model with less training data. Could u show more experimental results？

ggjy commented 9 months ago

Thank you for your interest in our work. 🙏

Indeed, we did not use data from other tasks for training, so our model currently does not have the capability for zero-shot learning, such as depth estimation. Even with in-context prompts, our model would not be able to perform such tasks as they are not included in our training data.

We open-sourced our data processing code, which you can refer to for generating data suitable for internLM, potentially applicable to tasks like depth estimation. However, it's important to note that based on our experience, if the training data is too limited and lacks sufficient data augmentation, the model's performance can be significantly affected.

yuhaoliu7456 commented 8 months ago

Thank you for your interest in our work. 🙏

Indeed, we did not use data from other tasks for training, so our model currently does not have the capability for zero-shot learning, such as depth estimation. Even with in-context prompts, our model would not be able to perform such tasks as they are not included in our training data.

We open-sourced our data processing code, which you can refer to for generating data suitable for internLM, potentially applicable to tasks like depth estimation. However, it's important to note that based on our experience, if the training data is too limited and lacks sufficient data augmentation, the model's performance can be significantly affected.

I see in LVM there are some examples/tasks are not included in their training sets but still can work well. Can you briefly explain why your model can not do this? thx.