search

Jungwoo4021 / OS-KDFT

10 stars 1 forks source link

Request for Pre-trained Model and Guidance on Using SSL-Compression as Feature Extractor #2

Open hungdinhxuan opened 10 months ago

hungdinhxuan commented 10 months ago

Hello @Jungwoo4021 ,

I am currently working on a project that could greatly benefit from the pre-trained models you have developed. I have encountered some challenges in training models from scratch and believe that using a pre-trained model would significantly enhance my work.

Additionally, I am interested in utilizing the SSL-compression method as a feature extractor within my project. I would appreciate any documentation or examples on how to implement this effectively.

Could you please provide the weight checkpoints for the pre-trained models? This would be incredibly helpful for comparison purposes and to advance my research. Furthermore, any guidance or instructions on using SSL-compression for feature extraction would be greatly appreciated.

Thank you for your work.

Jungwoo4021 commented 10 months ago

Hello @hungdinhxuan,

Thank you for your interest in OS-KDFT. Could you please explain a bit more about the feature extraction with SSL compression? It would be helpful if you could share your specific goals and usage scenarios, so I can provide more tailored guidance.

Regarding your request for 'weight checkpoints for the pre-trained models', I'd like to clarify which models you're referring to. Are you referring to the pre-trained SSL model, the HuBERT teacher, or the HuBERT student model which is compressed using the OS-KDFT method? If it's the former, Huggingface officially offers them (you can refer to the link here: https://(https://huggingface.co/facebook/hubert-base-ls960). If it's the latter, we plan to upload them in the next 1-2 days. These weight checkpoints will be provided and can be used in (provide information on how they will be provided and how they can be used).