search

huggingface / transformers

šŸ¤— Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.
https://huggingface.co/transformers
Apache License 2.0
133.23k stars 26.61k forks source link

Problems with layoutlm language model #20138

Closed mv96 closed 1 year ago

mv96 commented 1 year ago

Feature request

Hi,

I was recently reading Layoutlm and it's variants and I figured out that they do exist in hugging face but there are several problems with It or I am not very sure to understand it just by reading the documentation.

Q1) Layoutlm is a multimodal machine learning transformer then why is it listed in the text transformers category on HuggingFace ?

Q2) Even if it is multimodal, layoutlm does not tend to use the image anywhere in the examples of layoutlm Layoutlm ?

Q3) For Layoutlmv2 there is no tensorflow class like TFlayoutlm as it is available for version1 ?

Q4) there is no MLM head class as it available for it's version 1 ? so I am not sure if I want to pretrain this model from scratch how do I do that ?

Q5) Same as Q4 there is not MLM head class, so in this case if I have my own tokenizer and I want to pretrain from scratch layoutlm and then simply want to change the transformer with one line of code change, that's not possible because they have different heads ?

I am a bit new to HF interface so forgive me if I asked something super basic. I don't know if I have something wrong with understanding layoutlm in the first place or these are valid questions, But I would be very happy if anyone can shed some light on this !! Once again thanks for taking the time to read šŸ˜ŠšŸ˜Š , have a good day !!

Motivation

Just find it very difficult to understand the implementation of the specific model from the transformers library

Your contribution

I can try to look into it, but first I need to know if the problem is really a problem or it is just my wrong understanding of the library.

sgugger commented 1 year ago

Please use the forums to ask such questions as we keep issues for bugs and feature requests only :-) cc @NielsRogge

NielsRogge commented 1 year ago

Please link your question on the forum, I'll answer there!

mv96 commented 1 year ago

@NielsRogge can you send me the link to access forums, I tried to look at the discord channel of hugging face but I am not exactly sure where to ask questions.

Thanks

github-actions[bot] commented 1 year ago

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.