How can I train VGT?

[Seoung-wook]

In my case, I use command like below.

CUDA_VISIBLE_DEVICES=2 python train_VGT.py --config-file Configs/cascade/D4LA_VGT_cascade_PTM.yaml --num-gpus 1 MODEL.WEIGHTS ./pretrained/D4LA_VGT_model.pth OUTPUT_DIR ./trained

error accured like below.

Traceback (most recent call last): File "/workspace/future_task/AdvancedLiterateMachinery/DocumentUnderstanding/VGT/object_detection/train_VGT.py", line 156, in launch( File "/home/swchoi/.conda/envs/vgt/lib/python3.9/site-packages/detectron2/engine/launch.py", line 84, in launch main_func(*args) File "/workspace/future_task/AdvancedLiterateMachinery/DocumentUnderstanding/VGT/object_detection/train_VGT.py", line 131, in main trainer = MyTrainer(cfg) File "/workspace/future_task/AdvancedLiterateMachinery/DocumentUnderstanding/VGT/object_detection/ditod/VGTTrainer.py", line 432, in init model = self.build_model(cfg) File "/workspace/future_task/AdvancedLiterateMachinery/DocumentUnderstanding/VGT/object_detection/ditod/VGTTrainer.py", line 571, in build_model model = build_model(cfg) File "/home/swchoi/.conda/envs/vgt/lib/python3.9/site-packages/detectron2/modeling/meta_arch/build.py", line 22, in build_model model = META_ARCH_REGISTRY.get(meta_arch)(cfg) File "/home/swchoi/.conda/envs/vgt/lib/python3.9/site-packages/detectron2/config/config.py", line 190, in wrapped init_func(self, **explicit_args) File "/workspace/future_task/AdvancedLiterateMachinery/DocumentUnderstanding/VGT/object_detection/ditod/VGT.py", line 48, in init self.Wordgrid_embedding = WordnnEmbedding(vocab_size, hidden_size, embedding_dim, \ File "/workspace/future_task/AdvancedLiterateMachinery/DocumentUnderstanding/VGT/object_detection/ditod/Wordnn_embedding.py", line 39, in init self.init_weights(bros_embedding_path) File "/workspace/future_task/AdvancedLiterateMachinery/DocumentUnderstanding/VGT/object_detection/ditod/Wordnn_embedding.py", line 47, in init_weights state_dict = torch.load(bros_embedding_path + "pytorch_model.bin", map_location='cpu') File "/home/swchoi/.conda/envs/vgt/lib/python3.9/site-packages/torch/serialization.py", line 699, in load with _open_file_like(f, 'rb') as opened_file: File "/home/swchoi/.conda/envs/vgt/lib/python3.9/site-packages/torch/serialization.py", line 231, in _open_file_like return _open_file(name_or_buffer, mode) File "/home/swchoi/.conda/envs/vgt/lib/python3.9/site-packages/torch/serialization.py", line 212, in init super(_open_file, self).init(open(name, mode)) FileNotFoundError: [Errno 2] No such file or directory: '/path/layoutlm-base-uncased/pytorch_model.bin'

I think FileNotFoundError: [Errno 2] No such file or directory: '/path/layoutlm-base-uncased/pytorch_model.bin' caused by Wordnn_embedding.py's init_weights method.

Please answer how I can solve the problem.

[sibam23]

I solved this issue by using git clone and downloading the layoutlm tokenizer from huggingface

[youseon-jang]

@sibam23 Hi! Can you explain more about your solution? Thanks!

[Harsh19012003]

Actually the error is you donot have weights file which code is suppose to download on the go from http url. To be precise, code directly specifies the downloading of layoutlm-base-uncased/pytorch_model.bin from external link but unfortunately its private, so weights downloading fails
You can use this code to download weights, by creating a python file and downloading directly from library. You can then use this directly.

# Use transformers library
from transformers import LayoutLMModel

# Load pretrained model from transformers
model = LayoutLMModel.from_pretrained("microsoft/layoutlm-base-uncased")

# Save it in local directory structure
model.save_pretrained("./weights/layoutlm/")

or
you can also try download the weights externally from Hugging Face and provide path to this model present in your directory (Not sure about this method!)