dandelin / ViLT

Code for the ICML 2021 (long talk) paper: "ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision"
Apache License 2.0
1.36k stars 209 forks source link

How to set the config to create a stand_alone commandline demo ? #70

Closed Ngheissari closed 1 year ago

Ngheissari commented 1 year ago

Here is the function that loads the model and tokenizer. What would be the _config ??

using @ex.automain would result in the follwing warning:

WARNING - timm.models.helpers - No pretrained weights exist or were found for this model. Using random initialization.

Here is the function.

def prepare_model(_config): _config = copy.deepcopy(_config)

loss_names = {
    "itm": 0,
    "mlm": 0.5,
    "mpp": 0,
    "vqa": 0,
    "imgcls": 0,
    "nlvr2": 0,
    "irtr": 0,
    "arc": 0,
}
tokenizer = get_pretrained_tokenizer(_config["tokenizer"])

_config.update(
    {
        "loss_names": loss_names,
    }
)

model = ViLTransformerSS(_config)
model.setup("test")
model.eval()

# device = "cuda:0" if _config["num_gpus"] > 0 else "cpu"
device = "cpu"
model.to(device)
return tokenizer,model, device
Ngheissari commented 1 year ago

fixed it. My code works. There was a problem with my environment.