search

NielsRogge / Transformers-Tutorials

This repository contains demos I made with the Transformers library by HuggingFace.
MIT License
9.56k stars 1.46k forks source link

Why do you need custom Features? #441

Closed gordon-lim closed 4 months ago

gordon-lim commented 5 months ago

Why do we need the following in the Fine-tuning LayoutLMv2ForSequenceClassification on RVL-CDIP notebook?

# we need to define custom features
features = Features({
    'image': Array3D(dtype="int64", shape=(3, 224, 224)),
    'input_ids': Sequence(feature=Value(dtype='int64')),
    'attention_mask': Sequence(Value(dtype='int64')),
    'token_type_ids': Sequence(Value(dtype='int64')),
    'bbox': Array2D(dtype="int64", shape=(512, 4)),
    'labels': ClassLabel(num_classes=len(labels), names=labels),
})
NielsRogge commented 5 months ago

Hi,

I should actually update this notebook as now we can use the native Image feature rather than Array3D for the image column. Did you try to run the code without defining custom features, and if so, did it work?

gordon-lim commented 4 months ago

My apologies for the late response. I was asking because I faced some bug when I did not define custom features as you have. However, I don't recall how I eventually solved the bug (nor what the bug was) but regardless I was curious why we needed that block of code.