Anurich
commented
3 years ago Ok I found the solution if someone face same problem the solution is
bbox_input = []
for batch_index in range(len(tokenized_inputs["input_ids"])):
word_ids = tokenized_inputs.word_ids(batch_index=batch_index)
org_batch_index = tokenized_inputs["overflow_to_sample_mapping"][batch_index]
previous_word_idx = None
label = labels[org_batch_index]
bounding_box = bboxes[org_batch_index]
label_ids = []
bbox_ids = []
for word_idx in word_ids:
# Special tokens have a word id that is None. We set the label to -100 so they are automatically
# ignored in the loss function.
if word_idx is None:
label_ids.append(-100)
bbox_ids.append([0,0,0,0])
# We set the label for the first token of each word.
elif word_idx != previous_word_idx:
label_ids.append(label2id[label[word_idx]])
bbox_ids.append(bounding_box[word_idx])
# For the other tokens in a word, we set the label to either the current label or -100, depending on
# the label_all_tokens flag.
else:
label_ids.append(label2id[label[word_idx]] if label_all_tokens else -100)
bbox_ids.append(bounding_box[word_idx])
previous_word_idx = word_idx
#assert len(label_ids) == len(tokenized_inputs["input_ids"][batch_index])
labels_token.append(label_ids)
bbox_input.append(bbox_ids)
#assert len(labels) == len(tokenized_inputs["input_ids"])
tokenized_inputs["labels"] = labels_token
tokenized_inputs["bbox"] = bbox_input
rahulchsrs
tstanislawek
I clone this repository after going through the code. I don't find how they preprocess the bounding box currently they expect BBOX to be shape (batch, seq, 4) and original bounding box is of shape (x,y, h, w) . Anyone has any idea about how to do it.