gandharvsuri
commented
3 years ago @patil-suraj
Closed gandharvsuri closed 3 years ago
gandharvsuri
commented
3 years ago @patil-suraj
patrickvonplaten
commented
3 years ago Hey @gandharvsuri,
Running your (slighly adapted) code example does not give me any errors.
Also, please don't ping more people if you don't receive an answer within a day (no need to also ping @patil-suraj). We try to answer all issues and to do so efficiently, it is not helpful to be pinged unnecessarly. Thanks!
gandharvsuri
commented
3 years ago Hey @patrickvonplaten, my apologies, I just saw him replying on similar issues so thought of pinging him as well, I agree I shouldn't have done that.
Can you try with this custom input?
text = """Dred Scott v. Sandford, 60 U.S. (19 How.) 393 (1857), often referred to as the Dred Scott decision, was a landmark decision of the US Supreme Court in which the Court held that the US Constitution was not meant to include American citizenship for black people, regardless of whether they were enslaved or free, and so the rights and privileges that the Constitution confers upon American citizens could not apply to them.The decision was made in the case of Dred Scott, an enslaved black man whose owners had taken him from Missouri, which was a slave-holding state, into Illinois and the Wisconsin Territory, which were free areas where slavery was illegal. When his owners later brought him back to Missouri, Scott sued in court for his freedom and claimed that because he had been taken into "free" U.S. territory, he had automatically been freed and was legally no longer a slave. Scott sued first in Missouri state court, which ruled that he was still a slave under its law. He then sued in US federal court, which ruled against him by deciding that it had to apply Missouri law to the case. He then appealed to the US Supreme Court.
In March 1857, the Supreme Court issued a 7–2 decision against Dred Scott. In an opinion written by Chief Justice Roger Taney, the Court ruled that black people "are not included, and were not intended to be included, under the word 'citizens' in the Constitution, and can therefore claim none of the rights and privileges which that instrument provides for and secures to citizens of the United States." Taney supported his ruling with an extended survey of American state and local laws from the time of the Constitution's drafting in 1787 that purported to show that a "perpetual and impassable barrier was intended to be erected between the white race and the one which they had reduced to slavery." Because the Court ruled that Scott was not an American citizen, he was also not a citizen of any state and, accordingly, could never establish the "diversity of citizenship" that Article III of the US Constitution requires for a US federal court to be able to exercise jurisdiction over a case. After ruling on those issues surrounding Scott, Taney continued further and struck down the entire Missouri Compromise as a limitation on slavery that exceeded the US Congress's constitutional powers.
Although Taney and several of the other justices hoped that the decision would permanently settle the slavery controversy, which was increasingly dividing the American public, the decision's effect was the complete opposite. Taney's majority opinion suited the slaveholding states, but was intensely decried in all the other states. The decision inflamed the national debate over slavery and deepened the divide that led ultimately to the Civil War. In 1865, after the Union won the Civil War, the Dred Scott ruling was voided by the Thirteenth Amendment to the US Constitution, which abolished slavery except as punishment for a crime, and the Fourteenth Amendment, which guaranteed citizenship for "all persons born or naturalized in the United States, and subject to the jurisdiction thereof."
The Supreme Court's decision has been widely denounced ever since, both for how overtly racist the decision was and its crucial role in the near destruction of the United States four years later. Bernard Schwartz said that it "stands first in any list of the worst Supreme Court decisions—Chief Justice Hughes called it the Court's greatest self-inflicted wound." Junius P. Rodriguez said that it is "universally condemned as the U.S. Supreme Court's worst decision". Historian David Thomas Konig said that it was "unquestionably, our court's worst decision ever."
Abraham Lincoln (; February 12, 1809 – April 15, 1865) was an American statesman and lawyer who served as the 16th president of the United States from 1861 until his assassination in 1865. Lincoln led the nation through the American Civil War, the country's greatest moral, cultural, constitutional, and political crisis. He succeeded in preserving the Union, abolishing slavery, bolstering the federal government, and modernizing the U.S. economy.
Lincoln was born into poverty in a log cabin and was raised on the frontier primarily in Indiana. He was self-educated and became a lawyer, Whig Party leader, Illinois state legislator, and U.S. Congressman from Illinois. In 1849, he returned to his law practice but became vexed by the opening of additional lands to slavery as a result of the Kansas–Nebraska Act. He reentered politics in 1854, becoming a leader in the new Republican Party, and he reached a national audience in the 1858 debates against Stephen Douglas. Lincoln ran for President in 1860, sweeping the North in victory. Pro-slavery elements in the South equated his success with the North's rejection of their right to practice slavery, and southern states began seceding from the union. To secure its independence, the new Confederate States fired on Fort Sumter, a U.S. fort in the South, and Lincoln called up forces to suppress the rebellion and restore the Union.
As the leader of moderate Republicans, Lincoln had to navigate a contentious array of factions with friends and opponents on both sides. War Democrats rallied a large faction of former opponents into his moderate camp, but they were countered by Radical Republicans, who demanded harsh treatment of the Southern Confederates. Anti-war Democrats (called "Copperheads") despised him, and irreconcilable pro-Confederate elements plotted his assassination. Lincoln managed the factions by exploiting their mutual enmity, by carefully distributing political patronage, and by appealing to the U.S. people. His Gettysburg Address became a historic clarion call for nationalism, republicanism, equal rights, liberty, and democracy. Lincoln scrutinized the strategy and tactics in the war effort, including the selection of generals and the naval blockade of the South's trade. He suspended habeas corpus, and he averted British intervention by defusing the Trent Affair. He engineered the end to slavery with his Emancipation Proclamation and his order that the Army protect and recruit former slaves. He also encouraged border states to outlaw slavery, and promoted the Thirteenth Amendment to the United States Constitution, which outlawed slavery across the country.
Lincoln managed his own successful re-election campaign. He sought to heal the war-torn nation through reconciliation. On April 14, 1865, just days after the war's end at Appomattox, Lincoln was attending a play at Ford's Theatre with his wife Mary when he was assassinated by Confederate sympathizer John Wilkes Booth. His marriage had produced four sons, two of whom preceded him in death, with severe emotional impact upon them and Mary. Lincoln is remembered as the martyr hero of the United States and he is consistently ranked as one of the greatest presidents in American history."""It gives me the above mentioned error and later, all inputs even for the ones for which the model earlier was working, start giving the same error.
patrickvonplaten
commented
3 years ago Hey @gandharvsuri,
I sadly still cannot reproduce the error, I adapted the code snippet with the text you provided. Running the adapted code example does not throw an error for me -> could you maybe create a colab showing the error instead?
patrickvonplaten
commented
3 years ago Ah, one probably has to call summarise to reproduce the error. When I call gpt2.summarise(...) I'm getting some import errors (numpy is not imported). Could you please provide a complete code snippet that includes all imports to reproduce the error? :-)
gandharvsuri
commented
3 years ago Sure, You can use the following notebook, I've added you as an editor. https://colab.research.google.com/drive/17-STvWmqNROY8tlD8mfdm1grcRCFD4hy?usp=sharing
gandharvsuri
commented
3 years ago I misunderstood complete code snippet meaning, here it is.
!pip install torch
!pip install transformers
import numpy as np
import os
# tried this to resolve the error as well :)
os.environ['CUDA_LAUNCH_BLOCKING'] = "1"
import torch
import numpy as np
device = 'cuda' if torch.cuda.is_available() else 'cpu'
from transformers import GPT2LMHeadModel, GPT2Tokenizer
model = GPT2LMHeadModel.from_pretrained("gpt2")
tokenizer = GPT2Tokenizer.from_pretrained("gpt2")
text = """Dred Scott v. Sandford, 60 U.S. (19 How.) 393 (1857), often referred to as the Dred Scott decision, was a landmark decision of the US Supreme Court in which the Court held that the US Constitution was not meant to include American citizenship for black people, regardless of whether they were enslaved or free, and so the rights and privileges that the Constitution confers upon American citizens could not apply to them.The decision was made in the case of Dred Scott, an enslaved black man whose owners had taken him from Missouri, which was a slave-holding state, into Illinois and the Wisconsin Territory, which were free areas where slavery was illegal. When his owners later brought him back to Missouri, Scott sued in court for his freedom and claimed that because he had been taken into "free" U.S. territory, he had automatically been freed and was legally no longer a slave. Scott sued first in Missouri state court, which ruled that he was still a slave under its law. He then sued in US federal court, which ruled against him by deciding that it had to apply Missouri law to the case. He then appealed to the US Supreme Court.
In March 1857, the Supreme Court issued a 7–2 decision against Dred Scott. In an opinion written by Chief Justice Roger Taney, the Court ruled that black people "are not included, and were not intended to be included, under the word 'citizens' in the Constitution, and can therefore claim none of the rights and privileges which that instrument provides for and secures to citizens of the United States." Taney supported his ruling with an extended survey of American state and local laws from the time of the Constitution's drafting in 1787 that purported to show that a "perpetual and impassable barrier was intended to be erected between the white race and the one which they had reduced to slavery." Because the Court ruled that Scott was not an American citizen, he was also not a citizen of any state and, accordingly, could never establish the "diversity of citizenship" that Article III of the US Constitution requires for a US federal court to be able to exercise jurisdiction over a case. After ruling on those issues surrounding Scott, Taney continued further and struck down the entire Missouri Compromise as a limitation on slavery that exceeded the US Congress's constitutional powers.
Although Taney and several of the other justices hoped that the decision would permanently settle the slavery controversy, which was increasingly dividing the American public, the decision's effect was the complete opposite. Taney's majority opinion suited the slaveholding states, but was intensely decried in all the other states. The decision inflamed the national debate over slavery and deepened the divide that led ultimately to the Civil War. In 1865, after the Union won the Civil War, the Dred Scott ruling was voided by the Thirteenth Amendment to the US Constitution, which abolished slavery except as punishment for a crime, and the Fourteenth Amendment, which guaranteed citizenship for "all persons born or naturalized in the United States, and subject to the jurisdiction thereof."
The Supreme Court's decision has been widely denounced ever since, both for how overtly racist the decision was and its crucial role in the near destruction of the United States four years later. Bernard Schwartz said that it "stands first in any list of the worst Supreme Court decisions—Chief Justice Hughes called it the Court's greatest self-inflicted wound." Junius P. Rodriguez said that it is "universally condemned as the U.S. Supreme Court's worst decision". Historian David Thomas Konig said that it was "unquestionably, our court's worst decision ever."
Abraham Lincoln (; February 12, 1809 – April 15, 1865) was an American statesman and lawyer who served as the 16th president of the United States from 1861 until his assassination in 1865. Lincoln led the nation through the American Civil War, the country's greatest moral, cultural, constitutional, and political crisis. He succeeded in preserving the Union, abolishing slavery, bolstering the federal government, and modernizing the U.S. economy.
Lincoln was born into poverty in a log cabin and was raised on the frontier primarily in Indiana. He was self-educated and became a lawyer, Whig Party leader, Illinois state legislator, and U.S. Congressman from Illinois. In 1849, he returned to his law practice but became vexed by the opening of additional lands to slavery as a result of the Kansas–Nebraska Act. He reentered politics in 1854, becoming a leader in the new Republican Party, and he reached a national audience in the 1858 debates against Stephen Douglas. Lincoln ran for President in 1860, sweeping the North in victory. Pro-slavery elements in the South equated his success with the North's rejection of their right to practice slavery, and southern states began seceding from the union. To secure its independence, the new Confederate States fired on Fort Sumter, a U.S. fort in the South, and Lincoln called up forces to suppress the rebellion and restore the Union.
As the leader of moderate Republicans, Lincoln had to navigate a contentious array of factions with friends and opponents on both sides. War Democrats rallied a large faction of former opponents into his moderate camp, but they were countered by Radical Republicans, who demanded harsh treatment of the Southern Confederates. Anti-war Democrats (called "Copperheads") despised him, and irreconcilable pro-Confederate elements plotted his assassination. Lincoln managed the factions by exploiting their mutual enmity, by carefully distributing political patronage, and by appealing to the U.S. people. His Gettysburg Address became a historic clarion call for nationalism, republicanism, equal rights, liberty, and democracy. Lincoln scrutinized the strategy and tactics in the war effort, including the selection of generals and the naval blockade of the South's trade. He suspended habeas corpus, and he averted British intervention by defusing the Trent Affair. He engineered the end to slavery with his Emancipation Proclamation and his order that the Army protect and recruit former slaves. He also encouraged border states to outlaw slavery, and promoted the Thirteenth Amendment to the United States Constitution, which outlawed slavery across the country.
Lincoln managed his own successful re-election campaign. He sought to heal the war-torn nation through reconciliation. On April 14, 1865, just days after the war's end at Appomattox, Lincoln was attending a play at Ford's Theatre with his wife Mary when he was assassinated by Confederate sympathizer John Wilkes Booth. His marriage had produced four sons, two of whom preceded him in death, with severe emotional impact upon them and Mary. Lincoln is remembered as the martyr hero of the United States and he is consistently ranked as one of the greatest presidents in American history."""
# Another text with lesser number of tokens
text_2 = """At least nine people were killed and 25 others injured after a powerful blast outside Pakistan’s famous Sufi shrine Data Darbar in Lahore. According to initial police reports, the explosion took place close to two police vehicles near Gate 2 of Data Darbar. The nature and exact target of the explosion is yet to be ascertained. Rescue operations are underway. The blast comes as the country marks the fasting month of Ramzan.
Data Darbar, located in Pakistan’s Lahore city, is one of the oldest Sufi shrines in South Asia. Considered to be one of the most sacred places in Lahore, the shrine houses the remains of Sufi saint Abul Hassan Ali Hujwiri, commonly known as Data Ganj Baksh. He is said to have lived on the site in the 11th century and was reputed to have miraculous powers.
Data Darbar attracts a lot of visitors to its annual Urs festival. The Urs marks the death anniversary of the Sufi saint.
According to the BBC, the shrine was originally established as a simple grave next to the mosque which Hujwiri had built on the outskirts of Lahore in the 11th century. It was later expanded in the 13th century to commemorate the burial site of Hujwiri after his spiritual powers became popular.
For centuries, the shrine has seen visitors from all religions. Pakistan’s former prime minister Nawaz Sharif is also a frequent visitor to the shrine.
In 2010, two suicide bombers detonated their explosive vests outside the shrine, killing close to 50 people. More than 200 people were injured in the blasts"""
class GPT2():
def __init__(self,device,model,tokenizer):
self.name = "GPT2"
self.device = device
self.model = model.to(device)
self.tokenizer = tokenizer
# self.tokenizer.add_special_tokens({'pad_token': '[PAD]'})
self.model.resize_token_embeddings(len(self.tokenizer))
def summarise(self,text):
if text == np.nan or len(str(text)) < 10:
return np.nan
text = str(text)
text = text + " TL;DR:"
generated = self.tokenizer(text,return_tensors = 'pt', truncation=True, max_length = 1024)
context = generated['input_ids'].to(self.device)
past = None
generated = []
for i in range(50):
output = self.model(context, past_key_values=past)
past = output["past_key_values"]
logits = output["logits"]
token = torch.argmax(logits[..., -1, :])
generated += [token.tolist()]
context = token.unsqueeze(0)
summary = self.tokenizer.decode(generated)
return summary
def __str__(self):
return self.name
gpt2 = GPT2("cuda",model,tokenizer)
# This cell works fine
print(gpt2.summarise(text_2))
# Throws error
print(gpt2.summarise(text))
# The same example which was working earlier also stopped working now.
print(gpt2.summarise(text_2))@patrickvonplaten I guess I found the error. I am running a for loop to predict the next 50 tokens. After each iteration, a new token would be added to the sequence thus increasing its size by 1. Now, this new sequence would be fed to the model (the one with the new predicted token) to predict the next token. So to predict the 50 tokens, I need to set max_length = (1024-50) so that in the last iteration, it does not exceed the sequence length limit.
patrickvonplaten
commented
3 years ago Hey @gandharvsuri,
Thanks for the very nice reproducible code snippet & sorry to answer so late. The problem is indeed an out-of-index error of the position ids matrix.
Those errors are often hard to catch on GPU because the error message is quite obscure
vikas95
commented
2 years ago Hi @gandharvsuri ,
Thanks for raising this issue. I get the same error but only for specific passages. But when I get this error for one passage, the BART model throws the same error for all the following passages.
I wanted to ask if you could please share the code change that you had done for - "So to predict the 50 tokens, I need to set max_length = (1024-50) so that in the last iteration, it does not exceed the sequence length limit."
And there are several posts suggesting - model.resize_token_embeddings(len(tokenizer)) I added it after loading the model and also after each generation call. It still throws the above error. I wanted to ask if you found the above useful anywhere in fixing this error?
Thanks and looking forward for your response :D
sr5434
commented
1 year ago I get this issue when finetuning Llama on Abirate/English_Quotes
lazypool
commented
11 months ago I get this issue when using model.resize_token_embeddings(len(tokenizer)).
It seems that the change of model's embeddings size is behind the config?
Environment info
transformersversion: 4.6.1Who can help
Information
Model I am using (Bert, XLNet ...): GPT2
The problem arises when using:
The tasks I am working on is:
To reproduce
Steps to reproduce the behavior:
TL;DR:tag at the end of the sequenceMy Script
The sequences are long (>1024) and I expect the
truncation = Trueto take care of that.Error Produced
Earlier tried to truncate manually checking the length of the input text but that gave ```IndexError: index out of range in self ```` Have tried #1805 but didn't work.
@patrickvonplaten, @LysandreJik