facebookresearch / DrQA

Reading Wikipedia to Answer Open-Domain Questions
Other
4.48k stars 898 forks source link

build_db.py Not working when file has paragraph as text #245

Open krutikabapat opened 4 years ago

krutikabapat commented 4 years ago

Hi! When I am trying to build db from the following format: {"id": "1", "text": "The American Civil War was fought in the United States from 1861 to 1865. The result of a long-standing controversy over slavery, war broke out in April 1861, when Confederates attacked Fort Sumter in South Carolina, shortly after President Abraham Lincoln was inaugurated. The nationalists of the Union proclaimed loyalty to the U.S. Constitution. They faced secessionists of the Confederate States, who advocated for states' rights to expand slavery."} {"id": "2", "text": "Among the 34 U.S. states in February 1861, seven Southern slave states individually declared their secession from the U.S. to form the Confederate States of America, or the South. The Confederacy grew to include eleven slave states. The Confederacy was never diplomatically recognized by the United States government, nor was it recognized by any foreign country (although the United Kingdom and France granted it belligerent status). The states that remained loyal to the U.S. (including the border states where slavery was legal) were known as the Union or the North."}

I get the error JSONDecodeError: Expecting , delimiter: line 1 column 23 (char 23) But when I have data in the below below format: {"id": "1", "text": "The American Civil War was fought in the} {"id": "2", "text": "The American Civil War was fought in the} i.e when the text is in only one line and not in paragraphs, the code is working fine.

Urgent help is required. I am working on a very important project :'(

imSrbh commented 4 years ago

@krutikabapat Thanks for asking,

JSON may not read multi-line strings. Use Delimiter for each line.

{
    "singleLine": "Some singleline String",
    "multiline": ["Line one", "line Two", "Line Three"]
}

or

""" """ try this,

Hope it helps

cs145442 commented 4 years ago

@krutikabapat Thanks for asking,

JSON may not read multi-line strings. Use Delimiter for each line.

{
    "singleLine": "Some singleline String",
    "multiline": ["Line one", "line Two", "Line Three"]
}

or

""" """ try this,

Hope it helps

I hope this helps me. I'll give the kudos if this works. Thanks!