search

scrapinghub / dateparser

python parser for human readable dates
BSD 3-Clause "New" or "Revised" License
2.56k stars 467 forks source link

Float division by zero when parsing date dense text #432

Closed stuartspotlight closed 4 years ago

stuartspotlight commented 6 years ago

I tried to pass this string:-

Holidays in United States in 2018
Change Year | Change Holidays | Change Country

Date    Weekday Holiday Name    Holiday Type    Where It is Observed
Jan 1   Monday  New Year's Day  Federal Holiday  
Jan 1   Monday  New Year's Eve observed State holiday   LA, MI, WI
Jan 6   Saturday    Epiphany    Christian    
Jan 7   Sunday  Orthodox Christmas Day  Orthodox     
Jan 12  Friday  Lee Jackson Day State holiday   Virginia
Jan 13  Saturday    Stephen Foster Memorial Day Observance   
Jan 14  Sunday  Orthodox New Year   Orthodox     
Jan 15  Monday  Martin Luther King Jr. Day  Federal Holiday  
Jan 15  Monday  Robert E Lee's Birthday State holiday   Alabama, Mississippi
Jan 15  Monday  Idaho Human Rights Day  State holiday   Idaho
Jan 15  Monday  Civil Rights Day    State holiday   Arizona, New Hampshire
Jan 19  Friday  Robert E Lee's Birthday State holiday   Florida
Jan 19  Friday  Confederate Memorial Day    State holiday   Texas
Jan 29  Monday  Kansas Day  Observance   
Jan 31  Wednesday   Tu Bishvat/Tu B'Shevat  Jewish holiday   
Feb 1   Thursday    National Freedom Day    Observance   
Feb 2   Friday  Groundhog Day   Observance   
Feb 2   Friday  National Wear Red Day   Observance   
Feb 4   Sunday  Rosa Parks Day  Local observance    California, Ohio
Feb 4   Sunday  Super Bowl  Sporting Event   
Feb 12  Monday  Lincoln's Birthday  State holiday   CT, IL, MO, NY
Feb 12  Monday  Lincoln's Birthday  Local observance    Florida
Feb 13  Tuesday Maha Shivaratri Hindu Holiday    
Feb 13  Tuesday Shrove Tuesday/Mardi Gras   Local observance    Florida
Feb 13  Tuesday Shrove Tuesday/Mardi Gras   State holiday   Alabama, Louisiana
Feb 13  Tuesday Shrove Tuesday/Mardi Gras   Observance   
Feb 14  Wednesday   Ash Wednesday   Christian    
Feb 14  Wednesday   Valentine's Day Observance   
Feb 14  Wednesday   Statehood Day in Arizona    Local observance    Arizona
Feb 15  Thursday    Susan B Anthony's Birthday  Local observance    CA, FL, NY, WI
Feb 16  Friday  Chinese New Year    Observance   
Feb 16  Friday  Elizabeth Peratrovich Day   Local observance    Alaska
Feb 19  Monday  Presidents' Day Federal Holiday All except DE, GA, IA, IN, KS, KY, LA, NC, NM, RI, WI
Feb 19  Monday  Daisy Gatson Bates Day  State holiday   Arkansas
Feb 28  Wednesday   Linus Pauling Day   Local observance    Oregon
Mar 1   Thursday    Purim   Jewish holiday   
Mar 1   Thursday    St. David's Day Observance   
Mar 2   Friday  Holi    Hindu Holiday    
Mar 2   Friday  Texas Independence Day  State holiday   Texas
Mar 2   Friday  Read Across America Day Observance   
Mar 2   Friday  Employee Appreciation Day   Observance   
Mar 5   Monday  Casimir Pulaski Day State holiday   Illinois
Mar 6   Tuesday Town Meeting Day Vermont    State holiday   Vermont
Mar 11  Sunday  Daylight Saving Time starts Clock change/Daylight Saving Time    
Mar 16  Friday  Evacuation Day observed State holiday   Massachusetts
Mar 17  Saturday    St. Patrick's Day   Observance   
Mar 17  Saturday    Evacuation Day  State holiday   Massachusetts
Mar 20  Tuesday March equinox   Season   
Mar 25  Sunday  Palm Sunday Christian    
Mar 25  Sunday  Maryland Day    Local observance    Maryland
Mar 26  Monday  Maryland Day observed   Local observance    Maryland
Mar 26  Monday  Prince Jonah Kuhio Kalanianaole Day State holiday   Hawaii
Mar 26  Monday  Seward's Day    State holiday   Alaska
Mar 29  Thursday    Maundy Thursday Christian    
Mar 29  Thursday    National Vietnam War Veterans Day   Observance   
Mar 30  Friday  Good Friday State holiday   13 states
Mar 31  Saturday    Holy Saturday   Christian    
Mar 31  Saturday    Passover (first day)    Jewish holiday   
Mar 31  Saturday    César Chávez Day    State holiday   CA, CO*, TX*
Apr 1   Sunday  Easter Sunday   Observance, Christian    
Apr 2   Monday  Easter Monday   Christian    
Apr 2   Monday  Pascua Florida Day  Local observance    Florida
Apr 6   Friday  Orthodox Good Friday    Orthodox     
Apr 6   Friday  National Tartan Day Observance   
Apr 7   Saturday    Last Day of Passover    Jewish holiday   
Apr 7   Saturday    Orthodox Holy Saturday  Orthodox     
Apr 8   Sunday  Orthodox Easter Orthodox     
Apr 9   Monday  Orthodox Easter Monday  Orthodox     
Apr 10  Tuesday National Library Workers' Day   Observance   
Apr 11  Wednesday   Yom HaShoah Jewish commemoration     
Apr 13  Friday  Isra and Mi'raj Muslim   
Apr 13  Friday  Thomas Jefferson's Birthday Observance   
Apr 15  Sunday  Father Damien Day   Local observance    Hawaii
Apr 16  Monday  Emancipation Day    State holiday   District of Columbia
Apr 16  Monday  Patriot's Day   State holiday   Maine, Massachusetts
Apr 16  Monday  Boston Marathon Sporting Event   
Apr 17  Tuesday Tax Day Observance   
Apr 19  Thursday    Yom Ha'atzmaut  Jewish holiday   
Apr 21  Saturday    San Jacinto Day State holiday   Texas
Apr 22  Sunday  Oklahoma Day    Local observance    Oklahoma
Apr 23  Monday  Confederate Memorial Day    State holiday   Alabama, Florida
Apr 23  Monday  State Holiday   State holiday   Georgia
Apr 25  Wednesday   Administrative Professionals Day    Observance   
Apr 26  Thursday    Confederate Memorial Day    Local observance    Florida
Apr 26  Thursday    Take our Daughters and Sons to Work Day Observance   
Apr 27  Friday  Arbor Day   State holiday   Nebraska
Apr 30  Monday  Confederate Memorial Day    State holiday   Mississippi
May 1   Tuesday Law Day Observance   
May 1   Tuesday Loyalty Day Observance   
May 1   Tuesday Lei Day Local observance    Hawaii
May 3   Thursday    Lag BaOmer  Jewish holiday   
May 3   Thursday    National Day of Prayer  Observance   
May 4   Friday  Kent State Shootings Remembrance    Local observance    Ohio
May 4   Friday  Kentucky Oaks   Sporting Event   
May 4   Friday  Rhode Island Independence Day   Local observance    Rhode Island
May 5   Saturday    Cinco de Mayo   Observance   
May 5   Saturday    Kentucky Derby  Sporting Event   
May 5   Saturday    National Explosive Ordnance Disposal (EOD) Day  Observance   
May 6   Sunday  National Nurses Day Observance   
May 8   Tuesday Truman Day  State holiday   Missouri
May 8   Tuesday Primary Election Day West Virginia  State holiday   West Virginia
May 10  Thursday    Ascension Day   Christian    
May 10  Thursday    Confederate Memorial Day    State holiday   South Carolina
May 10  Thursday    Confederate Memorial Day    Local observance    North Carolina
May 11  Friday  Military Spouse Appreciation Day    Observance   
May 13  Sunday  Mother's Day    Observance   
May 15  Tuesday Peace Officers Memorial Day Observance   
May 16  Wednesday   Ramadan starts  Muslim   
May 18  Friday  National Defense Transportation Day Observance   
May 19  Saturday    Armed Forces Day    Observance   
May 19  Saturday    Preakness Stakes    Sporting Event   
May 20  Sunday  Pentecost   Christian    
May 20  Sunday  Shavuot Jewish holiday   
May 21  Monday  Whit Monday Christian    
May 22  Tuesday National Maritime Day   Observance   
May 22  Tuesday Harvey Milk Day Local observance    California
May 23  Wednesday   Emergency Medical Services for Children Day Observance   
May 25  Friday  National Missing Children's Day Observance   
May 27  Sunday  Trinity Sunday  Christian    
May 28  Monday  Memorial Day    Federal Holiday  
May 28  Monday  Jefferson Davis Birthday    Local observance    Mississippi
May 31  Thursday    Corpus Christi  Christian    
Jun 1   Friday  Statehood Day   Local observance    Kentucky, Tennessee
Jun 3   Sunday  Jefferson Davis Birthday    Local observance    Florida
Jun 4   Monday  Jefferson Davis Birthday    State holiday   Alabama
Jun 6   Wednesday   D-Day   Observance   
Jun 9   Saturday    Belmont Stakes  Sporting Event   
Jun 10  Sunday  Lailat al-Qadr  Muslim   
Jun 11  Monday  Kamehameha Day  State holiday   Hawaii
Jun 14  Thursday    Army Birthday   Observance   
Jun 14  Thursday    Flag Day    Observance   
Jun 15  Friday  Eid al-Fitr Muslim   
Jun 17  Sunday  Father's Day    Observance   
Jun 17  Sunday  Bunker Hill Day State holiday   Massachusetts
Jun 18  Monday  Bunker Hill Day observed    State holiday   Massachusetts
Jun 19  Tuesday Juneteenth  Local observance    All except HI, MD, MP, MT, ND, NH, SD, TX, UT
Jun 19  Tuesday Emancipation Day    State holiday   Texas
Jun 20  Wednesday   West Virginia Day   State holiday   West Virginia
Jun 20  Wednesday   American Eagle Day  Observance   
Jun 21  Thursday    June Solstice   Season   
Jul 4   Wednesday   Independence Day    Federal Holiday  
Jul 14  Saturday    Bastille Day    Observance   
Jul 22  Sunday  Tisha B'Av  Jewish holiday   
Jul 22  Sunday  Parents' Day    Observance   
Jul 24  Tuesday Pioneer Day State holiday   Utah
Jul 27  Friday  National Korean War Veterans Armistice Day  Observance   
Aug 1   Wednesday   Colorado Day    Local observance    Colorado
Aug 4   Saturday    Coast Guard Birthday    Observance   
Aug 7   Tuesday Purple Heart Day    Observance   
Aug 13  Monday  Victory Day State holiday   Rhode Island
Aug 15  Wednesday   Assumption of Mary  Christian    
Aug 16  Thursday    Bennington Battle Day   State holiday   Vermont
Aug 17  Friday  Statehood Day in Hawaii State holiday   Hawaii
Aug 19  Sunday  National Aviation Day   Observance   
Aug 21  Tuesday Senior Citizens Day Observance   
Aug 22  Wednesday   Eid al-Adha Muslim   
Aug 25  Saturday    Raksha Bandhan  Hindu Holiday    
Aug 26  Sunday  Women's Equality Day    Observance   
Aug 27  Monday  Lyndon Baines Johnson Day   State holiday   Texas
Sep 2   Sunday  Janmashtami Hindu Holiday    
Sep 3   Monday  Labor Day   Federal Holiday  
Sep 8   Saturday    Carl Garner Federal Lands Cleanup Day   Observance   
Sep 9   Sunday  National Grandparents Day   Observance   
Sep 9   Sunday  California Admission Day    Local observance    California
Sep 10  Monday  Rosh Hashana    State holiday   Texas
Sep 10  Monday  Rosh Hashana    Jewish holiday   
Sep 11  Tuesday Patriot Day Observance   
Sep 12  Wednesday   Ganesh Chaturthi    Hindu Holiday    
Sep 12  Wednesday   Muharram    Muslim   
Sep 15  Saturday    National CleanUp Day    Observance   
Sep 17  Monday  Constitution Day and Citizenship Day    Observance   
Sep 18  Tuesday Air Force Birthday  Observance   
Sep 19  Wednesday   Yom Kippur  State holiday   Texas
Sep 19  Wednesday   Yom Kippur  Jewish holiday   
Sep 21  Friday  National POW/MIA Recognition Day    Observance   
Sep 22  Saturday    Emancipation Day    Local observance    Ohio
Sep 23  Sunday  September equinox   Season   
Sep 24  Monday  First Day of Sukkot Jewish holiday   
Sep 28  Friday  Native Americans' Day   Local observance    California
Sep 30  Sunday  Last Day of Sukkot  Jewish holiday   
Sep 30  Sunday  Gold Star Mother's Day  Observance   
Oct 1   Monday  Shmini Atzeret  Jewish holiday   
Oct 1   Monday  Child Health Day    Observance   
Oct 2   Tuesday Simchat Torah   Jewish holiday   
Oct 4   Thursday    Feast of St Francis of Assisi   Christian    
Oct 7   Sunday  Chicago Marathon    Sporting Event   
Oct 8   Monday  Columbus Day    Federal Holiday All except AK, AR, CA, DE, FL, HI, MI, MN, ND, NV, OR, SD, TX, VT, WA, WY
Oct 8   Monday  Columbus Day    Local observance    Florida
Oct 8   Monday  Native Americans' Day   Local observance    South Dakota
Oct 8   Monday  Indigenous People's Day Local observance    Alaska
Oct 8   Monday  Indigenous People's Day Local observance    CA*, MN*, WA*
Oct 9   Tuesday Navaratri   Hindu Holiday    
Oct 9   Tuesday Leif Erikson Day    Observance   
Oct 13  Saturday    Navy Birthday   Observance   
Oct 15  Monday  White Cane Safety Day   Observance   
Oct 16  Tuesday Boss's Day  Observance   
Oct 18  Thursday    Dussehra    Hindu Holiday    
Oct 18  Thursday    Alaska Day  State holiday   Alaska
Oct 20  Saturday    Sweetest Day    Observance  13 states
Oct 26  Friday  Nevada Day  State holiday   Nevada
Oct 31  Wednesday   Halloween   Observance   
Nov 1   Thursday    All Saints' Day Christian    
Nov 2   Friday  All Souls' Day  Christian    
Nov 4   Sunday  Daylight Saving Time ends   Clock change/Daylight Saving Time    
Nov 4   Sunday  New York City Marathon  Sporting Event   
Nov 6   Tuesday Election Day    State holiday   14 states
Nov 6   Tuesday Diwali/Deepavali    Observance   
Nov 8   Thursday    Return Day Delaware State holiday   Delaware
Nov 10  Saturday    Marine Corps Birthday   Observance   
Nov 11  Sunday  Veterans Day    Federal Holiday  
Nov 12  Monday  Veterans Day observed   Federal Holiday  
Nov 21  Wednesday   The Prophet's Birthday  Muslim   
Nov 22  Thursday    Thanksgiving Day    Federal Holiday  
Nov 23  Friday  State Holiday   State holiday   Georgia
Nov 23  Friday  Presidents' Day State holiday   New Mexico
Nov 23  Friday  Lincoln's Birthday/Lincoln's Day    State holiday   Indiana, West Virginia
Nov 23  Friday  Day After Thanksgiving  State holiday   22 states
Nov 23  Friday  Family Day  State holiday   Nevada
Nov 23  Friday  Black Friday    Observance   
Nov 23  Friday  American Indian Heritage Day    State holiday   Maryland
Nov 26  Monday  Cyber Monday    Observance   
Dec 2   Sunday  First Sunday of Advent  Christian    
Dec 3   Monday  Chanukah/Hanukkah (first day)   Jewish holiday   
Dec 6   Thursday    St Nicholas' Day    Observance   
Dec 7   Friday  Pearl Harbor Remembrance Day    Observance   
Dec 8   Saturday    Feast of the Immaculate Conception  Christian    
Dec 10  Monday  Last Day of Chanukah    Jewish holiday   
Dec 12  Wednesday   Feast of Our Lady of Guadalupe  Observance   
Dec 13  Thursday    National Guard Birthday Observance   
Dec 17  Monday  Pan American Aviation Day   Observance   
Dec 17  Monday  Wright Brothers Day Observance   
Dec 21  Friday  December Solstice   Season   
Dec 24  Monday  Christmas Eve   State holiday   KS*, LA, MI, NC, ND, OK, SC, TX, VA*, WI
Dec 24  Monday  Christmas Eve   Observance, Christian    
Dec 24  Monday  Washington's Birthday   State holiday   Indiana
Dec 24  Monday  Washington's Birthday   State holiday   Georgia
Dec 25  Tuesday Christmas Day   Federal Holiday  
Dec 26  Wednesday   Kwanzaa (until Jan 1)   Observance   
Dec 26  Wednesday   Day After Christmas Day State holiday   KS, KY, NC, NH, SC, TX
Dec 31  Monday  New Year's Eve  Observance   
Dec 31  Monday  New Year's Eve  State holiday   LA, MI, WI

Through dateparser.search.search_dates and receved a ZeroDivisionError, the full traceback is here:-

Traceback (most recent call last):

  File "<ipython-input-2-5e5708d845bf>", line 1, in <module>
    runfile('/home/Documents/Python_programs/date_extractor/date_extractor_main.py', wdir='/home/Documents/Python_programs/date_extractor')

  File "/usr/lib/python3/dist-packages/spyderlib/widgets/externalshell/sitecustomize.py", line 699, in runfile
    execfile(filename, namespace)

  File "/usr/lib/python3/dist-packages/spyderlib/widgets/externalshell/sitecustomize.py", line 88, in execfile
    exec(compile(open(filename, 'rb').read(), filename, 'exec'), namespace)

  File "/home/Documents/Python_programs/date_extractor/date_extractor_main.py", line 301, in <module>
    matches = dateparser.search.search_dates(test_long_text)

  File "/usr/local/lib/python3.5/dist-packages/dateparser/search/__init__.py", line 33, in search_dates
    result = _search_with_detection.search_dates(text=text, languages=languages, settings=settings)

  File "/usr/local/lib/python3.5/dist-packages/dateparser/conf.py", line 81, in wrapper
    return f(*args, **kwargs)

  File "/usr/local/lib/python3.5/dist-packages/dateparser/search/search.py", line 221, in search_dates
    settings=settings)}

  File "/usr/local/lib/python3.5/dist-packages/dateparser/search/search.py", line 153, in search_parse
    original=original, translated=translated, settings=settings)

  File "/usr/local/lib/python3.5/dist-packages/dateparser/search/search.py", line 140, in parse_found_objects
    parsed_best, substrings_best = self.choose_best_split(possible_parsed, possible_substrings)

  File "/usr/local/lib/python3.5/dist-packages/dateparser/search/search.py", line 59, in choose_best_split
    rating.append([num_substrings, float(not_parsed)/float(num_substrings),

ZeroDivisionError: float division by zero

I thought this might be related to this bug:- https://github.com/scrapinghub/dateparser/pull/429

Thanks

wRAR commented 6 years ago

Hello @stuartspotlight, this indeed doesn't happen in the current master version, can you try it?