search

rgarner / cma-tna-crawlers

Scraping old cases from TNA for CMA, no TLAs.
0 stars 3 forks source link

Fill in mergers summary automatically from title #17

Closed rgarner closed 9 years ago

rgarner commented 9 years ago

2010-2014

There is a bit of an issue with mergers though. The summaries really all need to say something along the following lines: ‘OFT closed case: Anticipated acquisition by Arcapita B.S.C.(C) of Viridian Group plc’. This is the title of the web page for the 2014-2010 cases. See for example: http://webarchive.nationalarchives.gov.uk/20140402142426/http://www.oft.gov.uk/OFTwork/mergers/decisions/2013/extra-msa

Would it be possible to set the scraper so that it scrapes this info off the page into the summary field? We can then write ‘OFT closed case:’ before it (if you can do this automatically?).

2003-2009

For the 2003-2009 pages this information is on the 2nd page you get to though, rather than the landing page for the merger, so I assumed you couldn’t scrape this into the summary field very easily? See for example: > http://webarchive.nationalarchives.gov.uk/20140402142426/http://www.oft.gov.uk/OFTwork/mergers/Mergers_Cases/2009/NBTY. We’ve been adding this text to the summary column of the spreadsheets for 2003 to 2009.

Ambiguous titles for old-style mergers cases

Some cases (although certainly not most - 25, according to an ad-hoc report) have more than one title in their subpages, and it's not easy for a machine to tell which should be the summary. I think I'll fill in the summaries from the titles in the subpages on the basis that 1225 correct ones is good, and then get the body generator to remove any with more than one markup section. Here are the 25:

http://www.oft.gov.uk/OFTwork/mergers/Mergers_Cases/2003/sibelco http://www.oft.gov.uk/OFTwork/mergers/Mergers_Cases/2003/stena http://www.oft.gov.uk/OFTwork/mergers/Mergers_Cases/2004/firstgroup2 http://www.oft.gov.uk/OFTwork/mergers/Mergers_Cases/2005/britannic http://www.oft.gov.uk/OFTwork/mergers/Mergers_Cases/2005/national http://www.oft.gov.uk/OFTwork/mergers/Mergers_Cases/2005/william-hill http://www.oft.gov.uk/OFTwork/mergers/Mergers_Cases/2006/aggregate http://www.oft.gov.uk/OFTwork/mergers/Mergers_Cases/2006/boots http://www.oft.gov.uk/OFTwork/mergers/Mergers_Cases/2006/coop2 http://www.oft.gov.uk/OFTwork/mergers/Mergers_Cases/2007/Co-op http://www.oft.gov.uk/OFTwork/mergers/Mergers_Cases/2007/Flybe http://www.oft.gov.uk/OFTwork/mergers/Mergers_Cases/2007/Inchcape http://www.oft.gov.uk/OFTwork/mergers/Mergers_Cases/2007/lloyds http://www.oft.gov.uk/OFTwork/mergers/Mergers_Cases/2007/Tesco http://www.oft.gov.uk/OFTwork/mergers/Mergers_Cases/2008/AirFrance http://www.oft.gov.uk/OFTwork/mergers/Mergers_Cases/2008/coop-somerfield http://www.oft.gov.uk/OFTwork/mergers/Mergers_Cases/2008/Diageo http://www.oft.gov.uk/OFTwork/mergers/Mergers_Cases/2008/Dunfermline http://www.oft.gov.uk/OFTwork/mergers/Mergers_Cases/2008/globalradio http://www.oft.gov.uk/OFTwork/mergers/Mergers_Cases/2008/Home http://www.oft.gov.uk/OFTwork/mergers/Mergers_Cases/2008/SRCL-Cliniserve http://www.oft.gov.uk/OFTwork/mergers/Mergers_Cases/2009/Aggregate http://www.oft.gov.uk/OFTwork/mergers/Mergers_Cases/2009/Co-operative2 http://www.oft.gov.uk/OFTwork/mergers/Mergers_Cases/2009/cooperative2 http://www.oft.gov.uk/OFTwork/mergers/Mergers_Cases/2009/sports-direct-inter