rgarner / cma-tna-crawlers

Scraping old cases from TNA for CMA, no TLAs.
0 stars 3 forks source link

Write a crawler for each completed case page #13

Closed rgarner closed 9 years ago

rgarner commented 9 years ago

Going through the current cases/year archive does not seem to be enough. There are more cases on these entry pages:

rgarner commented 9 years ago

Existence report generated from https://github.com/rgarner/cma-tna-crawlers/blob/e5f535a44aadc8d769360c818a69442f253779a3/lib/cma/oft/check_completeness_crawler.rb:

{
  "dont_exist": [
    "http://www.oft.gov.uk/OFTwork/consumer-enforcement/consumer-enforcement-completed/air-travel/",
    "http://www.oft.gov.uk/OFTwork/consumer-enforcement/consumer-enforcement-completed/carpet-flooring/",
    "http://www.oft.gov.uk/OFTwork/consumer-enforcement/consumer-enforcement-completed/furniture-carpet/",
    "http://www.oft.gov.uk/OFTwork/consumer-enforcement/consumer-enforcement-completed/hanover/",
    "http://www.oft.gov.uk/OFTwork/consumer-enforcement/consumer-enforcement-completed/hart/",
    "http://www.oft.gov.uk/OFTwork/consumer-enforcement/consumer-enforcement-completed/pegasus/",
    "http://www.oft.gov.uk/OFTwork/consumer-enforcement/consumer-enforcement-completed/quick-house-sales/",
    "http://www.oft.gov.uk/OFTwork/consumer-enforcement/consumer-enforcement-completed/university-terms/",
    "http://www.oft.gov.uk/OFTwork/consumer-enforcement/consumer-enforcement-completed/energy-efficiency/",
    "http://www.oft.gov.uk/OFTwork/markets-work/quick-house-sales/",
    "http://www.oft.gov.uk/OFTwork/markets-work/airports",
    "http://www.oft.gov.uk/OFTwork/markets-work/government-ict/",
    "http://www.oft.gov.uk/OFTwork/markets-work/pensions/"
  ]
}

... so cartels are ok, but we need to crawl a few consumer/market cases.

rgarner commented 9 years ago

Fixed in 7390e43e022bffc7861f89ba6c57af40d3360dc3