robbrad / UKBinCollectionData

UK Council Bin Collection Data Parser Outputting Bin Data as a JSON
MIT License
104 stars 73 forks source link

fix: Crawley Borough Council inconsistent formatting #743

Closed mattdy closed 1 month ago

mattdy commented 1 month ago

I haven't opened an issue since I'd already found a fix, but the Crawley Borough Council website is slightly inconsistent with the formatting when a Garden Waste bin is registered at the property:

<div class="block-title">
<div class="title">
          Rubbish and Small Electricals Collection
        </div>
<i class="fa fa-trash" aria-hidden="true"></i>
</div>

vs

<div class="block-title">
        GREENbin Garden Waste Collection 
        <i class="fa fa-trash" aria-hidden="true"></i>
</div>

As such, the search for a type fails:

(uk-bin-collection-py3.11) matt@kessel:~/projects/UKBinCollectionData/uk_bin_collection/uk_bin_collection$ python collect_data.py CrawleyBoroughCouncil https://my.crawley.gov.uk/ -s -u <REDACTED>-n <REDACTED>
Traceback (most recent call last):
  File "/home/matt/projects/UKBinCollectionData/uk_bin_collection/uk_bin_collection/collect_data.py", line 133, in <module>
    run()
  File "/home/matt/projects/UKBinCollectionData/uk_bin_collection/uk_bin_collection/collect_data.py", line 129, in run
    print(app.run())
          ^^^^^^^^^
  File "/home/matt/projects/UKBinCollectionData/uk_bin_collection/uk_bin_collection/collect_data.py", line 102, in run
    return self.client_code(
           ^^^^^^^^^^^^^^^^^
  File "/home/matt/projects/UKBinCollectionData/uk_bin_collection/uk_bin_collection/collect_data.py", line 121, in client_code
    return get_bin_data_class.template_method(address_url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/matt/projects/UKBinCollectionData/uk_bin_collection/uk_bin_collection/get_bin_data.py", line 80, in template_method
    bin_data_dict = self.parse_data(
                    ^^^^^^^^^^^^^^^^
  File "/home/matt/projects/UKBinCollectionData/uk_bin_collection/uk_bin_collection/councils/CrawleyBoroughCouncil.py", line 55, in parse_data
    "type": titles[bin_index].strip(),
            ~~~~~~^^^^^^^^^^^
IndexError: list index out of range

This PR changes the search to look for the CSS block-title class rather than the inner title which isn't present for Garden Waste. No other text is present within block-title for any waste type, so the result is identical:

(uk-bin-collection-py3.11) matt@kessel:~/projects/UKBinCollectionData/uk_bin_collection/uk_bin_collection$ python collect_data.py CrawleyBoroughCouncil https://my.crawley.gov.uk/ -s -u <REDACTED_GREEN> -n <REDACTED_GREEN>
{
    "bins": [
        {
            "type": "Rubbish and Small Electricals Collection",
            "collectionDate": "04/06/2024"
        },
        {
            "type": "Recycling and Textiles Collection",
            "collectionDate": "11/06/2024"
        },
        {
            "type": "GREENbin Garden Waste Collection",
            "collectionDate": "05/06/2024"
        }
    ]
}
(uk-bin-collection-py3.11) matt@kessel:~/projects/UKBinCollectionData/uk_bin_collection/uk_bin_collection$ python collect_data.py CrawleyBoroughCouncil https://my.crawley.gov.uk/ -s -u <REDACTED_NONE> -n <REDACTED_NONE>
{
    "bins": [
        {
            "type": "Rubbish and Small Electricals Collection",
            "collectionDate": "03/06/2024"
        },
        {
            "type": "Recycling and Textiles Collection",
            "collectionDate": "10/06/2024"
        }
    ]
}

I hope that's an acceptable fix!

codecov-commenter commented 1 month ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 96.41%. Comparing base (599f617) to head (b543506). Report is 628 commits behind head on master.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #743 +/- ## ========================================== - Coverage 98.61% 96.41% -2.21% ========================================== Files 2 3 +1 Lines 144 195 +51 ========================================== + Hits 142 188 +46 - Misses 2 7 +5 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

mattdy commented 1 month ago

Apologies, apparently I hadn't followed your standards on the commit message. This has now been updated.