search

opencivicdata / python-legistar-scraper

Scrapes municipal data from Legistar websites
BSD 3-Clause "New" or "Revised" License
42 stars 28 forks source link

Repair eComment scraping #103

Closed hancush closed 4 years ago

hancush commented 4 years ago

Description

The row containing the eComment link in the event detail page looks like this:

<tr valign="top">
    <td valign="top" style="width: 150px">
        <span id="ctl00_ContentPlaceHolder1_lblVideoX" style="color:Navy;font-family:Tahoma;font-size:10pt;">Meeting video:</span>
    </td>
    <td id="ctl00_ContentPlaceHolder1_tdVideo" valign="top" style="width: 250px">
        <div id="ctl00_ContentPlaceHolder1_divVideo">
            <table id="ctl00_ContentPlaceHolder1_tableVideoX" border="0" cellpadding="0" cellspacing="0">
                <tr id="ctl00_ContentPlaceHolder1_trVideoX" valign="top">
                    <td id="ctl00_ContentPlaceHolder1_tdVideoX">

                        <a id="ctl00_ContentPlaceHolder1_hypVideo" class="audioDownloadNotAvailableLink" data-event-id="6188ca77-ac26-4ba7-adc6-9a7f5abc9e66" data-running-text="In&amp;nbsp;progress" class="videolink" style="color:Gray;font-family:Tahoma;font-size:10pt;">Not&nbsp;available</a>

                        <br />
                    </td>
                </tr>
            </table>

        </div>
    </td>
    <td id="ctl00_ContentPlaceHolder1_tdeComment1" style="width: 125px">
        <span id="ctl00_ContentPlaceHolder1_LabeleComment" style="color:Navy;font-family:Tahoma;font-size:10pt;">eComment:</span>
    </td>
    <td id="ctl00_ContentPlaceHolder1_tdeComment2">
        <a id="ctl00_ContentPlaceHolder1_hypeComment" class="ecomment disabledCalendarLink" link="#" data-event-id="0" data-rollover-id="3" style="color:Gray;font-family:Tahoma;font-size:10pt;display:block">Not&nbsp;available</a>
    </td>
    <td>&nbsp;
    </td>
</tr>

This PR extends the parseDetails xpath to grab elements matching ctl00_ContentPlaceHolder1_Label*, so the eComment link is scraped appropriately.