search

home-assistant / core

:house_with_garden: Open source home automation that puts local control and privacy first.
https://www.home-assistant.io
Apache License 2.0
72.18k stars 30.21k forks source link

Feedreader keeps spamming with old messages #73397

Closed r-vit closed 2 years ago

r-vit commented 2 years ago

The problem

After the update to 2022.6.5 feedreader started reporting all the old messages every time it updates the sources (every 10 minues in my case).

What version of Home Assistant Core has the issue?

2022.6.5

What was the last working version of Home Assistant Core?

2022.6.4

What type of installation are you running?

Home Assistant Supervised

Integration causing the issue

No response

Link to integration documentation on our website

No response

Diagnostics information

No response

Example YAML snippet

feedreader:
  urls:
    - https://www.youtube.com/feeds/videos.xml?channel_id=UCO8DQrSp5yEP937qNqTooOw
  scan_interval:
    minutes: 10
  max_entries: 20

Anything in the logs that might be useful for us?

No response

Additional information

No response

probot-home-assistant[bot] commented 2 years ago

feedreader documentation feedreader source (message by IssueLinks)

jeremysherriff commented 2 years ago

Same same.

I've amended my automation to give me a full dump of {{ trigger }} in an attempt to see if there's anything from a content point of view to indicate what might be triggering the repeat events, but since I edited the automation it stopped spamming me (?!)

I noticed this yesterday too: I disabled the notification automation while I was at work to stop the spam and then re-enabled when I got home, but then it (correctly) never fired again until a legit new feed entry today. Once the automation triggered, it sent a notification every 10 minutes with the same content every time (10 minutes is my configured scan_interval period). Very frustrating!

I'll post the {{ trigger }} content when I get it (presumably tomorrow, none of my feeds are very active) but I suspect it will show nothing useful - after all, the automation is subscribed to an event not a state, and it's the event that should not be firing in the first place.

jeremysherriff commented 2 years ago

I'm presuming that #73208 broke it

jeremysherriff commented 2 years ago

contents of {{ trigger }}:

{
    'id': '0',
    'idx': '0',
    'platform': 'event',
    'event': '[snipped content]',
    tags=[
        {
            'term': 'media-release',
            'scheme': 'http://www.e.govt.nz/standards/nz/2009-03-01#information-type',
            'label': None
        }
    ],
    published=2022-06-14T12:02:38+12:00,
    published_parsed=time.struct_time(tm_year=2022, tm_mon=6, tm_mday=14, tm_hour=0, tm_min=2, tm_sec=38, tm_wday=1, tm_yday=165, tm_isdst=0),
    updated=2022-06-14T12:59:48+12:00,
    updated_parsed=time.struct_time(tm_year=2022, tm_mon=6, tm_mday=14, tm_hour=0, tm_min=59, tm_sec=48, tm_wday=1, tm_yday=165, tm_isdst=0),
    authors=[
        {
            'name': 'jwilkinson'
        }
    ],
    author=jwilkinson,
    author_detail=name=jwilkinson,
    id=tag:www.health.govt.nz,2022-06-14:/node/16181,
    guidislink=False,
    feed_url=https://www.health.govt.nz/feeds/media-atom.xml>,
    'description': "event 'feedreader'"
}

I'd say some of it got mangled (based on the spurious > on the end of the feed_url= tag) so if there is a need for better output please let me know and I'll devise a way to get it.

jeremysherriff commented 2 years ago

This might be better:

this:
  entity_id: automation.email_on_feedreader_new_content
  state: 'on'
  attributes:
    last_triggered: '2022-06-14T07:17:05.261343+00:00'
    mode: parallel
    current: 1
    max: 20
    id: '1608277128632'
    friendly_name: Email on FeedReader RSS new content
  last_changed: '2022-06-14T07:04:12.006715+00:00'
  last_updated: '2022-06-14T07:17:05.261406+00:00'
  context:
    id: 01G5GH5DKDE6EP590BDXTCZ59W
    parent_id: 01G5GH5DKDRTTCR1DTFGY7PKCP
    user_id: null
trigger:
  id: '0'
  idx: '0'
  platform: event
  event:
    event_type: feedreader
    data:
      title: Allen and Clarke PCR Testing Rapid Review published
      title_detail:
        type: text/plain
        language: en
        base: https://www.health.govt.nz/feeds/media-atom.xml
        value: Allen and Clarke PCR Testing Rapid Review published
      links:
        - rel: alternate
          type: text/html
          href: >-
            https://www.health.govt.nz/news-media/media-releases/allen-and-clarke-pcr-testing-rapid-review-published
      link: >-
        https://www.health.govt.nz/news-media/media-releases/allen-and-clarke-pcr-testing-rapid-review-published
      content:
        - type: application/xhtml+xml
          language: en
          base: https://www.health.govt.nz/feeds/media-atom.xml
          value: "<div class=\"field field-name-field-news-type field-type-taxonomy-term-reference field-label-hidden\"><div class=\"field-items\"><div class=\"field-item even\"><h4>Media release</h4></div></div></div><div class=\"field field-name-field-published-date field-type-date field-label-hidden\"><div class=\"field-items\"><div class=\"field-item even\"><span class=\"date-display-single\">14 June 2022</span></div></div></div><div class=\"field field-name-body field-type-text-with-summary field-label-hidden\"><div class=\"field-items\"><div class=\"field-item even\"><p>The Ministry of Health has today published an <a href=\"https://www.health.govt.nz/publication/covid-19-pcr-testing-backlog-rapid-review\">external review into delays in PCR testing processing</a> during the early stages of the Omicron outbreak.</p>\n\n<p>The review released today comes after a media briefing on 1 March during which the Director-General of Health Dr Ashley Bloomfield outlined the delays, what was being done about them and apologised to those people affected.</p>\n\n<p>Today, Dr Bloomfield said he welcomed the review and its recommendations.</p>\n\n<p>“I’d also like to reassure all New Zealanders that all practical steps were taken to address these issues at the time, including sending more than 9,000 tests to Australia for processing, rolling out rapid antigen tests, and increasing lab capacity.”</p>\n\n<p>As of 1 March, a third of people with delayed test results had already been re-tested and the full backlog of tests were all processed by mid-March. \_</p>\n\n<p>In response to the testing delays, Dr Bloomfield also commissioned Allen and Clarke to undertake a review of the circumstances that led to the delays, including forecasting of testing capacity in the lead-up to the Omicron outbreak.</p>\n\n<p>This review has now found four areas contributed to the issues, including laboratory capacity; planning; reporting; and organisational design. It makes a number of recommendations for improvements.</p>\n\n<p>“As we have been throughout the pandemic, we are committed to learning from this experience, and work is well underway to implement the recommendations. The Ministry’s response to this report is also being published today.”</p>\n\n<p>By mid-February, Omicron caused a rapid increase in demand on PCR testing and positivity rates among those tested, resulting in a reduction in pooling of testing that impacted capacity.</p>\n\n<p>Lab staffing was affected by prolonged pressure at the time; some workers testing positive for COVID-19; limits caused by international test reagent availability; distribution and logistics issues.</p>\n\n<p>“These issues arose at a time that the health system was responding rapidly on many fronts to the highly transmissible Omicron variant and the emerging evidence about how it impacted our capacity and resources, including our labs. ”</p>\n\n<p>Since March 2020, laboratories across the motu have processed more than 7 million PCR tests to support the public health response. That’s a mammoth effort – and I want to personally thank laboratory staff across the motu for their ongoing commitment to the health of all New Zealanders. \_</p>\n\n<p>“I want to emphasise that this is not the fault of our labs – these staff have been doing phenomenal work throughout the pandemic, and similar issues have faced labs overseas during an Omicron outbreak.</p>\n\n<p>“Overall, New Zealand has done comparatively well in responding to Omicron and this is reflected in hospitalisation and mortality figures. Part of that success is down to the immense work of our lab staff.”</p>\n\n<p>The review also examined the Ministry’s role in the circumstances that led to delays and identified some processes that led to inconsistencies in reporting of testing capacity. Changes are already being made to address these gaps.</p>\n\n<p>“It’s clear from the review’s finding that, we could – and should – have done better on measuring and communicating lab capacity at that time. \_</p>\n\n<p>“When I became aware of these inconsistencies, I informed Ministers and worked with our testing team to ensure we could provide a more consistent and accurate picture of capacity that carefully considered the impact of Omicron.”</p>\n\n<p>This includes an updated Testing Plan that, alongside the latest COVID-19 modelling, will ensure we have a clearer picture of our lab capacity and enhance our ability to respond to future outbreaks. </p>\n\n<p>We have already strengthened testing expertise within the Ministry, and, alongside Health New Zealand, we are working with the laboratory network and DHBs to ensure testing continues to support New Zealand’s COVID-19 response as it evolves.</p>\n\n<p>These improvements are being embedded and built upon as responsibility for testing and supply now transitions to Health New Zealand. \_</p>\n\n<p>The Allen and Clarke PCR Testing Rapid Review can be found <a href=\"https://www.health.govt.nz/publication/covid-19-pcr-testing-backlog-rapid-review\">here</a> and our response to its recommendations are included below.</p>\n\n<p>The Ministry has also today published some additional documents relating to delay in PCR process, to provide greater transparency around its decision making at the time.</p>\n\n<ul>\n\t<li>View the <a href=\"https://www.health.govt.nz/node/16175/\">Proactive release of documents relating to PCR testing capacity</a>.</li>\n</ul>\n\n<h2>Background</h2>\n\n<p>Below are the Allen and Clarke review’s recommendations and the Ministry of Health’s response</p>\n\n<p><strong>1. Recommendation: </strong>Review the resourcing model of the COVID-19 Testing and Supply Group to address any capacity deficits and enable the Group to manage current workload and transition away from operation surge settings.<br />\n<br />\n<strong>Ministry response:</strong> The Ministry will work to strengthen capability and capacity within the Testing Team to ensure the right mix of skills is obtained to support the implementation of the Updated Testing Plan.<br />\n\_</p>\n\n<p><strong>2. Recommendation: </strong>Develop a clear Testing Plan that determines the role of PCR testing and other modalities going forward, as well as providing clarity about the roles and expectations of the laboratories.<br />\n<br />\n<strong>Ministry response:\_</strong> The Ministry has updated the Testing and COVID-19 Surveillance Plans based on ‘most likely’ COVID-19 scenarios for consideration by Cabinet. The updates were informed by modelling from COVID-19 Modelling Aotearoa and recent experiences in overseas jurisdictions.<br />\n<br />\nThe Testing team is developing service implementation plan(s) ensuring the testing requirements for the Updated Testing Plan are met. This will include testing service requirements and specifications for both collection and testing providers<br />\n\_</p>\n\n<p><strong>3. Recommendation: </strong>As part of the strategic planning (recommendation 2), address how COVID-19 testing transitions into a business-as-usual laboratory testing regime.<br />\n<br />\n<strong>Ministry response:</strong> The Ministry agrees that a new operating model is required as we transition from the current state to a business-as-usual laboratory testing regime and is developing a new Operating Model and a new Commissioning Model.\_<br />\n<br />\nThe development of the new Commissioning Model will bring clarity to:</p>\n\n<ul class=\"rteindent1\">\n\t<li>What we are commissioning and how different types of testing (ie non-COVID-19) can be integrated into the model\_</li>\n\t<li>Baseline and surge capacity requirements</li>\n\t<li>Stakeholder engagement.<br />\n\t\_</li>\n</ul>\n\n<p><strong>4. Recommendation: </strong>Determine the level of standing PCR testing infrastructure required for future variants or pathogens and where this infrastructure should be maintained as part of the strategic planning (refer to recommendations 2 and 3). In the absence of guidance from the Ministry, some laboratories have signalled they may need to reduce their PCR capacity, including mothballing equipment and reducing staff.<br />\n<br />\n<strong>Ministry response: </strong>The Updated Testing Plan and new Commissioning Model will enable the Ministry to engage with the DHB and laboratory sector around contractual arrangements to enable them to make informed decisions about the level of PCR infrastructure needed to plan for in the medium term, particularly as we enter the first winter with fewer border restrictions.<br />\n<br />\nThe assumptions from all COVID-19 service plans will inform service delivery requirements, the level of NAAT testing capacity and where it is needed.<br />\n\_</p>\n\n<p><strong>5. Recommendation: </strong>Review the approach to contracting laboratory services to facilitate greater transparency and national coordination in a pandemic. The health systems reforms and transition to new entities (Health NZ, Māori Health Authority, and the Public Health Agency), provides an opportunity to consider how laboratory testing can be better used for surveillance and public health outcomes as part of any review of laboratory services.<br />\n<br />\n<strong>Ministry response: T</strong>he Ministry is working through a procurement process to procure COVID-19 testing services with laboratory providers for 1 July. This will involve:</p>\n\n<ul class=\"rteindent1\">\n\t<li>Determining baseline and surge capacity requirements</li>\n\t<li>Revise the funding model</li>\n\t<li>Market analysis and stakeholder engagement in the process</li>\n</ul>\n\n<p>The Ministry is also developing a new contracting model that will establish Service Level Agreements for contracting laboratory services with key performance and reporting metrics. The Service Level Agreements will cover pricing, minimum specified standards, capacity, consistent performance metrics and a monitoring framework.<br />\n\_</p>\n\n<p><strong>6. Recommendation: </strong>Consider greater interoperability of information technology platforms so that the data that is generated from laboratory testing can be better used for surveillance and public health outcomes as part of any review of laboratory services.<br />\n<br />\n<strong>Ministry response: </strong>The Ministry will undertake a gap assessment of current and future end-to-end IT interoperability requirements from test order to reporting of results. This will analyse the current IT state region by region including orders/collections, testing, and reporting.<br />\n<br />\nOnce the gap assessment is completed consistent standards and reporting requirements for testing services will be incorporated into all service level agreement contracts.<br />\n<br />\nThe Ministry will also catalogue all data sources and methods of collection to map end to end data and information flows.<br />\n\_</p>\n\n<p><strong>7. Recommendation: </strong>Consider how the Ministry and Health NZ can achieve more integrated ways of operating with laboratories when planning for managing the ongoing testing requirements of COVID-19, and in planning undertaken for future pandemic readiness.<br />\n<br />\n<strong>Ministry response: </strong>The Ministry agrees and will develop a new contracting model with laboratories that includes:</p>\n\n<ul class=\"rteindent1\">\n\t<li>Service level agreements with clear performance metrics and reporting</li>\n\t<li>Defined communication pathways</li>\n\t<li>Clear roles and responsibilities.</li>\n</ul>\n\n<p>The Ministry agrees with the need to improve transparency of decision making and have timely input from laboratory stakeholders in relation to service design for the Testing response.<br />\n\_</p>\n\n<p><strong>8. Recommendation: </strong>Ensure there are clear, formal governance arrangements in place between the Ministry and/or Health NZ and laboratories that enable sufficient centralised planning and management of testing in Aotearoa New Zealand’s pandemic approach.<br />\n<br />\n<strong>Ministry response:</strong> The Ministry will review the governance arrangements with Health NZ and the laboratory sector. The Ministry intends taking a sector stewardship approach to the new governance arrangements, treating the laboratory network as a system. Clear terms of reference, roles and responsibilities and accountabilities will be established for the sector reference groups the Ministry engages with.<br />\n\_</p>\n\n<p><strong>9. Recommendation: </strong>Consider options to strengthen the Chief Testing Adviser position and shape the role so that the Ministry and/or Health NZ is able to better leverage their subject matter expertise and insights across testing planning, modelling and reporting.<br />\n<br />\n<strong>Ministry response: </strong>The Ministry will review how work priorities are identified for the Chief Testing Advisor to maximise subject matter expertise across the Science, Surveillance and Insights and Testing and Supply work programme.</p>\n</div></div></div>"
      summary: "<div class=\"field field-name-field-news-type field-type-taxonomy-term-reference field-label-hidden\"><div class=\"field-items\"><div class=\"field-item even\"><h4>Media release</h4></div></div></div><div class=\"field field-name-field-published-date field-type-date field-label-hidden\"><div class=\"field-items\"><div class=\"field-item even\"><span class=\"date-display-single\">14 June 2022</span></div></div></div><div class=\"field field-name-body field-type-text-with-summary field-label-hidden\"><div class=\"field-items\"><div class=\"field-item even\"><p>The Ministry of Health has today published an <a href=\"https://www.health.govt.nz/publication/covid-19-pcr-testing-backlog-rapid-review\">external review into delays in PCR testing processing</a> during the early stages of the Omicron outbreak.</p>\n\n<p>The review released today comes after a media briefing on 1 March during which the Director-General of Health Dr Ashley Bloomfield outlined the delays, what was being done about them and apologised to those people affected.</p>\n\n<p>Today, Dr Bloomfield said he welcomed the review and its recommendations.</p>\n\n<p>“I’d also like to reassure all New Zealanders that all practical steps were taken to address these issues at the time, including sending more than 9,000 tests to Australia for processing, rolling out rapid antigen tests, and increasing lab capacity.”</p>\n\n<p>As of 1 March, a third of people with delayed test results had already been re-tested and the full backlog of tests were all processed by mid-March. \_</p>\n\n<p>In response to the testing delays, Dr Bloomfield also commissioned Allen and Clarke to undertake a review of the circumstances that led to the delays, including forecasting of testing capacity in the lead-up to the Omicron outbreak.</p>\n\n<p>This review has now found four areas contributed to the issues, including laboratory capacity; planning; reporting; and organisational design. It makes a number of recommendations for improvements.</p>\n\n<p>“As we have been throughout the pandemic, we are committed to learning from this experience, and work is well underway to implement the recommendations. The Ministry’s response to this report is also being published today.”</p>\n\n<p>By mid-February, Omicron caused a rapid increase in demand on PCR testing and positivity rates among those tested, resulting in a reduction in pooling of testing that impacted capacity.</p>\n\n<p>Lab staffing was affected by prolonged pressure at the time; some workers testing positive for COVID-19; limits caused by international test reagent availability; distribution and logistics issues.</p>\n\n<p>“These issues arose at a time that the health system was responding rapidly on many fronts to the highly transmissible Omicron variant and the emerging evidence about how it impacted our capacity and resources, including our labs. ”</p>\n\n<p>Since March 2020, laboratories across the motu have processed more than 7 million PCR tests to support the public health response. That’s a mammoth effort – and I want to personally thank laboratory staff across the motu for their ongoing commitment to the health of all New Zealanders. \_</p>\n\n<p>“I want to emphasise that this is not the fault of our labs – these staff have been doing phenomenal work throughout the pandemic, and similar issues have faced labs overseas during an Omicron outbreak.</p>\n\n<p>“Overall, New Zealand has done comparatively well in responding to Omicron and this is reflected in hospitalisation and mortality figures. Part of that success is down to the immense work of our lab staff.”</p>\n\n<p>The review also examined the Ministry’s role in the circumstances that led to delays and identified some processes that led to inconsistencies in reporting of testing capacity. Changes are already being made to address these gaps.</p>\n\n<p>“It’s clear from the review’s finding that, we could – and should – have done better on measuring and communicating lab capacity at that time. \_</p>\n\n<p>“When I became aware of these inconsistencies, I informed Ministers and worked with our testing team to ensure we could provide a more consistent and accurate picture of capacity that carefully considered the impact of Omicron.”</p>\n\n<p>This includes an updated Testing Plan that, alongside the latest COVID-19 modelling, will ensure we have a clearer picture of our lab capacity and enhance our ability to respond to future outbreaks. </p>\n\n<p>We have already strengthened testing expertise within the Ministry, and, alongside Health New Zealand, we are working with the laboratory network and DHBs to ensure testing continues to support New Zealand’s COVID-19 response as it evolves.</p>\n\n<p>These improvements are being embedded and built upon as responsibility for testing and supply now transitions to Health New Zealand. \_</p>\n\n<p>The Allen and Clarke PCR Testing Rapid Review can be found <a href=\"https://www.health.govt.nz/publication/covid-19-pcr-testing-backlog-rapid-review\">here</a> and our response to its recommendations are included below.</p>\n\n<p>The Ministry has also today published some additional documents relating to delay in PCR process, to provide greater transparency around its decision making at the time.</p>\n\n<ul>\n\t<li>View the <a href=\"https://www.health.govt.nz/node/16175/\">Proactive release of documents relating to PCR testing capacity</a>.</li>\n</ul>\n\n<h2>Background</h2>\n\n<p>Below are the Allen and Clarke review’s recommendations and the Ministry of Health’s response</p>\n\n<p><strong>1. Recommendation: </strong>Review the resourcing model of the COVID-19 Testing and Supply Group to address any capacity deficits and enable the Group to manage current workload and transition away from operation surge settings.<br />\n<br />\n<strong>Ministry response:</strong> The Ministry will work to strengthen capability and capacity within the Testing Team to ensure the right mix of skills is obtained to support the implementation of the Updated Testing Plan.<br />\n\_</p>\n\n<p><strong>2. Recommendation: </strong>Develop a clear Testing Plan that determines the role of PCR testing and other modalities going forward, as well as providing clarity about the roles and expectations of the laboratories.<br />\n<br />\n<strong>Ministry response:\_</strong> The Ministry has updated the Testing and COVID-19 Surveillance Plans based on ‘most likely’ COVID-19 scenarios for consideration by Cabinet. The updates were informed by modelling from COVID-19 Modelling Aotearoa and recent experiences in overseas jurisdictions.<br />\n<br />\nThe Testing team is developing service implementation plan(s) ensuring the testing requirements for the Updated Testing Plan are met. This will include testing service requirements and specifications for both collection and testing providers<br />\n\_</p>\n\n<p><strong>3. Recommendation: </strong>As part of the strategic planning (recommendation 2), address how COVID-19 testing transitions into a business-as-usual laboratory testing regime.<br />\n<br />\n<strong>Ministry response:</strong> The Ministry agrees that a new operating model is required as we transition from the current state to a business-as-usual laboratory testing regime and is developing a new Operating Model and a new Commissioning Model.\_<br />\n<br />\nThe development of the new Commissioning Model will bring clarity to:</p>\n\n<ul class=\"rteindent1\">\n\t<li>What we are commissioning and how different types of testing (ie non-COVID-19) can be integrated into the model\_</li>\n\t<li>Baseline and surge capacity requirements</li>\n\t<li>Stakeholder engagement.<br />\n\t\_</li>\n</ul>\n\n<p><strong>4. Recommendation: </strong>Determine the level of standing PCR testing infrastructure required for future variants or pathogens and where this infrastructure should be maintained as part of the strategic planning (refer to recommendations 2 and 3). In the absence of guidance from the Ministry, some laboratories have signalled they may need to reduce their PCR capacity, including mothballing equipment and reducing staff.<br />\n<br />\n<strong>Ministry response: </strong>The Updated Testing Plan and new Commissioning Model will enable the Ministry to engage with the DHB and laboratory sector around contractual arrangements to enable them to make informed decisions about the level of PCR infrastructure needed to plan for in the medium term, particularly as we enter the first winter with fewer border restrictions.<br />\n<br />\nThe assumptions from all COVID-19 service plans will inform service delivery requirements, the level of NAAT testing capacity and where it is needed.<br />\n\_</p>\n\n<p><strong>5. Recommendation: </strong>Review the approach to contracting laboratory services to facilitate greater transparency and national coordination in a pandemic. The health systems reforms and transition to new entities (Health NZ, Māori Health Authority, and the Public Health Agency), provides an opportunity to consider how laboratory testing can be better used for surveillance and public health outcomes as part of any review of laboratory services.<br />\n<br />\n<strong>Ministry response: T</strong>he Ministry is working through a procurement process to procure COVID-19 testing services with laboratory providers for 1 July. This will involve:</p>\n\n<ul class=\"rteindent1\">\n\t<li>Determining baseline and surge capacity requirements</li>\n\t<li>Revise the funding model</li>\n\t<li>Market analysis and stakeholder engagement in the process</li>\n</ul>\n\n<p>The Ministry is also developing a new contracting model that will establish Service Level Agreements for contracting laboratory services with key performance and reporting metrics. The Service Level Agreements will cover pricing, minimum specified standards, capacity, consistent performance metrics and a monitoring framework.<br />\n\_</p>\n\n<p><strong>6. Recommendation: </strong>Consider greater interoperability of information technology platforms so that the data that is generated from laboratory testing can be better used for surveillance and public health outcomes as part of any review of laboratory services.<br />\n<br />\n<strong>Ministry response: </strong>The Ministry will undertake a gap assessment of current and future end-to-end IT interoperability requirements from test order to reporting of results. This will analyse the current IT state region by region including orders/collections, testing, and reporting.<br />\n<br />\nOnce the gap assessment is completed consistent standards and reporting requirements for testing services will be incorporated into all service level agreement contracts.<br />\n<br />\nThe Ministry will also catalogue all data sources and methods of collection to map end to end data and information flows.<br />\n\_</p>\n\n<p><strong>7. Recommendation: </strong>Consider how the Ministry and Health NZ can achieve more integrated ways of operating with laboratories when planning for managing the ongoing testing requirements of COVID-19, and in planning undertaken for future pandemic readiness.<br />\n<br />\n<strong>Ministry response: </strong>The Ministry agrees and will develop a new contracting model with laboratories that includes:</p>\n\n<ul class=\"rteindent1\">\n\t<li>Service level agreements with clear performance metrics and reporting</li>\n\t<li>Defined communication pathways</li>\n\t<li>Clear roles and responsibilities.</li>\n</ul>\n\n<p>The Ministry agrees with the need to improve transparency of decision making and have timely input from laboratory stakeholders in relation to service design for the Testing response.<br />\n\_</p>\n\n<p><strong>8. Recommendation: </strong>Ensure there are clear, formal governance arrangements in place between the Ministry and/or Health NZ and laboratories that enable sufficient centralised planning and management of testing in Aotearoa New Zealand’s pandemic approach.<br />\n<br />\n<strong>Ministry response:</strong> The Ministry will review the governance arrangements with Health NZ and the laboratory sector. The Ministry intends taking a sector stewardship approach to the new governance arrangements, treating the laboratory network as a system. Clear terms of reference, roles and responsibilities and accountabilities will be established for the sector reference groups the Ministry engages with.<br />\n\_</p>\n\n<p><strong>9. Recommendation: </strong>Consider options to strengthen the Chief Testing Adviser position and shape the role so that the Ministry and/or Health NZ is able to better leverage their subject matter expertise and insights across testing planning, modelling and reporting.<br />\n<br />\n<strong>Ministry response: </strong>The Ministry will review how work priorities are identified for the Chief Testing Advisor to maximise subject matter expertise across the Science, Surveillance and Insights and Testing and Supply work programme.</p>\n</div></div></div>"
      tags:
        - term: media-release
          scheme: http://www.e.govt.nz/standards/nz/2009-03-01#information-type
          label: null
      published: '2022-06-14T12:02:38+12:00'
      published_parsed:
        - 2022
        - 6
        - 14
        - 0
        - 2
        - 38
        - 1
        - 165
        - 0
      updated: '2022-06-14T12:59:48+12:00'
      updated_parsed:
        - 2022
        - 6
        - 14
        - 0
        - 59
        - 48
        - 1
        - 165
        - 0
      authors:
        - name: jwilkinson
      author: jwilkinson
      author_detail:
        name: jwilkinson
      id: tag:www.health.govt.nz,2022-06-14:/node/16181
      guidislink: false
      feed_url: https://www.health.govt.nz/feeds/media-atom.xml
    origin: LOCAL
    time_fired: '2022-06-14T07:17:05.269810+00:00'
    context:
      id: 01G5GH5DKN053ETKJKND73Q6AR
      parent_id: null
      user_id: null
  description: event 'feedreader'
r-vit commented 2 years ago

I use feedreader object in Node-Red and send URLs of the new articles to my messenger. Since the update my messenger gets flooded with the same links every 10 minutes. Very frustrating.

Mariusthvdb commented 2 years ago

using a config like OP:

feedreader:
  urls:
    - https://alerts.home-assistant.io/feed.xml
    - https://www.home-assistant.io/atom.xml
    - https://hasspodcast.io/feed/podcast
    - https://www.gdacs.org/xml/rss.xml
  scan_interval:
    minutes: 15
  max_entries: 5

Feedreader behaves as it always had in my config. Wondering how you trigger the notifications (not using NOdered myself, but an automation based on the feedreader event:

automation:

  - alias: Notify Rss feed updated
    id: notify_rss_feed_updated
    mode: queued
    trigger:
      platform: event
      event_type: feedreader
    condition:
      - >
        {{trigger.event.data.title is not none}}
      - >
        {{'unknown' not in trigger.event.data.title}}
    action:
      service: persistent_notification.create
      data:
        title: >
          {{trigger.event.data.title}}
       message: etc etc
       notification_id: etc etc

has never been different, and currently running 2022.7.0.dev20220614

jeremysherriff commented 2 years ago

Using config similar to OP;

feedreader:
  urls:
    - https://alerts.home-assistant.io/feed.xml
    - https://www.health.govt.nz/feeds/media-atom.xml
    - https://www.civildefence.govt.nz/resources/rss
  scan_interval:
    minutes: 10
  max_entries: 3
alias: Email on FeedReader RSS new content
description: ''
trigger:
  - platform: event
    event_type: feedreader
condition: []
action:
  - service: notify.email_alerts
    data:
      title: RSS feedreader
      message: ''
      data:
        html: <p>{{ trigger.event.data.title }}</p><p>{{ trigger.event.data.link }} via {{ trigger.event.data.feed_url }}</p>
    enabled: false
mode: queued
max: 6
jeremysherriff commented 2 years ago

No sign of a fix in 2022.6.7

r-vit commented 2 years ago

I confirm that too. I think we need to inform the author (@d0nni3q84 I guess ) of original patch #73208 which supposedly broke the feedreader

jeremysherriff commented 2 years ago

I've just commented on the HA blog post for 2022.6 asking if others have the issue, I suspect it's not very widespread. I know for mine it's only one feed out of three, there's a chance that with such little uptake of this integration (0.18%) and only some feeds affected that there's very few people impacted at all.

jeremysherriff commented 2 years ago

I've just been reading through the commit, there's some debug output included in the code so we should enable debug of the component and see what the logs say about the published date vs the updated date.

I'll do that now and post logs later, assuming they show something.

jeremysherriff commented 2 years ago

Cutting the log file output up a bit for readability reasons (the feed contents are included in the debug output), and filtering out the two other feeds that are not causing issues. My scan_interval is at 7 minutes currently. Here's the updates for the 4pm hour so far this afternoon, that resulted in 3 alerts (16:22, 16:29, 16:36):

➜  ~ grep feedreader /config/home-assistant.log | grep "06-23 16:" | grep -vE "alerts.home-assistant.io|civildefence.govt.nz" | grep -v "Entry {"
2022-06-23 16:01:06 INFO (SyncWorker_1) [homeassistant.components.feedreader] Fetching new data from feed https://www.health.govt.nz/feeds/media-atom.xml
2022-06-23 16:01:06 DEBUG (SyncWorker_1) [homeassistant.components.feedreader] No new entries to be published in feed https://www.health.govt.nz/feeds/media-atom.xml
2022-06-23 16:01:06 INFO (SyncWorker_1) [homeassistant.components.feedreader] Fetch from feed https://www.health.govt.nz/feeds/media-atom.xml completed
2022-06-23 16:08:06 INFO (SyncWorker_13) [homeassistant.components.feedreader] Fetching new data from feed https://www.health.govt.nz/feeds/media-atom.xml
2022-06-23 16:08:06 DEBUG (SyncWorker_13) [homeassistant.components.feedreader] No new entries to be published in feed https://www.health.govt.nz/feeds/media-atom.xml
2022-06-23 16:08:06 INFO (SyncWorker_13) [homeassistant.components.feedreader] Fetch from feed https://www.health.govt.nz/feeds/media-atom.xml completed
2022-06-23 16:15:06 INFO (SyncWorker_5) [homeassistant.components.feedreader] Fetching new data from feed https://www.health.govt.nz/feeds/media-atom.xml
2022-06-23 16:15:06 DEBUG (SyncWorker_5) [homeassistant.components.feedreader] 10 entri(es) available in feed https://www.health.govt.nz/feeds/media-atom.xml
2022-06-23 16:15:06 DEBUG (SyncWorker_5) [homeassistant.components.feedreader] Processing only the first 3 entries in feed https://www.health.govt.nz/feeds/media-atom.xml
2022-06-23 16:15:06 DEBUG (SyncWorker_5) [homeassistant.components.feedreader] Fetching data from file /config/feedreader.pickle
2022-06-23 16:15:06 DEBUG (SyncWorker_5) [homeassistant.components.feedreader] Overwriting feed https://www.health.govt.nz/feeds/media-atom.xml timestamp in storage file /config/feedreader.pickle
2022-06-23 16:15:06 INFO (SyncWorker_5) [homeassistant.components.feedreader] Fetch from feed https://www.health.govt.nz/feeds/media-atom.xml completed
2022-06-23 16:22:06 INFO (SyncWorker_1) [homeassistant.components.feedreader] Fetching new data from feed https://www.health.govt.nz/feeds/media-atom.xml
2022-06-23 16:22:06 DEBUG (SyncWorker_1) [homeassistant.components.feedreader] 10 entri(es) available in feed https://www.health.govt.nz/feeds/media-atom.xml
2022-06-23 16:22:06 DEBUG (SyncWorker_1) [homeassistant.components.feedreader] Processing only the first 3 entries in feed https://www.health.govt.nz/feeds/media-atom.xml
2022-06-23 16:22:06 DEBUG (SyncWorker_1) [homeassistant.components.feedreader] Fetching data from file /config/feedreader.pickle
2022-06-23 16:22:06 DEBUG (SyncWorker_1) [homeassistant.components.feedreader] Overwriting feed https://www.health.govt.nz/feeds/media-atom.xml timestamp in storage file /config/feedreader.pickle
2022-06-23 16:22:06 INFO (SyncWorker_1) [homeassistant.components.feedreader] Fetch from feed https://www.health.govt.nz/feeds/media-atom.xml completed
2022-06-23 16:29:06 INFO (SyncWorker_2) [homeassistant.components.feedreader] Fetching new data from feed https://www.health.govt.nz/feeds/media-atom.xml
2022-06-23 16:29:06 DEBUG (SyncWorker_2) [homeassistant.components.feedreader] 10 entri(es) available in feed https://www.health.govt.nz/feeds/media-atom.xml
2022-06-23 16:29:06 DEBUG (SyncWorker_2) [homeassistant.components.feedreader] Processing only the first 3 entries in feed https://www.health.govt.nz/feeds/media-atom.xml
2022-06-23 16:29:06 DEBUG (SyncWorker_2) [homeassistant.components.feedreader] Fetching data from file /config/feedreader.pickle
2022-06-23 16:29:06 DEBUG (SyncWorker_2) [homeassistant.components.feedreader] Overwriting feed https://www.health.govt.nz/feeds/media-atom.xml timestamp in storage file /config/feedreader.pickle
2022-06-23 16:29:06 INFO (SyncWorker_2) [homeassistant.components.feedreader] Fetch from feed https://www.health.govt.nz/feeds/media-atom.xml completed
2022-06-23 16:36:06 INFO (SyncWorker_12) [homeassistant.components.feedreader] Fetching new data from feed https://www.health.govt.nz/feeds/media-atom.xml
2022-06-23 16:36:06 DEBUG (SyncWorker_12) [homeassistant.components.feedreader] 10 entri(es) available in feed https://www.health.govt.nz/feeds/media-atom.xml
2022-06-23 16:36:06 DEBUG (SyncWorker_12) [homeassistant.components.feedreader] Processing only the first 3 entries in feed https://www.health.govt.nz/feeds/media-atom.xml
2022-06-23 16:36:06 DEBUG (SyncWorker_12) [homeassistant.components.feedreader] Fetching data from file /config/feedreader.pickle
2022-06-23 16:36:06 DEBUG (SyncWorker_12) [homeassistant.components.feedreader] Overwriting feed https://www.health.govt.nz/feeds/media-atom.xml timestamp in storage file /config/feedreader.pickle
2022-06-23 16:36:06 INFO (SyncWorker_12) [homeassistant.components.feedreader] Fetch from feed https://www.health.govt.nz/feeds/media-atom.xml completed
➜

Here's the actual (snipped) contents of those three triggered events:

2022-06-23 16:22:06 DEBUG (SyncWorker_1) [homeassistant.components.feedreader] Entry {'title': 'Māori-led research team to undertake real-time research on key Māori health policy', 'title_detail': {'type': 'text/plain', 'language': 'en', 'base': 'https://www.health.govt.nz/feeds/media-atom.xml', 'value': 'Māori-led research team to undertake real-time research on key Māori health policy'}, 'links': [{'rel': 'alternate', 'type': 'text/html', 'href': 'https://www.health.govt.nz/news-media/news-items/maori-led-research-team-undertake-real-time-research-key-maori-health-policy'}], 'link': 'https://www.health.govt.nz/news-media/news-items/maori-led-research-team-undertake-real-time-research-key-maori-health-policy', 'content': [{..snipped..}], 'summary': '..snipped..', 'tags': [{'term': 'media-release', 'scheme': 'http://www.e.govt.nz/standards/nz/2009-03-01#information-type', 'label': None}], 'published': '2022-06-23T11:32:32+12:00', 'published_parsed': time.struct_time(tm_year=2022, tm_mon=6, tm_mday=22, tm_hour=23, tm_min=32, tm_sec=32, tm_wday=2, tm_yday=173, tm_isdst=0), 'updated': '2022-06-23T11:33:59+12:00', 'updated_parsed': time.struct_time(tm_year=2022, tm_mon=6, tm_mday=22, tm_hour=23, tm_min=33, tm_sec=59, tm_wday=2, tm_yday=173, tm_isdst=0), 'authors': [{'name': 'cara'}], 'author': 'cara', 'author_detail': {'name': 'cara'}, 'id': 'tag:www.health.govt.nz,2022-06-23:/node/16288', 'guidislink': False} already processed
2022-06-23 16:22:06 DEBUG (SyncWorker_1) [homeassistant.components.feedreader] Entry {'title': 'Join the fight against winter illness and wear a mask', 'title_detail': {'type': 'text/plain', 'language': 'en', 'base': 'https://www.health.govt.nz/feeds/media-atom.xml', 'value': 'Join the fight against winter illness and wear a mask'}, 'links': [{'rel': 'alternate', 'type': 'text/html', 'href': 'https://www.health.govt.nz/news-media/news-items/join-fight-against-winter-illness-and-wear-mask'}], 'link': 'https://www.health.govt.nz/news-media/news-items/join-fight-against-winter-illness-and-wear-mask', 'content': [{..snipped..}], 'summary': '..snipped..', 'tags': [{'term': 'media-release', 'scheme': 'http://www.e.govt.nz/standards/nz/2009-03-01#information-type', 'label': None}], 'published': '2022-06-23T10:24:25+12:00', 'published_parsed': time.struct_time(tm_year=2022, tm_mon=6, tm_mday=22, tm_hour=22, tm_min=24, tm_sec=25, tm_wday=2, tm_yday=173, tm_isdst=0), 'updated': '2022-06-23T10:25:20+12:00', 'updated_parsed': time.struct_time(tm_year=2022, tm_mon=6, tm_mday=22, tm_hour=22, tm_min=25, tm_sec=20, tm_wday=2, tm_yday=173, tm_isdst=0), 'authors': [{'name': 'cara'}], 'author': 'cara', 'author_detail': {'name': 'cara'}, 'id': 'tag:www.health.govt.nz,2022-06-23:/node/16287', 'guidislink': False} already processed
2022-06-23 16:29:06 DEBUG (SyncWorker_2) [homeassistant.components.feedreader] Entry {'title': 'Māori-led research team to undertake real-time research on key Māori health policy', 'title_detail': {'type': 'text/plain', 'language': 'en', 'base': 'https://www.health.govt.nz/feeds/media-atom.xml', 'value': 'Māori-led research team to undertake real-time research on key Māori health policy'}, 'links': [{'rel': 'alternate', 'type': 'text/html', 'href': 'https://www.health.govt.nz/news-media/news-items/maori-led-research-team-undertake-real-time-research-key-maori-health-policy'}], 'link': 'https://www.health.govt.nz/news-media/news-items/maori-led-research-team-undertake-real-time-research-key-maori-health-policy', 'content': [{..snipped..}], 'summary': '..snipped..', 'tags': [{'term': 'media-release', 'scheme': 'http://www.e.govt.nz/standards/nz/2009-03-01#information-type', 'label': None}], 'published': '2022-06-23T11:32:32+12:00', 'published_parsed': time.struct_time(tm_year=2022, tm_mon=6, tm_mday=22, tm_hour=23, tm_min=32, tm_sec=32, tm_wday=2, tm_yday=173, tm_isdst=0), 'updated': '2022-06-23T11:33:59+12:00', 'updated_parsed': time.struct_time(tm_year=2022, tm_mon=6, tm_mday=22, tm_hour=23, tm_min=33, tm_sec=59, tm_wday=2, tm_yday=173, tm_isdst=0), 'authors': [{'name': 'cara'}], 'author': 'cara', 'author_detail': {'name': 'cara'}, 'id': 'tag:www.health.govt.nz,2022-06-23:/node/16288', 'guidislink': False} already processed
2022-06-23 16:29:06 DEBUG (SyncWorker_2) [homeassistant.components.feedreader] Entry {'title': 'Join the fight against winter illness and wear a mask', 'title_detail': {'type': 'text/plain', 'language': 'en', 'base': 'https://www.health.govt.nz/feeds/media-atom.xml', 'value': 'Join the fight against winter illness and wear a mask'}, 'links': [{'rel': 'alternate', 'type': 'text/html', 'href': 'https://www.health.govt.nz/news-media/news-items/join-fight-against-winter-illness-and-wear-mask'}], 'link': 'https://www.health.govt.nz/news-media/news-items/join-fight-against-winter-illness-and-wear-mask', 'content': [{..snipped..}], 'summary': '..snipped..', 'tags': [{'term': 'media-release', 'scheme': 'http://www.e.govt.nz/standards/nz/2009-03-01#information-type', 'label': None}], 'published': '2022-06-23T10:24:25+12:00', 'published_parsed': time.struct_time(tm_year=2022, tm_mon=6, tm_mday=22, tm_hour=22, tm_min=24, tm_sec=25, tm_wday=2, tm_yday=173, tm_isdst=0), 'updated': '2022-06-23T10:25:20+12:00', 'updated_parsed': time.struct_time(tm_year=2022, tm_mon=6, tm_mday=22, tm_hour=22, tm_min=25, tm_sec=20, tm_wday=2, tm_yday=173, tm_isdst=0), 'authors': [{'name': 'cara'}], 'author': 'cara', 'author_detail': {'name': 'cara'}, 'id': 'tag:www.health.govt.nz,2022-06-23:/node/16287', 'guidislink': False} already processed
2022-06-23 16:36:06 DEBUG (SyncWorker_12) [homeassistant.components.feedreader] Entry {'title': 'Māori-led research team to undertake real-time research on key Māori health policy', 'title_detail': {'type': 'text/plain', 'language': 'en', 'base': 'https://www.health.govt.nz/feeds/media-atom.xml', 'value': 'Māori-led research team to undertake real-time research on key Māori health policy'}, 'links': [{'rel': 'alternate', 'type': 'text/html', 'href': 'https://www.health.govt.nz/news-media/news-items/maori-led-research-team-undertake-real-time-research-key-maori-health-policy'}], 'link': 'https://www.health.govt.nz/news-media/news-items/maori-led-research-team-undertake-real-time-research-key-maori-health-policy', 'content': [{..snipped..}], 'summary': '..snipped..', 'tags': [{'term': 'media-release', 'scheme': 'http://www.e.govt.nz/standards/nz/2009-03-01#information-type', 'label': None}], 'published': '2022-06-23T11:32:32+12:00', 'published_parsed': time.struct_time(tm_year=2022, tm_mon=6, tm_mday=22, tm_hour=23, tm_min=32, tm_sec=32, tm_wday=2, tm_yday=173, tm_isdst=0), 'updated': '2022-06-23T11:33:59+12:00', 'updated_parsed': time.struct_time(tm_year=2022, tm_mon=6, tm_mday=22, tm_hour=23, tm_min=33, tm_sec=59, tm_wday=2, tm_yday=173, tm_isdst=0), 'authors': [{'name': 'cara'}], 'author': 'cara', 'author_detail': {'name': 'cara'}, 'id': 'tag:www.health.govt.nz,2022-06-23:/node/16288', 'guidislink': False} already processed
2022-06-23 16:36:06 DEBUG (SyncWorker_12) [homeassistant.components.feedreader] Entry {'title': 'Join the fight against winter illness and wear a mask', 'title_detail': {'type': 'text/plain', 'language': 'en', 'base': 'https://www.health.govt.nz/feeds/media-atom.xml', 'value': 'Join the fight against winter illness and wear a mask'}, 'links': [{'rel': 'alternate', 'type': 'text/html', 'href': 'https://www.health.govt.nz/news-media/news-items/join-fight-against-winter-illness-and-wear-mask'}], 'link': 'https://www.health.govt.nz/news-media/news-items/join-fight-against-winter-illness-and-wear-mask', 'content': [{..snipped..}], 'summary': '..snipped..', 'tags': [{'term': 'media-release', 'scheme': 'http://www.e.govt.nz/standards/nz/2009-03-01#information-type', 'label': None}], 'published': '2022-06-23T10:24:25+12:00', 'published_parsed': time.struct_time(tm_year=2022, tm_mon=6, tm_mday=22, tm_hour=22, tm_min=24, tm_sec=25, tm_wday=2, tm_yday=173, tm_isdst=0), 'updated': '2022-06-23T10:25:20+12:00', 'updated_parsed': time.struct_time(tm_year=2022, tm_mon=6, tm_mday=22, tm_hour=22, tm_min=25, tm_sec=20, tm_wday=2, tm_yday=173, tm_isdst=0), 'authors': [{'name': 'cara'}], 'author': 'cara', 'author_detail': {'name': 'cara'}, 'id': 'tag:www.health.govt.nz,2022-06-23:/node/16287', 'guidislink': False} already processed

To my eye, there is no difference between the 3 runs and there should have been no event raised and no updating of the pickle file.

The thing that puzzles me is Processing only the first 3 entries in feed https://www.health.govt.nz/feeds/media-atom.xml yet there is only two lines of feed provided in the next Entry {...} lines, both of which then state already processed. Where's the 3rd entry? Maybe it's simply not logged in the debug output, but then what level of logging do I enable to see the new entry?

likeablob commented 2 years ago

@r-vit @jeremysherriff Hi, I'm in the same boat.

Based on my investigation, feedreader uses a published date to keep track of the last-notified feed entry even if the entry has both published and updated date. https://github.com/home-assistant/core/blob/cf5a11a1e7c7ef49a439753f3bb708d10b2c22b9/homeassistant/components/feedreader/__init__.py#L159-L170

And thus it detects false new entries by comparing a published date of the last-notified entries vs updated date of updated entries. (e.g. It detects the last-notified entry itself repeatedly when the entry got updated.)

FYI, I've forked the above PR as a custom integration. It can be a temporary workaround until the PR gets merged. https://github.com/likeablob/hass-feedreader-tempfix

jeremysherriff commented 2 years ago

Nice work @likeablob

I've been keeping an eye on the HA release notes for 2022.9 to see if it's being included but not so far.

jeremysherriff commented 2 years ago

Excellent, it's been accepted into 2022.9.0