search

feediron / ttrss_plugin-feediron

Evolution of ttrss_plugin-af_feedmod
https://discourse.tt-rss.org/t/plugin-update-feediron-v1-2-0/2018
MIT License
204 stars 34 forks source link

TypeError with xpath and/or tags filter #193

Open benfishbus opened 12 months ago

benfishbus commented 12 months ago

Expected Behavior

Extract tags from article page

Current Behavior

Error

Steps to Reproduce

I finally got around to updating to the latest feediron, and found the following config no longer works. 

    "rarefilm.net": {
        "type": "xpath",
        "xpath": "div[@class='entry']",
        "tags": {
            "type": "xpath",
            "replace-tags": true,
            "xpath": [
                {
                    "xpath": "p[@class='post-tag']\/a",
                    "index": "all"
                }
            ],
            "join_element": ",",
            "split": ","
        }
    }
Failure Logs ``` E_ERROR (1) plugins.local/feediron/filters/fi_mod_xpath/init.php:28 Uncaught TypeError: DOMNodeList::item(): Argument #1 ($index) must be of type int, string given in /var/www/html/tt-rss/plugins.local/feediron/filters/fi_mod_xpath/init.php:28 Stack trace: #0 /var/www/html/tt-rss/plugins.local/feediron/filters/fi_mod_xpath/init.php(28): DOMNodeList->item() #1 /var/www/html/tt-rss/plugins.local/feediron/filters/fi_mod_tags_xpath/init.php(20): fi_mod_xpath->perform_filter() #2 /var/www/html/tt-rss/plugins.local/feediron/init.php(372): fi_mod_tags_xpath->get_tags() #3 /var/www/html/tt-rss/plugins.local/feediron/init.php(224): Feediron->getArticleTags() #4 /var/www/html/tt-rss/plugins.local/feediron/init.php(817): Feediron->getNewContent() #5 /var/www/html/tt-rss/classes/pluginhandler.php(15): Feediron->test() #6 /var/www/html/tt-rss/backend.php(169): PluginHandler->catchall() #7 {main} thrown Forwarded For: 2001:ac8:31:8900:1012:b051:4a95:c76a Forwarded Protocol: https Remote IP: 172.19.0.1 Request URI: /tt-rss/backend.php User agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/117.0 ```

Context

Please provide any relevant information about your setup. This is important in case the issue is not reproducible except for under certain conditions.

see above, unless more is needed
dugite-code commented 11 months ago

The index can only be an int value. Did you have the old All-items xpath branch before?

I moved it to it's own experimental filter ages ago but never added it to the tags filter.

dugite-code commented 11 months ago

To get the all items to work with tags you would need to modify: https://github.com/feediron/ttrss_plugin-feediron/blob/091fd8a84e94f5d83f99f070f2c12c383bb224e5/filters/fi_mod_tags_xpath/init.php#L20

$newtag = ( new fi_mod_all_xpath() )->perform_filter( $html, $config, $settings );
benfishbus commented 9 months ago

Perhaps that was it. I made the edit you suggested. Now I am getting a lot of these:

Error: E_WARNING (2) Filename: plugins.local/feediron/filters/fi_mod_tags_xpath/init.php:24 Message:

1. plugins.local/feediron/filters/fi_mod_tags_xpath/init.php(24): ttrss_error_handler(Undefined array key 0, plugins.local/feediron/filters/fi_mod_tags_xpath/init.php)
2. plugins.local/feediron/init.php(372): get_tags(<!DOCTYPE html>
<html lang="en-US" class="no-js">
<head>

  <meta name="viewport" content="width=device-width, initial-scale=1">
  <meta name="theme-color" content="#d8352a">
  <meta name="msapplication-navbutton-color" content="#d8352a">
  <meta charset..., [{"type":"xpath","xpath":[{"xpath":"div[contains(@class, 'topic-list')]\/ul\/li","index":"all"}],"join_element":",","split":","}, [{"charset":"CP850"})
3. plugins.local/feediron/init.php(224): getArticleTags(<!DOCTYPE html>
<html lang="en-US" class="no-js">
<head>

  <meta name="viewport" content="width=device-width, initial-scale=1">
  <meta name="theme-color" content="#d8352a">
  <meta name="msapplication-navbutton-color" content="#d8352a">
  <meta charset..., [{"type":"xpath","xpath":[{"xpath":"div[contains(@class, 'topic-list')]\/ul\/li","index":"all"}],"join_element":",","split":","})
4. plugins.local/feediron/init.php(102): getNewContent(https://theconversation.com/supreme-court-considers-whether-to-uphold-law-that-keeps-guns-out-of-the-hands-of-domestic-abusers-217161, [{"type":"xpath","xpath":"article","cleanup":["header","div[contains(@class, 'topic-list')]","div[contains(@class, 'content-sidebar')]","aside","div[@class='inline-promos-skeleton']","@data-srcset","@sizes"],"modify":[{"type":"regex","pattern":["\/ data-src...)
5. classes/PluginHost.php(353): hook_article_filter([{"owner_uid":2,"guid":"2,tag:theconversation.com,2011:article\/217161","guid_hashed":"{\"ver\":2,\"uid\":2,\"hash\":\"SHA1:c95e8bcf0c8fe990b995fa5a07fdffa80bd06d44\"}","title":"Supreme Court considers whether to uphold law that keeps guns out of the hands ...)
6. classes/RSSUtils.php(888): chain_hooks_callback(hook_article_filter, {Closure}, [{"owner_uid":2,"guid":"2,tag:theconversation.com,2011:article\/217161","guid_hashed":"{\"ver\":2,\"uid\":2,\"hash\":\"SHA1:c95e8bcf0c8fe990b995fa5a07fdffa80bd06d44\"}","title":"Supreme Court considers whether to uphold law that keeps guns out of the hands ...)
7. update.php(239): update_rss_feed()

Error: E_WARNING (2) Filename: plugins.local/feediron/filters/fi_mod_all_xpath/init.php:8 Message:

1. plugins.local/feediron/filters/fi_mod_all_xpath/init.php(8): ttrss_error_handler(Undefined array key "debug", plugins.local/feediron/filters/fi_mod_all_xpath/init.php)
2. plugins.local/feediron/filters/fi_mod_tags_xpath/init.php(20): perform_filter(<!DOCTYPE html>
<html lang="en-US" class="no-js">
<head>

  <meta name="viewport" content="width=device-width, initial-scale=1">
  <meta name="theme-color" content="#d8352a">
  <meta name="msapplication-navbutton-color" content="#d8352a">
  <meta charset..., [{"type":"xpath","xpath":[{"xpath":"div[contains(@class, 'topic-list')]\/ul\/li","index":"all"}],"join_element":",","split":","}, [{"charset":"CP850"})
3. plugins.local/feediron/init.php(372): get_tags(<!DOCTYPE html>
<html lang="en-US" class="no-js">
<head>

  <meta name="viewport" content="width=device-width, initial-scale=1">
  <meta name="theme-color" content="#d8352a">
  <meta name="msapplication-navbutton-color" content="#d8352a">
  <meta charset..., [{"type":"xpath","xpath":[{"xpath":"div[contains(@class, 'topic-list')]\/ul\/li","index":"all"}],"join_element":",","split":","}, [{"charset":"CP850"})
4. plugins.local/feediron/init.php(224): getArticleTags(<!DOCTYPE html>
<html lang="en-US" class="no-js">
<head>

  <meta name="viewport" content="width=device-width, initial-scale=1">
  <meta name="theme-color" content="#d8352a">
  <meta name="msapplication-navbutton-color" content="#d8352a">
  <meta charset..., [{"type":"xpath","xpath":[{"xpath":"div[contains(@class, 'topic-list')]\/ul\/li","index":"all"}],"join_element":",","split":","})
5. plugins.local/feediron/init.php(102): getNewContent(https://theconversation.com/supreme-court-considers-whether-to-uphold-law-that-keeps-guns-out-of-the-hands-of-domestic-abusers-217161, [{"type":"xpath","xpath":"article","cleanup":["header","div[contains(@class, 'topic-list')]","div[contains(@class, 'content-sidebar')]","aside","div[@class='inline-promos-skeleton']","@data-srcset","@sizes"],"modify":[{"type":"regex","pattern":["\/ data-src...)
6. classes/PluginHost.php(353): hook_article_filter([{"owner_uid":2,"guid":"2,tag:theconversation.com,2011:article\/217161","guid_hashed":"{\"ver\":2,\"uid\":2,\"hash\":\"SHA1:c95e8bcf0c8fe990b995fa5a07fdffa80bd06d44\"}","title":"Supreme Court considers whether to uphold law that keeps guns out of the hands ...)
7. classes/RSSUtils.php(888): chain_hooks_callback(hook_article_filter, {Closure}, [{"owner_uid":2,"guid":"2,tag:theconversation.com,2011:article\/217161","guid_hashed":"{\"ver\":2,\"uid\":2,\"hash\":\"SHA1:c95e8bcf0c8fe990b995fa5a07fdffa80bd06d44\"}","title":"Supreme Court considers whether to uphold law that keeps guns out of the hands ...)
8. update.php(239): update_rss_feed()

Error: E_WARNING (2) Filename: plugins.local/feediron/filters/fi_mod_tags_xpath/init.php:24 Message:

1. plugins.local/feediron/filters/fi_mod_tags_xpath/init.php(24): ttrss_error_handler(Undefined array key 0, plugins.local/feediron/filters/fi_mod_tags_xpath/init.php)
2. plugins.local/feediron/init.php(372): get_tags(<!DOCTYPE html>
<html lang="en-US" class="no-js">
<head>

  <meta name="viewport" content="width=device-width, initial-scale=1">
  <meta name="theme-color" content="#d8352a">
  <meta name="msapplication-navbutton-color" content="#d8352a">
  <meta charset..., [{"type":"xpath","xpath":[{"xpath":"div[contains(@class, 'topic-list')]\/ul\/li","index":"all"}],"join_element":",","split":","}, [{"charset":"CP850"})
3. plugins.local/feediron/init.php(224): getArticleTags(<!DOCTYPE html>
<html lang="en-US" class="no-js">
<head>

  <meta name="viewport" content="width=device-width, initial-scale=1">
  <meta name="theme-color" content="#d8352a">
  <meta name="msapplication-navbutton-color" content="#d8352a">
  <meta charset..., [{"type":"xpath","xpath":[{"xpath":"div[contains(@class, 'topic-list')]\/ul\/li","index":"all"}],"join_element":",","split":","})
4. plugins.local/feediron/init.php(102): getNewContent(https://theconversation.com/generative-ai-like-chatgpt-could-help-boost-democracy-if-it-overcomes-key-hurdles-212664, [{"type":"xpath","xpath":"article","cleanup":["header","div[contains(@class, 'topic-list')]","div[contains(@class, 'content-sidebar')]","aside","div[@class='inline-promos-skeleton']","@data-srcset","@sizes"],"modify":[{"type":"regex","pattern":["\/ data-src...)
5. classes/PluginHost.php(353): hook_article_filter([{"owner_uid":2,"guid":"2,tag:theconversation.com,2011:article\/212664","guid_hashed":"{\"ver\":2,\"uid\":2,\"hash\":\"SHA1:51af924eee98b80b03ce33f81356866154522b9e\"}","title":"Generative AI like ChatGPT could help boost democracy \u2013 if it overcomes ke...)
6. classes/RSSUtils.php(888): chain_hooks_callback(hook_article_filter, {Closure}, [{"owner_uid":2,"guid":"2,tag:theconversation.com,2011:article\/212664","guid_hashed":"{\"ver\":2,\"uid\":2,\"hash\":\"SHA1:51af924eee98b80b03ce33f81356866154522b9e\"}","title":"Generative AI like ChatGPT could help boost democracy \u2013 if it overcomes ke...)
7. update.php(239): update_rss_feed()

Error: E_WARNING (2) Filename: plugins.local/feediron/filters/fi_mod_all_xpath/init.php:8 Message:

1. plugins.local/feediron/filters/fi_mod_all_xpath/init.php(8): ttrss_error_handler(Undefined array key "debug", plugins.local/feediron/filters/fi_mod_all_xpath/init.php)
2. plugins.local/feediron/filters/fi_mod_tags_xpath/init.php(20): perform_filter(<!DOCTYPE html>
<html lang="en-US" class="no-js">
<head>

  <meta name="viewport" content="width=device-width, initial-scale=1">
  <meta name="theme-color" content="#d8352a">
  <meta name="msapplication-navbutton-color" content="#d8352a">
  <meta charset..., [{"type":"xpath","xpath":[{"xpath":"div[contains(@class, 'topic-list')]\/ul\/li","index":"all"}],"join_element":",","split":","}, [{"charset":"CP850"})
3. plugins.local/feediron/init.php(372): get_tags(<!DOCTYPE html>
<html lang="en-US" class="no-js">
<head>

  <meta name="viewport" content="width=device-width, initial-scale=1">
  <meta name="theme-color" content="#d8352a">
  <meta name="msapplication-navbutton-color" content="#d8352a">
  <meta charset..., [{"type":"xpath","xpath":[{"xpath":"div[contains(@class, 'topic-list')]\/ul\/li","index":"all"}],"join_element":",","split":","}, [{"charset":"CP850"})
4. plugins.local/feediron/init.php(224): getArticleTags(<!DOCTYPE html>
<html lang="en-US" class="no-js">
<head>

  <meta name="viewport" content="width=device-width, initial-scale=1">
  <meta name="theme-color" content="#d8352a">
  <meta name="msapplication-navbutton-color" content="#d8352a">
  <meta charset..., [{"type":"xpath","xpath":[{"xpath":"div[contains(@class, 'topic-list')]\/ul\/li","index":"all"}],"join_element":",","split":","})
5. plugins.local/feediron/init.php(102): getNewContent(https://theconversation.com/generative-ai-like-chatgpt-could-help-boost-democracy-if-it-overcomes-key-hurdles-212664, [{"type":"xpath","xpath":"article","cleanup":["header","div[contains(@class, 'topic-list')]","div[contains(@class, 'content-sidebar')]","aside","div[@class='inline-promos-skeleton']","@data-srcset","@sizes"],"modify":[{"type":"regex","pattern":["\/ data-src...)
6. classes/PluginHost.php(353): hook_article_filter([{"owner_uid":2,"guid":"2,tag:theconversation.com,2011:article\/212664","guid_hashed":"{\"ver\":2,\"uid\":2,\"hash\":\"SHA1:51af924eee98b80b03ce33f81356866154522b9e\"}","title":"Generative AI like ChatGPT could help boost democracy \u2013 if it overcomes ke...)
7. classes/RSSUtils.php(888): chain_hooks_callback(hook_article_filter, {Closure}, [{"owner_uid":2,"guid":"2,tag:theconversation.com,2011:article\/212664","guid_hashed":"{\"ver\":2,\"uid\":2,\"hash\":\"SHA1:51af924eee98b80b03ce33f81356866154522b9e\"}","title":"Generative AI like ChatGPT could help boost democracy \u2013 if it overcomes ke...)
9. update.php(239): update_rss_feed()

Error: E_WARNING (2) Filename: plugins.local/feediron/filters/fi_mod_tags_xpath/init.php:24 Message: 

1. plugins.local/feediron/filters/fi_mod_tags_xpath/init.php(24): ttrss_error_handler(Undefined array key 0, plugins.local/feediron/filters/fi_mod_tags_xpath/init.php)
2. plugins.local/feediron/init.php(372): get_tags(<!DOCTYPE html>
<html lang="en-US" class="no-js">
<head>

  <meta name="viewport" content="width=device-width, initial-scale=1">
  <meta name="theme-color" content="#d8352a">
  <meta name="msapplication-navbutton-color" content="#d8352a">
  <meta charset..., [{"type":"xpath","xpath":[{"xpath":"div[contains(@class, 'topic-list')]\/ul\/li","index":"all"}],"join_element":",","split":","}, [{"charset":"CP850"})
3. plugins.local/feediron/init.php(224): getArticleTags(<!DOCTYPE html>
<html lang="en-US" class="no-js">
<head>

  <meta name="viewport" content="width=device-width, initial-scale=1">
  <meta name="theme-color" content="#d8352a">
  <meta name="msapplication-navbutton-color" content="#d8352a">
  <meta charset..., [{"type":"xpath","xpath":[{"xpath":"div[contains(@class, 'topic-list')]\/ul\/li","index":"all"}],"join_element":",","split":","})
4. plugins.local/feediron/init.php(102): getNewContent(https://theconversation.com/generative-ai-like-chatgpt-could-help-boost-democracy-if-it-overcomes-key-hurdles-212664, [{"type":"xpath","xpath":"article","cleanup":["header","div[contains(@class, 'topic-list')]","div[contains(@class, 'content-sidebar')]","aside","div[@class='inline-promos-skeleton']","@data-srcset","@sizes"],"modify":[{"type":"regex","pattern":["\/ data-src...)
5. classes/PluginHost.php(353): hook_article_filter([{"owner_uid":2,"guid":"2,tag:theconversation.com,2011:article\/212664","guid_hashed":"{\"ver\":2,\"uid\":2,\"hash\":\"SHA1:51af924eee98b80b03ce33f81356866154522b9e\"}","title":"Generative AI like ChatGPT could help boost democracy \u2013 if it overcomes ke...)
6. classes/RSSUtils.php(888): chain_hooks_callback(hook_article_filter, {Closure}, [{"owner_uid":2,"guid":"2,tag:theconversation.com,2011:article\/212664","guid_hashed":"{\"ver\":2,\"uid\":2,\"hash\":\"SHA1:51af924eee98b80b03ce33f81356866154522b9e\"}","title":"Generative AI like ChatGPT could help boost democracy \u2013 if it overcomes ke...)
7. update.php(239): update_rss_feed()

Error: E_WARNING (2) Filename: plugins.local/feediron/filters/fi_mod_all_xpath/init.php:8 Message:

1. plugins.local/feediron/filters/fi_mod_all_xpath/init.php(8): ttrss_error_handler(Undefined array key "debug", plugins.local/feediron/filters/fi_mod_all_xpath/init.php)
2. plugins.local/feediron/filters/fi_mod_tags_xpath/init.php(20): perform_filter(<!DOCTYPE html>
<html lang="en-US" class="no-js">
<head>

  <meta name="viewport" content="width=device-width, initial-scale=1">
  <meta name="theme-color" content="#d8352a">
  <meta name="msapplication-navbutton-color" content="#d8352a">
  <meta charset..., [{"type":"xpath","xpath":[{"xpath":"div[contains(@class, 'topic-list')]\/ul\/li","index":"all"}],"join_element":",","split":","}, [{"charset":"CP850"})
3. plugins.local/feediron/init.php(372): get_tags(<!DOCTYPE html>
<html lang="en-US" class="no-js">
<head>

  <meta name="viewport" content="width=device-width, initial-scale=1">
  <meta name="theme-color" content="#d8352a">
  <meta name="msapplication-navbutton-color" content="#d8352a">
  <meta charset..., [{"type":"xpath","xpath":[{"xpath":"div[contains(@class, 'topic-list')]\/ul\/li","index":"all"}],"join_element":",","split":","}, [{"charset":"CP850"})
4. plugins.local/feediron/init.php(224): getArticleTags(<!DOCTYPE html>
<html lang="en-US" class="no-js">
<head>

  <meta name="viewport" content="width=device-width, initial-scale=1">
  <meta name="theme-color" content="#d8352a">
  <meta name="msapplication-navbutton-color" content="#d8352a">
  <meta charset..., [{"type":"xpath","xpath":[{"xpath":"div[contains(@class, 'topic-list')]\/ul\/li","index":"all"}],"join_element":",","split":","})
5. plugins.local/feediron/init.php(102): getNewContent(https://theconversation.com/generative-ai-like-chatgpt-could-help-boost-democracy-if-it-overcomes-key-hurdles-212664, [{"type":"xpath","xpath":"article","cleanup":["header","div[contains(@class, 'topic-list')]","div[contains(@class, 'content-sidebar')]","aside","div[@class='inline-promos-skeleton']","@data-srcset","@sizes"],"modify":[{"type":"regex","pattern":["\/ data-src...)
6. classes/PluginHost.php(353): hook_article_filter([{"owner_uid":2,"guid":"2,tag:theconversation.com,2011:article\/212664","guid_hashed":"{\"ver\":2,\"uid\":2,\"hash\":\"SHA1:51af924eee98b80b03ce33f81356866154522b9e\"}","title":"Generative AI like ChatGPT could help boost democracy \u2013 if it overcomes ke...)
7. classes/RSSUtils.php(888): chain_hooks_callback(hook_article_filter, {Closure}, [{"owner_uid":2,"guid":"2,tag:theconversation.com,2011:article\/212664","guid_hashed":"{\"ver\":2,\"uid\":2,\"hash\":\"SHA1:51af924eee98b80b03ce33f81356866154522b9e\"}","title":"Generative AI like ChatGPT could help boost democracy \u2013 if it overcomes ke...)
8. update.php(239): update_rss_feed()
dugite-code commented 9 months ago

Ok there is one bug here that's been missed and I also need to just bite the bullet and merge the "all" index code into the xpath filter. Might as well improve the tags filter to handle an array as well.

dugite-code commented 9 months ago

I've started work on the all-tags branch