search

shouya / rss-funnel

Self-hosted RSS multi-tool
https://rss-funnel-demo.fly.dev
GNU General Public License v3.0
107 stars 4 forks source link

Google News issues #133

Closed uGeek closed 1 week ago

uGeek commented 1 month ago

I don't know what's happening, but when I merge some Google News RSS feeds, it gives an error. Here are some of them:

shouya commented 1 month ago

Could you detail what exact errors you're getting and provide your config file?

From my testing I'm getting normal result with the following endpoint definition:

  - path: /google-news.xml
    source:
      title: Google News
      link: https://news.google.com/
      format: rss
    filters:
      - merge:
          - https://news.google.com/rss/search?q=site:genbeta.com&hl=es&gl=ES&ceid=ES:es
          - https://news.google.com/rss/search?q=site:genbeta.com&hl=es&gl=ES&ceid=ES:es
          - https://news.google.com/rss/search?q=site:elandroidefeliz.com&hl=es&gl=ES&ceid=ES:es
          - https://news.google.com/rss/search?q=site:xatakamovil.com&hl=es&gl=ES&ceid=ES:es
          - https://news.google.com/rss/search?q=site:www.xatakandroid.com&hl=es&gl=ES&ceid=ES:es
uGeek commented 1 month ago

No errors are appearing in the logs, but no posts are shown. How can I verify this? If I combine sources, other sources are displayed but this particular feed is not. I don't understand because the other feeds are from Google News and they are functioning.

  - path: /genbeta.xml
    source:
      title: Genbeta
      link: https://news.google.com/
      format: rss
    filters:
      - merge:
          - https://news.google.com/rss/search?q=site:genbeta.com&hl=es&gl=ES&ceid=ES:es

image

uGeek commented 1 month ago

In this screenshot, I am showing 4 of the previous sources, but only the Happy Android Feed is working; the rest are not

image

shouya commented 1 month ago

Rss funnel shows no articles because these source feeds themselves don't have any articles.

For example, https://news.google.com/search?q=site:xatakamovil.com&amp;hl=es&amp;gl=ES&amp;ceid=ES:es has the following content (Notice the lack of <item>):

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<rss version="2.0" xmlns:media="http://search.yahoo.com/mrss/">
  <channel>
    <generator>NFE/5.0</generator>
    <title>"site:xatakamovil.com" - Google News</title>
    <link>https://news.google.com/search?q=site:xatakamovil.com&amp;hl=es&amp;gl=ES&amp;ceid=ES:es</link>
    <language>es</language>
    <webMaster>news-webmaster@google.com</webMaster>
    <copyright>2024 Google Inc.</copyright>
    <lastBuildDate>Mon, 29 Jul 2024 13:08:07 GMT</lastBuildDate>
    <description>Google News</description>
  </channel>
</rss>

In contrast, the working source https://news.google.com/rss/search?q=site:elandroidefeliz.com&hl=es&gl=ES&ceid=ES:es looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<rss xmlns:media="http://search.yahoo.com/mrss/" version="2.0">
  <channel>
    <generator>NFE/5.0</generator>
    <title>"site:elandroidefeliz.com" - Google News</title>
    <link>https://news.google.com/search?q=site:elandroidefeliz.com&amp;hl=es&amp;gl=ES&amp;ceid=ES:es</link>
    <language>es</language>
    <webMaster>news-webmaster@google.com</webMaster>
    <copyright>2024 Google Inc.</copyright>
    <lastBuildDate>Mon, 29 Jul 2024 13:10:08 GMT</lastBuildDate>
    <description>Google News</description>
    <item>
      <title>Tu web de Android y Tecnología - El Androide Feliz - El Androide Feliz</title>
      <link>https://news.google.com/rss/articles/CBMiHGh0dHBzOi8vZWxhbmRyb2lkZWZlbGl6LmNvbS_SAQA?oc=5</link>
      <guid isPermaLink="false">CBMiHGh0dHBzOi8vZWxhbmRyb2lkZWZlbGl6LmNvbS_SAQA</guid>
      <pubDate>Wed, 12 Apr 2017 18:52:03 GMT</pubDate>
      <description>&lt;a href="https://news.google.com/rss/articles/CBMiHGh0dHBzOi8vZWxhbmRyb2lkZWZlbGl6LmNvbS_SAQA?oc=5" target="_blank"&gt;Tu web de Android y Tecnología - El Androide Feliz&lt;/a&gt;&amp;nbsp;&amp;nbsp;&lt;font color="#6f6f6f"&gt;El Androide Feliz&lt;/font&gt;</description>
      <source url="https://elandroidefeliz.com">El Androide Feliz</source>
    </item>
    <item>
      <title>Fastboot en Android: todos los comandos y guía de uso práctico - El Androide Feliz</title>
      <link>https://news.google.com/rss/articles/CBMiSmh0dHBzOi8vZWxhbmRyb2lkZWZlbGl6LmNvbS90dXRvcmlhbC1mYXN0Ym9vdC1hbmRyb2lkLWNvbWFuZG9zLXktZWplbXBsb3Mv0gEA?oc=5</link>
      <guid isPermaLink="false">CBMiSmh0dHBzOi8vZWxhbmRyb2lkZWZlbGl6LmNvbS90dXRvcmlhbC1mYXN0Ym9vdC1hbmRyb2lkLWNvbWFuZG9zLXktZWplbXBsb3Mv0gEA</guid>
      <pubDate>Fri, 06 Oct 2017 15:35:20 GMT</pubDate>
      <description>&lt;a href="https://news.google.com/rss/articles/CBMiSmh0dHBzOi8vZWxhbmRyb2lkZWZlbGl6LmNvbS90dXRvcmlhbC1mYXN0Ym9vdC1hbmRyb2lkLWNvbWFuZG9zLXktZWplbXBsb3Mv0gEA?oc=5" target="_blank"&gt;Fastboot en Android: todos los comandos y guía de uso práctico&lt;/a&gt;&amp;nbsp;&amp;nbsp;&lt;font color="#6f6f6f"&gt;El Androide Feliz&lt;/font&gt;</description>
      <source url="https://elandroidefeliz.com">El Androide Feliz</source>
    </item>
    ...

It's not a bug of rss-funnel.

uGeek commented 1 month ago

Rss funnel shows no articles because these source feeds themselves don't have any articles.

For example, https://news.google.com/search?q=site:xatakamovil.com&amp;hl=es&amp;gl=ES&amp;ceid=ES:es has the following content (Notice the lack of <item>):

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<rss version="2.0" xmlns:media="http://search.yahoo.com/mrss/">
  <channel>
    <generator>NFE/5.0</generator>
    <title>"site:xatakamovil.com" - Google News</title>
    <link>https://news.google.com/search?q=site:xatakamovil.com&amp;hl=es&amp;gl=ES&amp;ceid=ES:es</link>
    <language>es</language>
    <webMaster>news-webmaster@google.com</webMaster>
    <copyright>2024 Google Inc.</copyright>
    <lastBuildDate>Mon, 29 Jul 2024 13:08:07 GMT</lastBuildDate>
    <description>Google News</description>
  </channel>
</rss>

In contrast, the working source https://news.google.com/rss/search?q=site:elandroidefeliz.com&hl=es&gl=ES&ceid=ES:es looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<rss xmlns:media="http://search.yahoo.com/mrss/" version="2.0">
  <channel>
    <generator>NFE/5.0</generator>
    <title>"site:elandroidefeliz.com" - Google News</title>
    <link>https://news.google.com/search?q=site:elandroidefeliz.com&amp;hl=es&amp;gl=ES&amp;ceid=ES:es</link>
    <language>es</language>
    <webMaster>news-webmaster@google.com</webMaster>
    <copyright>2024 Google Inc.</copyright>
    <lastBuildDate>Mon, 29 Jul 2024 13:10:08 GMT</lastBuildDate>
    <description>Google News</description>
    <item>
      <title>Tu web de Android y Tecnología - El Androide Feliz - El Androide Feliz</title>
      <link>https://news.google.com/rss/articles/CBMiHGh0dHBzOi8vZWxhbmRyb2lkZWZlbGl6LmNvbS_SAQA?oc=5</link>
      <guid isPermaLink="false">CBMiHGh0dHBzOi8vZWxhbmRyb2lkZWZlbGl6LmNvbS_SAQA</guid>
      <pubDate>Wed, 12 Apr 2017 18:52:03 GMT</pubDate>
      <description>&lt;a href="https://news.google.com/rss/articles/CBMiHGh0dHBzOi8vZWxhbmRyb2lkZWZlbGl6LmNvbS_SAQA?oc=5" target="_blank"&gt;Tu web de Android y Tecnología - El Androide Feliz&lt;/a&gt;&amp;nbsp;&amp;nbsp;&lt;font color="#6f6f6f"&gt;El Androide Feliz&lt;/font&gt;</description>
      <source url="https://elandroidefeliz.com">El Androide Feliz</source>
    </item>
    <item>
      <title>Fastboot en Android: todos los comandos y guía de uso práctico - El Androide Feliz</title>
      <link>https://news.google.com/rss/articles/CBMiSmh0dHBzOi8vZWxhbmRyb2lkZWZlbGl6LmNvbS90dXRvcmlhbC1mYXN0Ym9vdC1hbmRyb2lkLWNvbWFuZG9zLXktZWplbXBsb3Mv0gEA?oc=5</link>
      <guid isPermaLink="false">CBMiSmh0dHBzOi8vZWxhbmRyb2lkZWZlbGl6LmNvbS90dXRvcmlhbC1mYXN0Ym9vdC1hbmRyb2lkLWNvbWFuZG9zLXktZWplbXBsb3Mv0gEA</guid>
      <pubDate>Fri, 06 Oct 2017 15:35:20 GMT</pubDate>
      <description>&lt;a href="https://news.google.com/rss/articles/CBMiSmh0dHBzOi8vZWxhbmRyb2lkZWZlbGl6LmNvbS90dXRvcmlhbC1mYXN0Ym9vdC1hbmRyb2lkLWNvbWFuZG9zLXktZWplbXBsb3Mv0gEA?oc=5" target="_blank"&gt;Fastboot en Android: todos los comandos y guía de uso práctico&lt;/a&gt;&amp;nbsp;&amp;nbsp;&lt;font color="#6f6f6f"&gt;El Androide Feliz&lt;/font&gt;</description>
      <source url="https://elandroidefeliz.com">El Androide Feliz</source>
    </item>
    ...

It's not a bug of rss-funnel.

The feed I have added is not that one https://news.google.com/search?q=site:xatakamovil.com&amp;hl=es&amp;gl=ES&amp;ceid=ES:es it's this one: https://news.google.com/rss/search?q=site:xatakamovil.com&hl=es&gl=ES&ceid=ES:es

These that I put at the beginning don't work either:

shouya commented 1 month ago

What I'm saying is that all these feeds have no articles (except for "site:elandroidefeliz.com"). I have showed you one example, and you can check the others yourself.

Or try add these feeds to your feed reader and see which one has any contents. If they do not contain any content themselves there is nothing I can do.

uGeek commented 1 month ago

You're right! The problem has been that from Spain it does show the content of the feed, that's why I didn't understand anything you were telling me, but I have the rss-funnel on a vps outside of Spain and that's why I don't works. I've been to France and you can't see it either. So the problem is with Google News and not with rss-funnel. Sorry, but I didn't imagine that by geolocation it shows different feeds and if you search On genbeta.com it should show it everywhere....

shouya commented 1 month ago

I see! That explains what I see. If you can have proxy from Spain then you can use it for fetching the feed.

  - path: /google-news.xml
    source:
      title: Google News
      link: https://news.google.com/
      format: rss
    filters:
      - merge:
          sources:
            - https://news.google.com/rss/search?q=site:genbeta.com&hl=es&gl=ES&ceid=ES:es
            - https://news.google.com/rss/search?q=site:genbeta.com&hl=es&gl=ES&ceid=ES:es
            - https://news.google.com/rss/search?q=site:elandroidefeliz.com&hl=es&gl=ES&ceid=ES:es
            - https://news.google.com/rss/search?q=site:xatakamovil.com&hl=es&gl=ES&ceid=ES:es
            - https://news.google.com/rss/search?q=site:www.xatakandroid.com&hl=es&gl=ES&ceid=ES:es
          client:
            proxy: http://proxy.example.com:8080
uGeek commented 1 week ago

Yes. I was reading about it in the documentation. It's a great option to be able to use a proxy. I had never thought that you wouldn't have the same access from one country or another. rss-funnel is a great application.

shouya commented 1 week ago

Thank you for the kind words!