custom-components / feedparser

📰 RSS Feed Integration
MIT License
144 stars 32 forks source link

My feed wont populate #30

Open cperryoh opened 3 years ago

cperryoh commented 3 years ago

I have been messing around with my lovelace and I can't get it to work Here is what the card looks like: Image 1-29-21 at 10 00 PM

Here is my yaml for that card

type: 'custom:list-card'
entity: sensor.news
title: News feed
feed_attribute: entries
columns:
  - title: ''
    type: image
    add_link: link
    field: image
  - title: Title
    field: title
    style:
      - white-space: nowrap
  - title: Description
    field: description

Here is my config for feedparser:

sensor:
  - platform: feedparser
    name: news
    feed_url: 'http://rss.cnn.com/rss/cnn_topstories.rss'
    date_format: '%a, %b %d %I:%M %p'
    inclusions:
        - title
        - link
        - description
        - image
        - language
        - pubDate
iantrich commented 3 years ago

Card config fmdoesnt matter if your sensor is not being populated. Is there data in the feedparser sensor if you look in the Developer Toola > States?

cperryoh commented 3 years ago

There is

Screen Shot 2021-01-31 at 12 11 04 AM
iantrich commented 3 years ago

So each entry only has title and link, so that's the only fields you can use in the card

cperryoh commented 3 years ago

I got it to populate but the images for each story are just the hassio icon. I've tried multiple news feeds and it's the same issue.

type: 'custom:list-card'
entity: sensor.gamingnews
title: News
feed_attribute: entries
columns:
  - title: Image
    type: image
    add_link: link
    field: image
  - title: Title
    field: title

image

cperryoh commented 3 years ago

or this image

iantrich commented 3 years ago

Well is there an image in the feed? Looked like a link to the story itself

cperryoh commented 3 years ago

Yes there is

I'll take this top story for example: image

Here is what the feedparser lists as the image: http://feeds.feedburner.com/~ff/rss/cnn_topstories?d=yIl2AUoC8zA

Here is the actual article: https://www.cnn.com/2021/02/01/us/coronavirus-vaccines-variants-return-to-normal/index.html?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+rss%2Fcnn_topstories+%28RSS%3A+CNN+-+Top+Stories%29

Do you by chance have a feed that you know works with images that I can try out?

cecheto commented 3 years ago

Hi, Maybe this could help. I was experiencing the same issues as you had and I did the following: Remove inclusions and exclusions and let de feeder fetch al the value. To my surprise the description field in the xml was beed imported as "summary". Also noticed that the image was not been feeded.

cperryoh commented 3 years ago

Remove inclusions and exclusions and let de feeder fetch al the value.

I don't know if this is exactly what you meant but I blew away all inclusions and exclusions and now my feed won't populate. Maybe I misunderstood.

Here is the state of the parser: image

Here is my config:

  - platform: feedparser
    name: news
    feed_url: 'http://rss.cnn.com/rss/cnn_topstories.rss'
    date_format: '%a, %b %d %I:%M %p'

And my Lovelace end looks like this: image

cecheto commented 3 years ago

After doing some extra research I found that there are two problems:

1.- There are no images (at least article related images) inside the feed. The only solution I can see is modifying the feeder to search for images following the link. I am trying to do this, I will keep you updated.

2.- But also, the feed explorer does not support multiline images and the small images that appear on the feed are written across multiple lines. This can be solved bi adding a re.DOTALL as de last parameter of the re.findall function on line 95. But again the only images there are in the feed are the ones shown in your post.

Line 95: images = re.findall(r"<img.+?src=\"(.+?)\".+?>", entry['summary'], re.DOTALL)

cperryoh commented 3 years ago

That is a gross solution (Not your fault, just overall a lot of work). Do you know of a feed that works with this card that I can use without jumping through hoops?

cecheto commented 3 years ago

This is what you search for? image

cecheto commented 3 years ago

This is how I did it, first the sensor:

sensor:
  - platform: feedparser
    name: news
    feed_url: 'http://rss.cnn.com/rss/cnn_topstories.rss'   
    date_format: '%a, %b %d %I:%M %p'
    show_topn: 10
    inclusions:
      - title
      - link
      - linkimage

I only take 10 news because mi connection is slow and it has to load all the selected news.

Now the card:

entity: sensor.news
title: News feed
feed_attribute: entries
columns:
  - title: ''
    type: image
    add_link: link
    field: linkimage
    width: 126
  - title: Title
    field: title

Finally the code added at line 99:

                if 'linkimage' in self._inclusions and 'linkimage' not in entryValue.keys():
                    images = []
                    if 'link' in entry.keys():
                        req = Request(entry['link'])
                        html_page = urlopen(req)
                        images = re.findall(r"<img.+?src=\"(.+?)\".+?>", html_page.read().decode('utf-8'), re.DOTALL) 
                    if images:
                        entryValue['linkimage'] = images[0]

I know it is not nice. But you can not do it with just the feed you want. The image information is not there. I can not think of any other way of getting an image for then entities of the feed since there is no image in it.

Romkabouter commented 3 years ago

Hi, Maybe this could help. I was experiencing the same issues as you had and I did the following: Remove inclusions and exclusions and let de feeder fetch al the value. To my surprise the description field in the xml was beed imported as "summary". Also noticed that the image was not been feeded.

I had exactly the same, description was in the feed but somehow renamed to summary