danmactough / node-feedparser

Robust RSS, Atom, and RDF feed parsing in Node.js
Other
1.97k stars 192 forks source link

More fine grained end event #105

Closed bitliner closed 10 years ago

bitliner commented 10 years ago

Let's assume that in my event handler associated to the event readable my code runs an other asynchronous task.

This means that event end will not be fired necessarily when all the handlers of readable ended.

So I should add an other event to manage the end of the processing of each item (read in readable event).

Do you plan to add this event?

danmactough commented 10 years ago

Can you be more specific about what kind of async task you want to run on readable yet still care about the end event?

bitliner commented 10 years ago

In my use case it is fetching more information from the page the feed item relates to. It could be saving an item to the database too.

I need to hook the end of all those handlers. Of course I may use an array of promise or something similar (it may be better because the responsibility is not directly related to the module feedparser but to a parser built on top of it), but you may have thought to implement that in your module.

danmactough commented 10 years ago

Yeah, that's a concern that's outside the responsibility of feedparser. Are you doing this for every item instead of once per feed?

bitliner commented 10 years ago

Yep, for every item. How may I do that once per feed? May be I wouldn't do that once per feed, because even if a feed usually includes few items, I wouldn't use too much the memory

danmactough commented 10 years ago

How may I do that once per feed?

Without knowing what you're trying to do, it's hard to say. But if you only NEED to do it once per feed (for example, not every feed item points to a different site from which you need fetch information), then I would put that in your processing chain before feedparser. Something like this (pseudo-code):


request.get('http://something.com/feed.xml')
  .pipe([ your new thing to get other information ])
  .pipe(feedparser)

The iconv example also demonstrates how you could put something in front of feedparser.