http://la-grange.net/feed posts showing as already read.

[rik]

https://gist.github.com/Rik/050b257b775dc284aeb1

Author is @karlcow

[karlcow]

And to help @Rik in his inquiry http://feedvalidator.org/check.cgi?url=http%3A%2F%2Fla-grange.net%2Ffeed

[benubois]

Hi @Rik & @karlcow,

Happy to look into it. At first glance it does appear that what is in Feedbin matches what is in the feed.

What appears to be missing?

[rik]

Oh yeah, I can see them in my account, under the All section. So I think they all showed up as already read.

[karlcow]

@Rik is it because they are old? compared to the other ones? something like mark as read if older than n days

[rik]

I don't know why it's happening but it is very annoying. I've changed the title to be closer to the issue.

[karlcow]

I guess the dates are important in this case.

The feed date is

<updated>2015-11-17T14:38:00Z</updated>

The structure of a feed entry is:

<entry>
  <id>tag:la-grange.net,2015-09-14:2015/09/14/nomade</id>
  <link rel="alternate" type="text/html" href="http://www.la-grange.net/2015/09/14/nomade"/>
  <title>Confins de l'immensité</title>

  <published>2015-09-14T23:59:00+09:00</published>
  <updated>2015-11-17T14:38:00Z</updated>

  <content type="xhtml">
   <div xmlns="http://www.w3.org/1999/xhtml">
      <article>
      […]
      </article>
    </div>
  </content>
  <link rel="license" href="http://creativecommons.org/licenses/by/2.0/fr/"/>
</entry>

[karlcow]

(note I don't know feedbin code, just looking at it now.)

@Rik @benubois

Maybe I found the culprit https://github.com/feedbin/feedbin/blob/735b89d03f2656e41da3bbc24d5f0c93a394689a/app/models/subscription.rb#L21

In this piece of code:

def mark_as_unread
    entries = Entry.select(:id, :published, :created_at).where(feed_id: self.feed_id).where('published > ?', Time.now.ago(2.weeks))
    if entries.length == 0
      entries = Entry.select(:id, :published, :created_at).where(feed_id: self.feed_id).order('published DESC').limit(1)
    end
    unread_entries = []
    entries.each do |entry|
      unread_entries << UnreadEntry.new(user_id: self.user_id, feed_id: self.feed_id, entry_id: entry.id, published: entry.published, entry_created_at: entry.created_at)
    end
    UnreadEntry.import(unread_entries, validate: false)
  end

What is the goal of

.where('published > ?', Time.now.ago(2.weeks))

Does that mean that anything which is more than 2 weeks old is considered as read? In this case all my entries are published (created) indeed more than 2 weeks ago, more often on the verge of the 2 months. The updated field is the one which matters for the freshness.

[rik]

@benubois Any update on this with @karlcow's insight?

[rik]

Just had the same issue with http://code.flickr.net.

[benubois]

Hi @rik,

http://code.flickr.net seems unrelated. What exactly is the issue you're having with that one?

For @karlcow's site it is related to the published date on the feed being far in the past. Feedbin does not mark posts older than one month as unread, because it solved a problem with other feeds that had old duplicate entries showing up.

Unfortunately, changing this behavior would mean causing problems with other feeds. @karlcow would it be possible to have the published date reflect when an entry is actually published in the feed?

[karlcow]

@benubois I could.

From RFC 4287 about published

   The "atom:published" element is a Date construct indicating an
   instant in time associated with an event early in the life cycle of
   the entry.

   atomPublished = element atom:published { atomDateConstruct }

   Typically, atom:published will be associated with the initial
   creation or first availability of the resource.

because it solved a problem with other feeds that had old duplicate entries showing up.

With the same atom:id?

[benubois]

@karlcow Cool! I don't think you're doing anything invalid in your feed. I've just noticed an old entry showing up in a feed is a strong indicator that something is wrong with the feed, which is what led to this work-around in the first place.

With the same atom:id?

Unfortunately not all feeds use id, guid, atom:id etc… In these cases Feedbin constructs one based on other elements of an entry.

[rik]

http://code.flickr.net seems unrelated. What exactly is the issue you're having with that one?

This post http://code.flickr.net/2016/04/05/our-justified-layout-goes-open-source/ was marked as read.

[rik]

Unfortunately not all feeds use id, guid, atom:id etc… In these cases Feedbin constructs one based on other elements of an entry.

What about using the id provided by the feed if it exists and keep the current 2 week workaround for feeds without ids?

[karlcow]

About

This post http://code.flickr.net/2016/04/05/our-justified-layout-goes-open-source/ was marked as read.

The feed says:

    <lastBuildDate>Tue, 05 Apr 2016 18:19:55 +0000</lastBuildDate>

and the specific item:

    <item>
        <title>Our Justified Layout Goes Open Source</title>
        <link>http://code.flickr.net/2016/04/05/our-justified-layout-goes-open-source/</link>
        <pubDate>Tue, 05 Apr 2016 17:01:53 +0000</pubDate>
        <dc:creator><![CDATA[jimwhimpey]]></dc:creator>
                <category><![CDATA[open source]]></category>

        <guid isPermaLink="false">http://code.flickr.net/?p=3370</guid>
…
</xml>

So Maybe another issue.

[benubois]

What about using the id provided by the feed if it exists and keep the current 2 week workaround for feeds without ids?

@rik That sounds like a great idea! I'll give it a try.

[rik]

I've tried to setup an environment to work on this. Sadly, I couldn't complete.

I've updated the install instructions in https://github.com/feedbin/feedbin/pull/211 based on my findings.

I'm still stuck at rake db:setup:

/usr/local/lib/ruby/gems/2.4.0/gems/activesupport-5.0.1/lib/active_support/xml_mini.rb:51: warning: constant ::Fixnum is deprecated
/usr/local/lib/ruby/gems/2.4.0/gems/activesupport-5.0.1/lib/active_support/xml_mini.rb:52: warning: constant ::Bignum is deprecated
/usr/local/lib/ruby/gems/2.4.0/gems/therubyracer-0.12.2/lib/v8/conversion.rb:21: warning: constant ::Fixnum is deprecated
rake aborted!
Bundler::GemRequireError: There was an error while trying to load the gem 'coffee-rails'.
Gem Load Error is: wrong argument type Class (expected Module)
Backtrace for gem load error is:
/usr/local/lib/ruby/gems/2.4.0/gems/therubyracer-0.12.2/lib/v8/conversion.rb:23:in `include'
/usr/local/lib/ruby/gems/2.4.0/gems/therubyracer-0.12.2/lib/v8/conversion.rb:23:in `block (2 levels) in <top (required)>'
/usr/local/lib/ruby/gems/2.4.0/gems/therubyracer-0.12.2/lib/v8/conversion.rb:22:in `class_eval'
/usr/local/lib/ruby/gems/2.4.0/gems/therubyracer-0.12.2/lib/v8/conversion.rb:22:in `block in <top (required)>'
/usr/local/lib/ruby/gems/2.4.0/gems/therubyracer-0.12.2/lib/v8/conversion.rb:21:in `each'
/usr/local/lib/ruby/gems/2.4.0/gems/therubyracer-0.12.2/lib/v8/conversion.rb:21:in `<top (required)>'
/usr/local/lib/ruby/gems/2.4.0/gems/therubyracer-0.12.2/lib/v8.rb:22:in `require'
/usr/local/lib/ruby/gems/2.4.0/gems/therubyracer-0.12.2/lib/v8.rb:22:in `<top (required)>'
/usr/local/lib/ruby/gems/2.4.0/gems/execjs-2.7.0/lib/execjs/ruby_racer_runtime.rb:108:in `require'
/usr/local/lib/ruby/gems/2.4.0/gems/execjs-2.7.0/lib/execjs/ruby_racer_runtime.rb:108:in `available?'
/usr/local/lib/ruby/gems/2.4.0/gems/execjs-2.7.0/lib/execjs/runtimes.rb:63:in `each'
/usr/local/lib/ruby/gems/2.4.0/gems/execjs-2.7.0/lib/execjs/runtimes.rb:63:in `find'
/usr/local/lib/ruby/gems/2.4.0/gems/execjs-2.7.0/lib/execjs/runtimes.rb:63:in `best_available'
/usr/local/lib/ruby/gems/2.4.0/gems/execjs-2.7.0/lib/execjs/runtimes.rb:57:in `autodetect'
/usr/local/lib/ruby/gems/2.4.0/gems/execjs-2.7.0/lib/execjs.rb:5:in `<module:ExecJS>'
/usr/local/lib/ruby/gems/2.4.0/gems/execjs-2.7.0/lib/execjs.rb:4:in `<top (required)>'
/usr/local/lib/ruby/gems/2.4.0/gems/coffee-script-2.4.1/lib/coffee_script.rb:1:in `require'
/usr/local/lib/ruby/gems/2.4.0/gems/coffee-script-2.4.1/lib/coffee_script.rb:1:in `<top (required)>'
/usr/local/lib/ruby/gems/2.4.0/gems/coffee-script-2.4.1/lib/coffee-script.rb:1:in `require'
/usr/local/lib/ruby/gems/2.4.0/gems/coffee-script-2.4.1/lib/coffee-script.rb:1:in `<top (required)>'
/usr/local/lib/ruby/gems/2.4.0/gems/coffee-rails-4.2.1/lib/coffee-rails.rb:1:in `require'
/usr/local/lib/ruby/gems/2.4.0/gems/coffee-rails-4.2.1/lib/coffee-rails.rb:1:in `<top (required)>'
/usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler/runtime.rb:91:in `require'
/usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler/runtime.rb:91:in `block (2 levels) in require'
/usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler/runtime.rb:86:in `each'
/usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler/runtime.rb:86:in `block in require'
/usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler/runtime.rb:75:in `each'
/usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler/runtime.rb:75:in `require'
/usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler.rb:107:in `require'
/Users/rik24d/code/feedbin/config/application.rb:8:in `<top (required)>'
/usr/local/Cellar/ruby/2.4.1_1/lib/ruby/2.4.0/rubygems/core_ext/kernel_require.rb:55:in `require'
/usr/local/Cellar/ruby/2.4.1_1/lib/ruby/2.4.0/rubygems/core_ext/kernel_require.rb:55:in `require'
/Users/rik24d/code/feedbin/Rakefile:4:in `<top (required)>'
/usr/local/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/rake_module.rb:28:in `load'
/usr/local/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/rake_module.rb:28:in `load_rakefile'
/usr/local/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/application.rb:687:in `raw_load_rakefile'
/usr/local/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/application.rb:96:in `block in load_rakefile'
/usr/local/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/application.rb:178:in `standard_exception_handling'
/usr/local/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/application.rb:95:in `load_rakefile'
/usr/local/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/application.rb:79:in `block in run'
/usr/local/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/application.rb:178:in `standard_exception_handling'
/usr/local/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/application.rb:77:in `run'
/usr/local/lib/ruby/gems/2.4.0/gems/rake-12.0.0/exe/rake:27:in `<top (required)>'
/usr/local/bin/rake:22:in `load'
/usr/local/bin/rake:22:in `<main>'
Bundler Error Backtrace:
/usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler/runtime.rb:94:in `rescue in block (2 levels) in require'
/usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler/runtime.rb:90:in `block (2 levels) in require'
/usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler/runtime.rb:86:in `each'
/usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler/runtime.rb:86:in `block in require'
/usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler/runtime.rb:75:in `each'
/usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler/runtime.rb:75:in `require'
/usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler.rb:107:in `require'
/Users/rik24d/code/feedbin/config/application.rb:8:in `<top (required)>'
/usr/local/Cellar/ruby/2.4.1_1/lib/ruby/2.4.0/rubygems/core_ext/kernel_require.rb:55:in `require'
/usr/local/Cellar/ruby/2.4.1_1/lib/ruby/2.4.0/rubygems/core_ext/kernel_require.rb:55:in `require'
/Users/rik24d/code/feedbin/Rakefile:4:in `<top (required)>'
/usr/local/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/rake_module.rb:28:in `load'
/usr/local/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/rake_module.rb:28:in `load_rakefile'
/usr/local/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/application.rb:687:in `raw_load_rakefile'
/usr/local/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/application.rb:96:in `block in load_rakefile'
/usr/local/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/application.rb:178:in `standard_exception_handling'
/usr/local/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/application.rb:95:in `load_rakefile'
/usr/local/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/application.rb:79:in `block in run'
/usr/local/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/application.rb:178:in `standard_exception_handling'
/usr/local/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/application.rb:77:in `run'
/usr/local/lib/ruby/gems/2.4.0/gems/rake-12.0.0/exe/rake:27:in `<top (required)>'
/usr/local/bin/rake:22:in `load'
/usr/local/bin/rake:22:in `<main>'
TypeError: wrong argument type Class (expected Module)
/usr/local/lib/ruby/gems/2.4.0/gems/therubyracer-0.12.2/lib/v8/conversion.rb:23:in `include'
/usr/local/lib/ruby/gems/2.4.0/gems/therubyracer-0.12.2/lib/v8/conversion.rb:23:in `block (2 levels) in <top (required)>'
/usr/local/lib/ruby/gems/2.4.0/gems/therubyracer-0.12.2/lib/v8/conversion.rb:22:in `class_eval'
/usr/local/lib/ruby/gems/2.4.0/gems/therubyracer-0.12.2/lib/v8/conversion.rb:22:in `block in <top (required)>'
/usr/local/lib/ruby/gems/2.4.0/gems/therubyracer-0.12.2/lib/v8/conversion.rb:21:in `each'
/usr/local/lib/ruby/gems/2.4.0/gems/therubyracer-0.12.2/lib/v8/conversion.rb:21:in `<top (required)>'
/usr/local/lib/ruby/gems/2.4.0/gems/therubyracer-0.12.2/lib/v8.rb:22:in `require'
/usr/local/lib/ruby/gems/2.4.0/gems/therubyracer-0.12.2/lib/v8.rb:22:in `<top (required)>'
/usr/local/lib/ruby/gems/2.4.0/gems/execjs-2.7.0/lib/execjs/ruby_racer_runtime.rb:108:in `require'
/usr/local/lib/ruby/gems/2.4.0/gems/execjs-2.7.0/lib/execjs/ruby_racer_runtime.rb:108:in `available?'
/usr/local/lib/ruby/gems/2.4.0/gems/execjs-2.7.0/lib/execjs/runtimes.rb:63:in `each'
/usr/local/lib/ruby/gems/2.4.0/gems/execjs-2.7.0/lib/execjs/runtimes.rb:63:in `find'
/usr/local/lib/ruby/gems/2.4.0/gems/execjs-2.7.0/lib/execjs/runtimes.rb:63:in `best_available'
/usr/local/lib/ruby/gems/2.4.0/gems/execjs-2.7.0/lib/execjs/runtimes.rb:57:in `autodetect'
/usr/local/lib/ruby/gems/2.4.0/gems/execjs-2.7.0/lib/execjs.rb:5:in `<module:ExecJS>'
/usr/local/lib/ruby/gems/2.4.0/gems/execjs-2.7.0/lib/execjs.rb:4:in `<top (required)>'
/usr/local/lib/ruby/gems/2.4.0/gems/coffee-script-2.4.1/lib/coffee_script.rb:1:in `require'
/usr/local/lib/ruby/gems/2.4.0/gems/coffee-script-2.4.1/lib/coffee_script.rb:1:in `<top (required)>'
/usr/local/lib/ruby/gems/2.4.0/gems/coffee-script-2.4.1/lib/coffee-script.rb:1:in `require'
/usr/local/lib/ruby/gems/2.4.0/gems/coffee-script-2.4.1/lib/coffee-script.rb:1:in `<top (required)>'
/usr/local/lib/ruby/gems/2.4.0/gems/coffee-rails-4.2.1/lib/coffee-rails.rb:1:in `require'
/usr/local/lib/ruby/gems/2.4.0/gems/coffee-rails-4.2.1/lib/coffee-rails.rb:1:in `<top (required)>'
/usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler/runtime.rb:91:in `require'
/usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler/runtime.rb:91:in `block (2 levels) in require'
/usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler/runtime.rb:86:in `each'
/usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler/runtime.rb:86:in `block in require'
/usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler/runtime.rb:75:in `each'
/usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler/runtime.rb:75:in `require'
/usr/local/lib/ruby/gems/2.4.0/gems/bundler-1.14.6/lib/bundler.rb:107:in `require'
/Users/rik24d/code/feedbin/config/application.rb:8:in `<top (required)>'
/usr/local/Cellar/ruby/2.4.1_1/lib/ruby/2.4.0/rubygems/core_ext/kernel_require.rb:55:in `require'
/usr/local/Cellar/ruby/2.4.1_1/lib/ruby/2.4.0/rubygems/core_ext/kernel_require.rb:55:in `require'
/Users/rik24d/code/feedbin/Rakefile:4:in `<top (required)>'
/usr/local/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/rake_module.rb:28:in `load'
/usr/local/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/rake_module.rb:28:in `load_rakefile'
/usr/local/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/application.rb:687:in `raw_load_rakefile'
/usr/local/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/application.rb:96:in `block in load_rakefile'
/usr/local/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/application.rb:178:in `standard_exception_handling'
/usr/local/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/application.rb:95:in `load_rakefile'
/usr/local/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/application.rb:79:in `block in run'
/usr/local/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/application.rb:178:in `standard_exception_handling'
/usr/local/lib/ruby/gems/2.4.0/gems/rake-12.0.0/lib/rake/application.rb:77:in `run'
/usr/local/lib/ruby/gems/2.4.0/gems/rake-12.0.0/exe/rake:27:in `<top (required)>'
/usr/local/bin/rake:22:in `load'
/usr/local/bin/rake:22:in `<main>'

As a side note, it would great to have a Docker setup to simplify contributions.