TTRSS sync not fetching feeds, always needs "reinit"

[nico202]

• OS version: 86_64 GNU/Linux, guix 59b102c
• emacs version: GNU Emacs 28.2 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.30, cairo version 1.16.0)
• elfeed version: 3.4.1
• elfeed-protocol version: 0.8.1
• curl version: 7.85.0 (x86_64-pc-linux-gnu) libcurl/7.85.0 OpenSSL/3.0.5 zlib/1.2.11 brotli/1.0.9 zstd/1.5.2 libidn2/2.3.3 libpsl/0.21.0 (+libidn2/2.3.2) libssh/0.9.6/openssl/zlib nghttp2/1.49.0 librtmp/2.3
• server version: 22.10, hosted on NixOS (docker image and tag)
• is variable elfeed-protocol-ttrss-feeds empty: no

elfeed-log

Please collect logs in buffer *elfeed-log* with the following config before reporting issues:

(setq elfeed-log-level 'debug)
(toggle-debug-on-error)

;; for more logs if necessary
(setq elfeed-protocol-log-trace t)
(setq elfeed-protocol-fever-maxsize 5)
(setq elfeed-protocol-owncloud-maxsize 5)
(setq elfeed-protocol-ttrss-maxsize 5)
(setq elfeed-protocol-newsblur-maxpages 1)
;; I ALSO ADDED A "with data" param to the url retrive log entry to log sent data and replicate commands manually in curl

Log

[2023-02-27 14:10:43] [debug]: elfeed-protocol-ttrss: check is logged in
[2023-02-27 14:10:43] [debug]: retrieve (https://XXX.xyz/api/) with data {"op":"isLoggedIn","sid":"XXX"}
[2023-02-27 14:10:43] [debug]: elfeed-protocol-ttrss: {"seq":0,"status":0,"content":{"status":true}}
[2023-02-27 14:10:43] [debug]: elfeed-protocol-ttrss: update feed list
[2023-02-27 14:10:43] [debug]: elfeed-protocol-ttrss: update cagetory list
[2023-02-27 14:10:43] [debug]: retrieve (https://XXX.xyz/api/) with data {"op":"getCategories","sid":"XXX"}
[2023-02-27 14:10:43] [debug]: elfeed-protocol-ttrss: (CORRECT LIST OF CATEGORIES HERE)
[2023-02-27 14:10:43] [debug]: retrieve (https://XXX.xyz/api/) with data {"op":"getFeeds","sid":"XXX","cat_id":"-3"}
[2023-02-27 14:10:43] [debug]: elfeed-protocol-ttrss: (CORRECT LIST OF FEEDS HERE)
[2023-02-27 14:10:43] [debug]: elfeed-protocol-ttrss: found 18 feeds
[2023-02-27 14:10:43] [debug]: elfeed-protocol-ttrss: update entries with action update, arg 1037633,1037634,1037635,1037636,1037637
[2023-02-27 14:10:43] [debug]: retrieve (https://XXX.xyz/api/) with data {"op":"getArticle","sid":"XXX","article_id":"1037633,1037634,1037635,1037636,1037637"}
[2023-02-27 14:10:43] [debug]: elfeed-protocol-ttrss: {"seq":0,"status":0,"content":[]}
[2023-02-27 14:10:43] [debug]: elfeed-protocol-ttrss: update, parsing entries, entry-mark: 1037632
[2023-02-27 14:10:43] [debug]: elfeed-protocol-ttrss: update, parsed 0 entries(0 unread, 0 starred, min-entry-id -1, max-entry-id -1) with 0.000069s, entry-mark: 1037632

error backtrace

;; No errors

Hi! Thanks for this package! I'm having a small problem with the sync.

When running elfeed-protocol-ttrss-update, feeds are not updated. Elfeed reports last update to be days before this command. However, running elfeed-protocol-ttrss-reinit updates the articles. The list of article_id sent seems to be wrong

[fasheng]

Hi, elfeed-protocol-ttrss-update fetch latest entries IDs that generated one by one with the length defined in elfeed-protocol-ttrss-api-max-limit. Maybe your article entries are not continuous, how about run elfeed-protocol-ttrss-update multiple times until it return entries, the log will show the entry-mark increased each time.

(setq elfeed-protocol-ttrss-api-max-limit 200)

[nico202]

Maybe your article entries are not continuous Yeah I thought the same.

run elfeed-protocol-ttrss-update multiple times until it return entries, the log will show the entry-mark increased each time. Just tried but entry-mark: is not increasing:

[2023-02-27 14:44:54] [debug]: elfeed-protocol-ttrss: update, parsed 0 entries(0 unread, 0 starred, min-entry-id -1, max-entry-id -1) with 0.000077s, entry-mark: 1037632
[...]
[2023-02-27 14:45:06] [debug]: elfeed-protocol-ttrss: update, parsed 0 entries(0 unread, 0 starred, min-entry-id -1, max-entry-id -1) with 0.000090s, entry-mark: 1037632

[nico202]

Don't know if it helps, but when asking for headlines ("getHeadlines", feed_id: 0, is_cat:1), ids are in the range 1023206-1023235. Maybe it's possible to get new article ids with the getHeadlines since_id param?

[fasheng]

I see, the bug will be solved soon. And I use getArticle instead of getHeadlines here is for making fetch older entries is possible.

[nico202]

Ok thanks a lot!

[fasheng]

Besides, how about you set the max limit much bigger than 200. Maybe it could solve your problem, I'm I am more curious about the results it return :)

(setq elfeed-protocol-ttrss-api-max-limit 1000)

[fasheng]

Well, the bug is more complex than I thought. We can't just simple increase the mark ID number when it return 0 entry for there is no method to determine if the TTRSS article IDs are not continuous or there is just no new article. So here only some workarounds for you:

1. try set the max limit much bigger than 200
2. increase the update mark ID manually

   (let* (proto-id "ttrss+https://user@your-host.com")
     (elfeed-protocol-ttrss-set-update-mark
      proto-id 'update (+ elfeed-protocol-ttrss-api-max-limit 
                          (elfeed-protocol-ttrss-get-update-mark proto-id 'update))))

3. use elfeed-protocol-ttrss-reinit instead

I used to use TTRSS for a while, and never had this problem before. The article IDs may be discontinuous, but not on this scale.

[nico202]

I think there was a missing parens, used this for future reference:

(let* ((proto-id "ttrss+https://XXX.xyz"))
  (elfeed-protocol-ttrss-set-update-mark
   proto-id 'update (+ elfeed-protocol-ttrss-api-max-limit
                       (elfeed-protocol-ttrss-get-update-mark proto-id 'update))))

after setting elfeed-protocol-ttrss-api-max-limit, with no luck, and tried again a few times. Run re-init and now entry-mark: 867244. I'll wait for new articles to appear on the server and I'll try again. Thanks for your help