Question: how to show captions

[Jonghyun-Yun]

I am very excited in this package. I was able to fetch feeds. When I open it, it displays (empty) as its content. I just don't know where to look at to fix the problem. Would you kindly advise me? I am using Doom emacs on OS X if this is relevant.

[karthink]

@Jonghyun-Yun I'm assuming you have the use-package block from the Readme in your init file and that it has been evaluated.

In the entry that says "(empty)", can you try running C-u M-x elfeed-tube-fetch?

EDIT: To be sure that this doesn't work, you may have to wait for ~20 seconds after you run it.

[Jonghyun-Yun]

@karthink Thanks for the help! Yes, I am using the use-package blocks. I followed your guide, and nothing changed.

[karthink]

In that case we'll need to run some tests to see what's wrong. Can you try the following?

1. M-x load-file and load elfeed-tube-test.el
2. M-x ert and enter t as the input to run all tests.
3. Paste the output of the tests here.

[Jonghyun-Yun]

Selector: t
Passed:  4
Failed:  2 (2 unexpected)
Skipped: 1
Total:   7/7

Started at:   2022-06-16 15:25:27-0500
Finished.
Finished at:  2022-06-16 15:25:53-0500

F...F.s

F elfeed-tube--caps-fetch-test
    Test if captions are retrieved correctly for Youtube videos.
    (wrong-type-argument number-or-marker-p nil)

F elfeed-tube--fetch-1-test
    Fetch all available data for an Elfeed entry
    (wrong-type-argument number-or-marker-p nil)

[karthink]

Thanks, that's helpful! In the results buffer for the tests, can you place the cursor over either failed test and press b? This will give us a backtrace with the error. You can paste that here.

Also I assume you're on the latest commit? I recently addressed a similar error in #2.

[Jonghyun-Yun]

Yes, it's the latest commit. Please find the backtrace below.

Backtrace for test ‘elfeed-tube--caps-fetch-test’:
  signal(wrong-type-argument (number-or-marker-p nil))
  #f(compiled-function () #<bytecode -0x1a47d4fb57190e7e>)()
  aio-wait-for(#s(aio-promise #[0 "\301\300\242@\300\242A\"\207" [((wrong-type-argument number-or-marker-p nil)) signal] 3] nil))
  (let* ((entry (record 'elfeed-entry '("www.youtube.com" . "yt:video:Pj-h6MEgE7I") "You Are Not Where You Think You Are" "https://www.youtube.com/watch?v=Pj-h6MEgE7I" 1652795984.0 nil 'html nil '(youtube) "https://www.youtube.com/feeds/videos.xml?channel_i..." '(:authors ((:name "Kurzgesagt – In a Nutshell" :uri "https://www.youtube.com/channel/UCsXVk37bltHxD1rDP..."))))) (caption-tracks (aio-wait-for (elfeed-tube--fetch-captions-tracks entry)))) (let* ((fn-135 #'cl-set-difference) (args-136 (condition-case err (let ((signal-hook-function ...)) (list (mapcar ... caption-tracks) '... :test #'string=)) (error (progn (setq fn-135 ...) (list ... ...)))))) (let ((value-137 'ert-form-evaluation-aborted-138)) (let (form-description-139) (if (not (unwind-protect (setq value-137 ...) (setq form-description-139 ...) (ert--signal-should-execution form-description-139))) nil (ert-fail form-description-139))) value-137)) (let* ((fn-140 #'cl-set-difference) (args-141 (condition-case err (let ((signal-hook-function ...)) (list (mapcar ... caption-tracks) '... :test #'string=)) (error (progn (setq fn-140 ...) (list ... ...)))))) (let ((value-142 'ert-form-evaluation-aborted-143)) (let (form-description-144) (if (not (unwind-protect (setq value-142 ...) (setq form-description-144 ...) (ert--signal-should-execution form-description-144))) nil (ert-fail form-description-144))) value-142)) (let* ((elfeed-tube-captions-languages '("english" "english (auto generated)")) (caption-xml (aio-wait-for (elfeed-tube--fetch-captions-url caption-tracks entry))) caption-sexp) (let* ((fn-145 #'string=) (args-146 (condition-case err (let (...) (list ... "English")) (error (progn ... ...))))) (let ((value-147 'ert-form-evaluation-aborted-148)) (let (form-description-149) (if (unwind-protect (setq value-147 ...) (setq form-description-149 ...) (ert--signal-should-execution form-description-149)) nil (ert-fail form-description-149))) value-147)) (let ((value-150 (gensym "ert-form-evaluation-aborted-"))) (let (form-description-151) (if (unwind-protect (setq value-150 (and ... ...)) (setq form-description-151 (list ... :form ... :value value-150)) (ert--signal-should-execution form-description-151)) nil (ert-fail form-description-151))) value-150) (setq caption-sexp (let ((temp-buffer (generate-new-buffer " *temp*" t))) (save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn (save-excursion ...) (libxml-parse-xml-region ... ...)) (and (buffer-name temp-buffer) (kill-buffer temp-buffer)))))) (let* ((fn-152 #'cl-mismatch) (args-153 (condition-case err (let (...) (list ... ...)) (error (progn ... ...))))) (let ((value-154 'ert-form-evaluation-aborted-155)) (let (form-description-156) (if (not (unwind-protect ... ... ...)) nil (ert-fail form-description-156))) value-154))))
  (let ((lexical-binding nil)) (let* ((entry (record 'elfeed-entry '("www.youtube.com" . "yt:video:Pj-h6MEgE7I") "You Are Not Where You Think You Are" "https://www.youtube.com/watch?v=Pj-h6MEgE7I" 1652795984.0 nil 'html nil '(youtube) "https://www.youtube.com/feeds/videos.xml?channel_i..." '(:authors (...)))) (caption-tracks (aio-wait-for (elfeed-tube--fetch-captions-tracks entry)))) (let* ((fn-135 #'cl-set-difference) (args-136 (condition-case err (let (...) (list ... ... :test ...)) (error (progn ... ...))))) (let ((value-137 'ert-form-evaluation-aborted-138)) (let (form-description-139) (if (not (unwind-protect ... ... ...)) nil (ert-fail form-description-139))) value-137)) (let* ((fn-140 #'cl-set-difference) (args-141 (condition-case err (let (...) (list ... ... :test ...)) (error (progn ... ...))))) (let ((value-142 'ert-form-evaluation-aborted-143)) (let (form-description-144) (if (not (unwind-protect ... ... ...)) nil (ert-fail form-description-144))) value-142)) (let* ((elfeed-tube-captions-languages '("english" "english (auto generated)")) (caption-xml (aio-wait-for (elfeed-tube--fetch-captions-url caption-tracks entry))) caption-sexp) (let* ((fn-145 #'string=) (args-146 (condition-case err (let ... ...) (error ...)))) (let ((value-147 'ert-form-evaluation-aborted-148)) (let (form-description-149) (if (unwind-protect ... ... ...) nil (ert-fail form-description-149))) value-147)) (let ((value-150 (gensym "ert-form-evaluation-aborted-"))) (let (form-description-151) (if (unwind-protect (setq value-150 ...) (setq form-description-151 ...) (ert--signal-should-execution form-description-151)) nil (ert-fail form-description-151))) value-150) (setq caption-sexp (let ((temp-buffer (generate-new-buffer " *temp*" t))) (save-current-buffer (set-buffer temp-buffer) (unwind-protect (progn ... ...) (and ... ...))))) (let* ((fn-152 #'cl-mismatch) (args-153 (condition-case err (let ... ...) (error ...)))) (let ((value-154 'ert-form-evaluation-aborted-155)) (let (form-description-156) (if (not ...) nil (ert-fail form-description-156))) value-154)))))
  (lambda nil (let ((lexical-binding nil)) (let* ((entry (record 'elfeed-entry '("www.youtube.com" . "yt:video:Pj-h6MEgE7I") "You Are Not Where You Think You Are" "https://www.youtube.com/watch?v=Pj-h6MEgE7I" 1652795984.0 nil 'html nil '(youtube) "https://www.youtube.com/feeds/videos.xml?channel_i..." '(:authors ...))) (caption-tracks (aio-wait-for (elfeed-tube--fetch-captions-tracks entry)))) (let* ((fn-135 #'cl-set-difference) (args-136 (condition-case err (let ... ...) (error ...)))) (let ((value-137 'ert-form-evaluation-aborted-138)) (let (form-description-139) (if (not ...) nil (ert-fail form-description-139))) value-137)) (let* ((fn-140 #'cl-set-difference) (args-141 (condition-case err (let ... ...) (error ...)))) (let ((value-142 'ert-form-evaluation-aborted-143)) (let (form-description-144) (if (not ...) nil (ert-fail form-description-144))) value-142)) (let* ((elfeed-tube-captions-languages '("english" "english (auto generated)")) (caption-xml (aio-wait-for (elfeed-tube--fetch-captions-url caption-tracks entry))) caption-sexp) (let* ((fn-145 #'string=) (args-146 (condition-case err ... ...))) (let ((value-147 ...)) (let (form-description-149) (if ... nil ...)) value-147)) (let ((value-150 (gensym "ert-form-evaluation-aborted-"))) (let (form-description-151) (if (unwind-protect ... ... ...) nil (ert-fail form-description-151))) value-150) (setq caption-sexp (let ((temp-buffer ...)) (save-current-buffer (set-buffer temp-buffer) (unwind-protect ... ...)))) (let* ((fn-152 #'cl-mismatch) (args-153 (condition-case err ... ...))) (let ((value-154 ...)) (let (form-description-156) (if ... nil ...)) value-154))))))()
  ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test :name elfeed-tube--caps-fetch-test :documentation "Test if captions are retrieved correct..." :body (lambda nil (let (...) (let* ... ... ... ...))) :most-recent-result #s(ert-test-failed :messages "" :should-forms nil :duration 24.042237 :condition (wrong-type-argument number-or-marker-p nil) :backtrace (#s(backtrace-frame :evald t :fun signal :args ... :flags nil :locals ... :buffer nil :pos nil) #s(backtrace-frame :evald t :fun #f(compiled-function () #<bytecode -0x1a47d4fb57190e7e>) :args nil :flags nil :locals nil :buffer nil :pos nil) #s(backtrace-frame :evald t :fun aio-wait-for :args ... :flags nil :locals nil :buffer nil :pos nil) #s(backtrace-frame :evald nil :fun let* :args ... :flags nil :locals ... :buffer nil :pos nil) #s(backtrace-frame :evald nil :fun let :args ... :flags nil :locals ... :buffer nil :pos nil) #s(backtrace-frame :evald t :fun ... :args nil :flags nil :locals nil :buffer nil :pos nil) #s(backtrace-frame :evald t :fun ert--run-test-internal :args #0 :flags nil :locals ... :buffer nil :pos nil) #s(backtrace-frame :evald t :fun ert-run-test :args ... :flags nil :locals ... :buffer nil :pos nil) #s(backtrace-frame :evald t :fun ert-run-or-rerun-test :args ... :flags nil :locals ... :buffer nil :pos nil) #s(backtrace-frame :evald t :fun ert-run-tests :args ... :flags nil :locals ... :buffer nil :pos nil) #s(backtrace-frame :evald t :fun ert :args ... :flags nil :locals nil :buffer nil :pos nil) #s(backtrace-frame :evald t :fun funcall-interactively :args ... :flags nil :locals nil :buffer nil :pos nil) #s(backtrace-frame :evald t :fun command-execute :args ... :flags nil :locals ... :buffer nil :pos nil) #s(backtrace-frame :evald t :fun execute-extended-command :args ... :flags nil :locals ... :buffer nil :pos nil) #s(backtrace-frame :evald t :fun funcall-interactively :args ... :flags nil :locals nil :buffer nil :pos nil) #s(backtrace-frame :evald t :fun command-execute :args ... :flags nil :locals ... :buffer nil :pos nil)) :infos nil) :expected-result-type :passed :tags nil) :result #s(ert-test-failed :messages "" :should-forms nil :duration 24.042237 :condition (wrong-type-argument number-or-marker-p nil) :backtrace (#s(backtrace-frame :evald t :fun signal :args (wrong-type-argument ...) :flags nil :locals (... ... ... ...) :buffer nil :pos nil) #s(backtrace-frame :evald t :fun #f(compiled-function () #<bytecode -0x1a47d4fb57190e7e>) :args nil :flags nil :locals nil :buffer nil :pos nil) #s(backtrace-frame :evald t :fun aio-wait-for :args (#s(aio-promise #[0 "\301\300\242@\300\242A\"\207" [((wrong-type-argument number-or-marker-p nil)) signal] 3] nil)) :flags nil :locals nil :buffer nil :pos nil) #s(backtrace-frame :evald nil :fun let* :args (... ... ... ...) :flags nil :locals (...) :buffer nil :pos nil) #s(backtrace-frame :evald nil :fun let :args (... ...) :flags nil :locals (...) :buffer nil :pos nil) #s(backtrace-frame :evald t :fun (lambda nil ...) :args nil :flags nil :locals nil :buffer nil :pos nil) #s(backtrace-frame :evald t :fun ert--run-test-internal :args #0 :flags nil :locals (... ... ... ... ... ...) :buffer nil :pos nil) #s(backtrace-frame :evald t :fun ert-run-test :args (...) :flags nil :locals (... ... ...) :buffer nil :pos nil) #s(backtrace-frame :evald t :fun ert-run-or-rerun-test :args (... ... #f(compiled-function (event-type &rest event-args) #<bytecode -0x1ed2d7806087c88>)) :flags nil :locals (...) :buffer nil :pos nil) #s(backtrace-frame :evald t :fun ert-run-tests :args (t #f(compiled-function (event-type &rest event-args) #<bytecode -0x1ed2d7806087c88>) t) :flags nil :locals (...) :buffer nil :pos nil) #s(backtrace-frame :evald t :fun ert :args (t nil) :flags nil :locals nil :buffer nil :pos nil) #s(backtrace-frame :evald t :fun funcall-interactively :args (ert t nil) :flags nil :locals nil :buffer nil :pos nil) #s(backtrace-frame :evald t :fun command-execute :args (ert record) :flags nil :locals (...) :buffer nil :pos nil) #s(backtrace-frame :evald t :fun execute-extended-command :args (nil "ert" ...) :flags nil :locals (...) :buffer nil :pos nil) #s(backtrace-frame :evald t :fun funcall-interactively :args (execute-extended-command nil "ert" ...) :flags nil :locals nil :buffer nil :pos nil) #s(backtrace-frame :evald t :fun command-execute :args (execute-extended-command) :flags nil :locals (...) :buffer nil :pos nil)) :infos nil) :exit-continuation #f(compiled-function () #<bytecode 0x1e1fb9fae42683>) :next-debugger debug :ert-debug-on-error nil))
  ert-run-test(#s(ert-test :name elfeed-tube--caps-fetch-test :documentation "Test if captions are retrieved correctly for..." :body (lambda nil (let ((lexical-binding nil)) (let* ((entry ...) (caption-tracks ...)) (let* (... ...) (let ... ... value-137)) (let* (... ...) (let ... ... value-142)) (let* (... ... caption-sexp) (let* ... ...) (let ... ... value-150) (setq caption-sexp ...) (let* ... ...))))) :most-recent-result #s(ert-test-failed :messages "" :should-forms nil :duration 24.042237 :condition (wrong-type-argument number-or-marker-p nil) :backtrace (#s(backtrace-frame :evald t :fun signal :args (wrong-type-argument (number-or-marker-p nil)) :flags nil :locals ((debugger-may-continue . t) (inhibit-redisplay) (inhibit-debugger . t) (inhibit-changing-match-data)) :buffer nil :pos nil) #s(backtrace-frame :evald t :fun #f(compiled-function () #<bytecode -0x1a47d4fb57190e7e>) :args nil :flags nil :locals nil :buffer nil :pos nil) #s(backtrace-frame :evald t :fun aio-wait-for :args (#s(aio-promise #[0 "\301\300\242@\300\242A\"\207" [((wrong-type-argument number-or-marker-p nil)) signal] 3] nil)) :flags nil :locals nil :buffer nil :pos nil) #s(backtrace-frame :evald nil :fun let* :args ((... ...) (let* ... ...) (let* ... ...) (let* ... ... ... ... ...)) :flags nil :locals ((entry . ...)) :buffer nil :pos nil) #s(backtrace-frame :evald nil :fun let :args ((...) (let* ... ... ... ...)) :flags nil :locals ((lexical-binding)) :buffer nil :pos nil) #s(backtrace-frame :evald t :fun (lambda nil (let ... ...)) :args nil :flags nil :locals nil :buffer nil :pos nil) #s(backtrace-frame :evald t :fun ert--run-test-internal :args (#s(ert--test-execution-info :test #1 :result #2 :exit-continuation #f(compiled-function () #<bytecode 0x1e1fb9fae42683>) :next-debugger debug :ert-debug-on-error nil)) :flags nil :locals ((ert--infos) (debug-ignored-errors) (debug-on-quit . t) (backtrace-on-error-noninteractive) (debug-on-error . t) (debugger . #f(compiled-function (&rest args) #<bytecode 0x12bd203f05612608>))) :buffer nil :pos nil) #s(backtrace-frame :evald t :fun ert-run-test :args #0 :flags nil :locals ((ert--running-tests #1) (message-log-max . t) (ert--should-execution-observer . #f(compiled-function (form-description) #<bytecode 0x1eca1c0c9f69cf26>))) :buffer nil :pos nil) #s(backtrace-frame :evald t :fun ert-run-or-rerun-test :args (#s(ert--stats :selector t :tests ... :test-map #<hash-table eql 7/7 0x1fff3ec52eef> :test-results ... :test-start-times ... :test-end-times ... :passed-expected 4 :passed-unexpected 0 :failed-expected 0 :failed-unexpected 2 :skipped 1 :start-time ... :end-time ... :aborted-p nil :current-test nil :next-redisplay 1655413809.270264) #1 #f(compiled-function (event-type &rest event-args) #<bytecode -0x1ed2d7806087c88>)) :flags nil :locals ((ert--current-run-stats . ...)) :buffer nil :pos nil) #s(backtrace-frame :evald t :fun ert-run-tests :args (t #f(compiled-function (event-type &rest event-args) #<bytecode -0x1ed2d7806087c88>) t) :flags nil :locals ((ert--current-run-stats . ...)) :buffer nil :pos nil) #s(backtrace-frame :evald t :fun ert :args (t nil) :flags nil :locals nil :buffer nil :pos nil) #s(backtrace-frame :evald t :fun funcall-interactively :args (ert t nil) :flags nil :locals nil :buffer nil :pos nil) #s(backtrace-frame :evald t :fun command-execute :args (ert record) :flags nil :locals ((current-minibuffer-command . ert)) :buffer nil :pos nil) #s(backtrace-frame :evald t :fun execute-extended-command :args (nil "ert" #("er" 0 2 ...)) :flags nil :locals ((prefix-arg)) :buffer nil :pos nil) #s(backtrace-frame :evald t :fun funcall-interactively :args (execute-extended-command nil "ert" #("er" 0 2 ...)) :flags nil :locals nil :buffer nil :pos nil) #s(backtrace-frame :evald t :fun command-execute :args (execute-extended-command) :flags nil :locals ((current-minibuffer-command . execute-extended-command)) :buffer nil :pos nil)) :infos nil) :expected-result-type :passed :tags nil))
  ert-run-or-rerun-test(#s(ert--stats :selector t :tests ... :test-map #<hash-table eql 7/7 0x1fff3ec52eef> :test-results ... :test-start-times ... :test-end-times ... :passed-expected 4 :passed-unexpected 0 :failed-expected 0 :failed-unexpected 2 :skipped 1 :start-time ... :end-time ... :aborted-p nil ...) #s(ert-test :name elfeed-tube--caps-fetch-test :documentation "Test if captio..." :body ... :most-recent-result ... :expected-result-type :passed :tags nil) #f(compiled-function (event-type &rest event-args) #<bytecode -0x1ed2d7806087c88>))
  ert-run-tests(t #f(compiled-function (event-type &rest event-args) #<bytecode -0x1ed2d7806087c88>) t)
  ert(t nil)
  funcall-interactively(ert t nil)
  command-execute(ert record)
  execute-extended-command(nil "ert" #("er" 0 2 (ws-butler-chg chg)))
  funcall-interactively(execute-extended-command nil "ert" #("er" 0 2 (ws-butler-chg chg)))
  command-execute(execute-extended-command)

[karthink]

Quick check: Can you open the '(empty)' Elfeed entry, then

1. Run M-x eval-expression or M-:
2. Run (elfeed-tube--youtube-p elfeed-show-entry), and
3. Run (elfeed-tube--get-video-id elfeed-show-entry)?

[Jonghyun-Yun]

Sure! The second give 0 regardless of entries. The third gives the correct ID of the YouTube video in the entry

[karthink]

All right. Can you try (elfeed-tube--gethash elfeed-show-entry) the same way?

EDIT: modified the elisp code.

[Jonghyun-Yun]

Sure, I got nil for all entries.

[karthink]

Okay. I'm still trying to figure out what could be going wrong for you. Some more questions:

1. Are you using elfeed-protocol or some other nonstandard way of fetching feeds into Elfeed?
2. What is your system locale set to and what language does Youtube use to show you its interface? (i.e. your region/browser localization)
3. Can you try running the following in a scratch buffer?

(let* ((url "https://www.youtube.com/watch?v=Pj-h6MEgE7I") (resp (aio-wait-for (elfeed-tube-curl-enqueue url :method "GET")))) (print (plist-get resp :status-code) (current-buffer)) (print (plist-get resp :success) (current-buffer)) (print (plist-get resp :error-message) (current-buffer)))

[Jonghyun-Yun]

Thanks again for the help. I'm using elfeed and elfeed-org packages to fetch feeds. The language is set to en-US. Lastly, the below is the results of the lisp block.

200

t

nil

[karthink]

Can you try running the caps-fetch test again like above?

1. M-x load-file and load elfeed-tube-test.el
2. M-x ert and choose elfeed-tube--caps-fetch-test
3. When it fails, place the cursor over the test and press l (this runs ert--results-test-at-point-no-redefinition)

The output of this should tell us exactly which assertion is failing.

[Jonghyun-Yun]

Sure, this is what I got.

‘should’ forms executed during test ‘elfeed-tube--caps-fetch-test’:

(Values are shallow copies and may have looked different during the test if they
have been modified destructively.)

(No should forms during this test.)

[Jonghyun-Yun]

After recent changes in both Doom Emacs and elfeed-tube, I can see the captions now. Thanks for your great work!

[unhammer]

I thought I had the same bug since I tried installing and M-x elfeed-tube-fetch didn't show captions, but it seems I need to do (elfeed-tube-setup) first. Is there any reason elfeed-tube-fetch couldn't call that automatically if it hasn't been done yet?

[karthink]

@unhammer Oversight, I added it, thanks.