search

karthink / elfeed-tube

Youtube integration for Elfeed, the feed reader for Emacs
The Unlicense
241 stars 11 forks source link

readme

+title: Elfeed Tube: Youtube on /your/ terms

:BADGE: [[https://melpa.org/#/elfeed-tube][file:https://melpa.org/packages/elfeed-tube-badge.svg]] :END:

Elfeed Tube is an Emacs package for a richer, interactive, noise-free and fully text-capable interface to your Youtube subscriptions and playlists using Elfeed, the RSS feed reader for Emacs. Think of it as supercharging Elfeed, or perhaps a taste of what the RSS/Atom protocol /could have become today/ if it had continue to evolve.

Elfeed Tube adds video descriptions, metadata and "live" transcripts (see below) for all Youtube video entries in Elfeed. Here is what a video entry looks like:

https://imgur.com/a/taiMOqG

+html:

([[https://i.imgur.com/nFK0n3m.png][Higher quality version]])

Here is the same Youtube video entry in Elfeed without Elfeed Tube:

+html:

Elfeed Tube is useful for any Youtube RSS feed, but it works particularly well for channels/playlists with longer form videos, podcasts, technical talks and lectures. For an introduction to Elfeed and RSS feeds, see [[https://www.youtube.com/watch?v=oHfAht9uTx8][this video]] by Protesilaos Stavrou.

** Description and other metadata Elfeed Tube adds the video duration, thumbnail, video description and chapters to each Youtube entry in Elfeed. These elements can be turned off individually or tweaked to your liking.

For example, you may prefer to avoid including video descriptions because they often have linkspam, or prefer larger thumbnails.

Youtube transcripts, including auto-generated ones, are automatically downloaded in your language of choice, formatted and displayed in your Elfeed buffers:

+html:

+html:

** Noise-free (No Ad reads) There are no sponsor reads, ads or promotions in the transcript. These are automatically recognized and either greyed out or hidden from view. Here's an Ad read section that you now don't have to read:

+html:

[[https://i.imgur.com/TUn4xmb.jpeg][Here]] is a more extensive version.

https://imgur.com/a/Oyly62g

You can turn this feature off, or add more recognized sections to the strike-list, like intros and outros.

** Interactive Clicking on a transcript segment opens the video at that time in your browser or video player of choice:

+html:

You can also move your cursor there and press return (~RET~). The tool-tip provides more help. ** Navigable with =imenu= Video chapters, when available, are included as headings in the transcript. The entry can be navigated through =imenu=. Here I jump between descriptions of different movies in this video:

+html:

** "Live" With MPV integration (optional), you can seek to any time in the video by clicking at the corresponding text in your buffer:

+html:

https://user-images.githubusercontent.com/8607532/172324761-66a1aed8-2d09-48d4-8673-86b5a13ffc4d.mp4

You can jump in Emacs to the current video position with =elfeed-tube-mpv-where= (~C-c C-w~):

+html:

Finally, you can also follow along with the video in Emacs:

+html:

+html:

+html:

+html:

To do this, turn on =elfeed-tube-mpv-follow-mode= (~C-c C-f~). When this mode is active you can pause video playback with ~SPC~. Clicking on any segment in the transcript will still seek to that point.

You can combine =elfeed-tube-mpv-follow-mode= with other Emacs buffer modes, like Emacs' =follow-mode= that shows a contiguous buffer in two windows:

+html:

This feature does not lock up Emacs. You can work in a different window while tracking updates in this one.

** Archivable and retrievable The information fetched by Elfeed Tube, including the transcripts (with timing information) is associated with the feed and stored in your Elfeed database like any other text entry. You can bookmark it in Emacs, refer to it in your other notes and so on.

The Elfeed database is plain text, optionally gzipped. You can be sure that it will be readable decades from now.

** Searchable and Org-linkable (It's just an Emacs buffer) You can text search your transcripts, Org-link them, quote from them, etc. Tagging support, date-oriented and metadata-based search is included as part of Elfeed.

When transcripts are available in some form, which is most of the time, they can be your primary entry and reference point for the video. This works especially well with technical content, tutorials, talks and podcasts.

/NOTE/: It would be fantastic to be able to search for caption text across your entire Elfeed database instead of inside an entry, but Elfeed does not (yet) support full-text search. You'll have to search for entries by supported fields instead, through a combination of the entry date range, tags, entry title and channel name etc.

** Putting it together Combined with the text manipulation capabilities of Emacs, you can combine these features into a very accessible video interface. You can handle video partially like you handle text:

MPV integration with the live transcripts is provided separately as =elfeed-tube-mpv=, you can install it with =M-x package-install⮐= =elfeed-tube-mpv=.

Requirements:

Dependencies (automatically installed):

For "live" transcripts with =elfeed-tube-mpv=:

** Set up with =use-package=

+BEGIN_SRC emacs-lisp

(use-package elfeed-tube :ensure t ;; or :straight t :after elfeed :demand t :config ;; (setq elfeed-tube-auto-save-p nil) ; default value ;; (setq elfeed-tube-auto-fetch-p t) ; default value (elfeed-tube-setup)

:bind (:map elfeed-show-mode-map
       ("F" . elfeed-tube-fetch)
       ([remap save-buffer] . elfeed-tube-save)
       :map elfeed-search-mode-map
       ("F" . elfeed-tube-fetch)
       ([remap save-buffer] . elfeed-tube-save)))

+END_SRC

If you want "live" captions and better MPV support:

+BEGIN_SRC emacs-lisp

(use-package elfeed-tube-mpv :ensure t ;; or :straight t :bind (:map elfeed-show-mode-map ("C-c C-f" . elfeed-tube-mpv-follow-mode) ("C-c C-w" . elfeed-tube-mpv-where)))

+END_SRC

** TODO Set up without =use-package=

+BEGIN_SRC emacs-lisp

(require 'elfeed-tube) (elfeed-tube-setup) (define-key elfeed-show-mode-map (kbd "F") 'elfeed-tube-fetch) (define-key elfeed-show-mode-map [remap save-buffer] 'elfeed-tube-save) (define-key elfeed-search-mode-map (kbd "F") 'elfeed-tube-fetch) (define-key elfeed-search-mode-map [remap save-buffer] 'elfeed-tube-save))

+END_SRC

If you want "live" captions and better MPV support:

+BEGIN_SRC emacs-lisp

(require 'elfeed-tube-mpv) (define-key elfeed-show-mode-map (kbd "C-c C-f") 'elfeed-tube-mpv-follow-mode) (define-key elfeed-show-mode-map (kbd "C-c C-w") 'elfeed-tube-mpv-where)

+END_SRC

Elfeed Tube tries its best to work out of the box with no set up, but Step I below is unavoidable if you're starting fresh.

** Step I: Add Youtube subscriptions to Elfeed Fortunately, Youtube still provides RSS feeds for channels and playlists. Unfortunately, Youtube doesn't make it easy to find them.

Elfeed Tube provides a helper function: =M-x elfeed-tube-add-feeds= to find the RSS feeds (asynchronously) for channels or playlists. When given one ore more Youtube video/playlist/channel URLs or plain text search terms, it will:

https://user-images.githubusercontent.com/8607532/172195966-33089c16-57b1-4bbc-9bcc-71ac3d4338bc.mp4

(Finding the feeds is also asynchronous)

Examples (=RET= means pressing return):

Queries are separated by Emacs' =crm-separator=, which is comma (,) by default. Be warned: URLs are safer, plain text queries might find the wrong channels!

When called noninteractively, it can accept a list of URLs or queries:

+BEGIN_SRC emacs-lisp

(elfeed-tube-add-feeds '("veritasium" "https://www.youtube.com/playlist?list=PLEoMzSkcN8oMc34dTjyFmTUWbXTKrNfZA" "quanta magazine" "julia computing" "https://www.youtube.com/watch?v=bSVfItpvG5Q" "https://youtu.be/7CM7Ef-dPWQ" "tom scott"))

+END_SRC

See the docstring for more options.

*** Other ways to find feeds You can use a web service like https://rssbox.herokuapp.com/, or look in the HTML of a Youtube channel page if you like your web browser's element inspector!

* Step II: Use Elfeed as usual That's it.* Assuming you've run =(elfeed-tube-setup)=, included in the above use-package block, there's nothing else to do.

If you're new to Elfeed, you can start with =M-x elfeed=.

In case the fetch for a Youtube entry fails you can call =M-x elfeed-tube-fetch= with a prefix argument (~C-u F~ or ~C-u M-x elfeed-tube-fetch~) to force a refetch.

** (Optional) Back-fill YouTube feeds

While Elfeed is a vault, Youtube RSS feeds only contain the last 15 videos from a channel. So you will only collect entries for videos from a little before when you subscribe to a channel's feed.

Elfeed Tube provides a command to add all historical entries for Youtube feeds to Elfeed. Call =elfeed-tube-fill-feeds= in an Elfeed search or entry buffer to fetch all entries for the feed at point. You can do this for many feeds at once by selecting a region of entries.

+html:

[[https://user-images.githubusercontent.com/8607532/184994136-2a36f81c-bb47-470f-86ca-0efd778f3b03.png]]

** (Optional) For "Live" captions with MPV For a "live" connection between the transcript and MPV:

Additionally, playback with "live" transcripts will behave as expected with buffer narrowing.

** (Optional) View entries instead of auto-saving them Not all Youtube videos contain gems of wisdom, to put it mildly. You may thus want to only /view/ video information instead of /adding/ it directly to the Elfeed database. This is especially salient since there's no (user-facing) way to delete items in Elfeed.

To do this you can set

+BEGIN_SRC emacs-lisp

(setq elfeed-tube-auto-save-p nil) ;This is the default value

+END_SRC

Now Youtube entries you view will feature a =[NOT SAVED]= marker:

+html:

The fetched info will be cached for this Emacs session. Entries that you deem useful (such as the handy Guix tutorial in the above entry) can be added to your Elfeed database by clicking on this marker, or with your =save-buffer= keybinding (typically ~C-x C-s~):

+html:

You can change the marker style through =elfeed-tube-save-indicator= if you'd like a more subdued indicator.

** (Optional) Fetch manually Finally, you can also disable auto-fetching data and call =M-x elfeed-tube-fetch= (bound to ~F~) from an Elfeed Search or Show buffer to fetch selectively. To do this, set

+BEGIN_SRC emacs-lisp

(setq elfeed-tube-auto-fetch-p nil)

+END_SRC

=M-x elfeed-tube-fetch= will fetch info for all Youtube entries in a selected region in an Elfeed Search buffer.

You can independently control auto/manual fetching of info and auto/manual saving of info to the Elfeed database.

(Optional) Elfeed Tube for any video You can call =elfeed-tube-fetch= outside of Elfeed to read an abritrary Youtube video URL (not just feed entries) and produce an Elfeed-like entry buffer for it with description, other metadata and live transcripts etc. Essentially you can use Elfeed Tube as a Youtube viewer. This feature is currently experimental. Commands summary =elfeed-tube= commands: | Command | Description | Where | suggested key-binding | |-----------------------------+----------------------------------------------+--------------------+-----------------------| | =elfeed-tube-add-feeds= | Find youtube feeds from search queries | Anywhere in Emacs | N/A | | =elfeed-tube-fetch= | Fetch info for any video | Anywhere in Emacs | N/A | | =elfeed-tube-fetch= (again) | Fetch video info for Elfeed entry (optional) | Anywhere in Elfeed | ~F~ | | =elfeed-tube-save= | Manually save entry to Elfeed DB (optional) | In an Elfeed entry | ~C-x C-s~ | | =elfeed-tube-fill-feeds= | Back-fill Youtube feed entries | Anywhere in Elfeed | N/A |

=elfeed-tube-mpv= commands, for live transcripts | Command | Description | Where | suggested key-binding | |-----------------------------+-----------------------------------------------+--------------------+-------------------------| | =elfeed-tube-mpv= | Open video at time or seek video to this time | In an Elfeed entry | ~mouse-1~, ~mouse-2~ or ~RET~ | | =elfeed-tube-mpv-where= | Jump to current video location in transcript | In an Elfeed entry | ~C-c C-w~ | | =elfeed-tube-mpv-follow-mode= | Continuously track a video playing in MPV | In an Elfeed entry | ~C-c C-f~ |

** Disabling fields Customize =elfeed-tube-fields=. To show only the duration and captions but no description, thumbnail or chapter locations:

+BEGIN_SRC emacs-lisp

;; Other options: thumbnail, description, chapters (setq elfeed-tube-fields '(duration captions))

+END_SRC

** Customizing fields

Thumbnails Control the size with =elfeed-tube-thumbnail-size=. Captions **** Languages =elfeed-tube-captions-languages=: Language preference. The first available matching transcript will be fetched:

+BEGIN_SRC emacs-lisp

;; Arabic or English or auto generated English captions (setq elfeed-tube-captions-languages '("ar" "en" "english (auto generated)"))

+END_SRC

**** Sponsored segments

** Persistence Set the boolean =elfeed-tube-auto-save-p= to =t= to automatically save fetched information to the Elfeed database.

The boolean =elfeed-tube-save-indicator= controls the style of indicator used to indicate unsaved content.

Yes, this does mean that the fetcher code is going to need updating often. Them's the breaks. *** Can I use the transcripts feature without installing Elfeed?

Not at present. Elfeed Tube depends on Elfeed to do a lot of the lifting.

However you can use it with any Youtube video, see: [[#optional-elfeed-tube-for-any-video][(Optional) Elfeed Tube for any video]]. If you're interested in creating a stand alone package for this please go ahead. *** What about these Youtube features? No support is currently planned for

Elfeed Tube is not a Youtube client for Emacs.

Instead, it plays to the strengths of RSS: to provide you with a regular digest of self-curated content in a more accessible way than otherwise possible. So search is not planned (see alternatives).

Video recommendations are sometimes useful for discovery, but in their current form they're designed primarily to keep you watching Youtube and often end in rabbit-holeing and doom spirals. Helping myself avoid these behaviors is an explicit goal of Elfeed Tube!

The top comments are occasionally useful, especially on technical videos. I might add support for these in the future, although none is planned as of now.

** TODO Fix persistence bug It disappears from the index first, then from the content store when =elfeed-db-gc= runs.

TODO Specify what data to fetch per feed instead of globally. TODO Elfeed search keyword for video duration: ("=crafters <25min" should return videos in the Elfeed DB from the "System Crafters" Youtube channel feed that are under 25 minutes long, etc.)

TODO Support for Youtube's official API TODO Re-punctuating auto-generated captions Using a [[https://github.com/ottokart/punctuator2][punctuator2]] web service

TODO Backfill Youtube feeds TODO Download features [0/3] :PROPERTIES: :COOKIE_DATA: todo recursive :END: *** TODO Download button?

** DONE Chapter extraction CLOSED: [2022-06-18 Sat 23:07] From the descriptions.

** DONE =imenu= support [2/2] CLOSED: [2022-06-18 Sat 22:58] Headings:

** Local

Local Variables:

eval: (when (featurep 'toc-org-mode) (toc-org-mode 1))

End: