Feature Request: Invidious Public Instances Redirect From YouTube

[80aff123-9163-4f3e-a93d-4a2f35af9be1]

How could I add a function that with a single keyboard shortcut opens the modeline and redirects me from youtube.com to choose amongst one of several online invidious instances?

I would like the modeline to sort the instances by current health of the site. See here or the dailyRatios and weeklyRatio json fields below for an example of what I mean about site health.

Any help or suggestions are greatly appreciated! A version of this program for online peertube instances would be cool too.

JSON code showing Invidious Instances as of Today:

[
  [
    "invidious.snopyta.org",
    {
      "flag": "🇫🇮",
      "region": "FI",
      "stats": {
        "version": "2.0",
        "software": {
          "name": "invidious",
          "version": "0.20.1-13f58d6",
          "branch": "master"
        },
        "openRegistrations": true,
        "usage": {
          "users": {
            "total": 3056,
            "activeHalfyear": 2546,
            "activeMonth": 1729
          }
        },
        "metadata": {
          "updatedAt": 1599289689,
          "lastChannelRefreshedAt": 1599289686
        }
      },
      "type": "https",
      "uri": "https://invidious.snopyta.org/",
      "monitor": {
        "monitorId": 782758961,
        "statusClass": "success",
        "weeklyRatio": {
          "ratio": "99.367",
          "label": "success"
        },
        "name": "invidious.snopyta.org",
        "type": "http(s)",
        "dailyRatios": [
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "95.569",
            "label": "warning"
          },
          {
            "ratio": "100.00",
            "label": "success"
          }
        ]
      }
    }
  ],
  [
    "yewtu.be",
    {
      "flag": "🇫🇷",
      "region": "FR",
      "stats": {
        "version": "2.0",
        "software": {
          "name": "invidious",
          "version": "0.20.1-4a6e920",
          "branch": "master"
        },
        "openRegistrations": true,
        "usage": {
          "users": {
            "total": 1548,
            "activeHalfyear": 1548,
            "activeMonth": 1053
          }
        },
        "metadata": {
          "updatedAt": 1599289694,
          "lastChannelRefreshedAt": 1599289694
        }
      },
      "type": "https",
      "uri": "https://yewtu.be",
      "monitor": {
        "monitorId": 784257752,
        "statusClass": "success",
        "weeklyRatio": {
          "ratio": "99.010",
          "label": "success"
        },
        "name": "yewtu.be",
        "type": "http(s)",
        "dailyRatios": [
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "97.237",
            "label": "warning"
          },
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "95.832",
            "label": "warning"
          }
        ]
      }
    }
  ],
  [
    "invidious.tube",
    {
      "flag": "🇳🇱",
      "region": "NL",
      "stats": {
        "version": "2.0",
        "software": {
          "name": "invidious",
          "version": "0.20.1-4a6e920",
          "branch": "master"
        },
        "openRegistrations": true,
        "usage": {
          "users": {
            "total": 1081,
            "activeHalfyear": 192,
            "activeMonth": 192
          }
        },
        "metadata": {
          "updatedAt": 1599289698,
          "lastChannelRefreshedAt": 1599289694
        }
      },
      "type": "https",
      "uri": "https://invidious.tube/",
      "monitor": null
    }
  ],
  [
    "invidious.xyz",
    {
      "flag": "🇺🇸",
      "region": "US",
      "stats": {
        "version": "2.0",
        "software": {
          "name": "invidious",
          "version": "0.20.1-45fda59",
          "branch": "master"
        },
        "openRegistrations": true,
        "usage": {
          "users": {
            "total": 78,
            "activeHalfyear": 78,
            "activeMonth": 78
          }
        },
        "metadata": {
          "updatedAt": 1599289670,
          "lastChannelRefreshedAt": 1599287429
        }
      },
      "type": "https",
      "uri": "https://invidious.xyz",
      "monitor": null
    }
  ],
  [
    "invidious.site",
    {
      "flag": "🇳🇱",
      "region": "NL",
      "stats": {
        "error": "Statistics are not enabled."
      },
      "type": "https",
      "uri": "https://invidious.site/",
      "monitor": null
    }
  ],
  [
    "vid.mint.lgbt",
    {
      "flag": "🇺🇸",
      "region": "US",
      "stats": {
        "error": "Statistics are not enabled."
      },
      "type": "https",
      "uri": "https://vid.mint.lgbt/",
      "monitor": {
        "monitorId": 785715379,
        "statusClass": "success",
        "weeklyRatio": {
          "ratio": "99.963",
          "label": "success"
        },
        "name": "vid.mint.lgbt",
        "type": "http(s)",
        "dailyRatios": [
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "99.744",
            "label": "success"
          },
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "100.00",
            "label": "success"
          }
        ]
      }
    }
  ],
  [
    "invidiou.site",
    {
      "flag": "🇺🇸",
      "region": "US",
      "stats": {
        "error": "Statistics are not enabled."
      },
      "type": "https",
      "uri": "https://invidiou.site/",
      "monitor": {
        "monitorId": 785715375,
        "statusClass": "success",
        "weeklyRatio": {
          "ratio": "100.00",
          "label": "success"
        },
        "name": "invidiou.site",
        "type": "http(s)",
        "dailyRatios": [
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "100.00",
            "label": "success"
          }
        ]
      }
    }
  ],
  [
    "invidious.fdn.fr",
    {
      "flag": null,
      "region": null,
      "stats": {
        "error": "Statistics are not enabled."
      },
      "type": "https",
      "uri": "https://invidious.fdn.fr/",
      "monitor": {
        "monitorId": 785419052,
        "statusClass": "success",
        "weeklyRatio": {
          "ratio": "99.531",
          "label": "success"
        },
        "name": "invidious.fdn.fr",
        "type": "http(s)",
        "dailyRatios": [
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "99.343",
            "label": "success"
          },
          {
            "ratio": "98.686",
            "label": "success"
          },
          {
            "ratio": "98.997",
            "label": "success"
          },
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "100.00",
            "label": "success"
          }
        ]
      }
    }
  ],
  [
    "invidious.toot.koeln",
    {
      "flag": "🇩🇪",
      "region": "DE",
      "stats": {
        "error": "Statistics are not enabled."
      },
      "type": "https",
      "uri": "https://invidious.toot.koeln",
      "monitor": {
        "monitorId": 784257754,
        "statusClass": "success",
        "weeklyRatio": {
          "ratio": "96.214",
          "label": "warning"
        },
        "name": "invidious.toot.koeln",
        "type": "http(s)",
        "dailyRatios": [
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "74.451",
            "label": "warning"
          },
          {
            "ratio": "99.045",
            "label": "success"
          },
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "100.00",
            "label": "success"
          }
        ]
      }
    }
  ],
  [
    "invidious.ggc-project.de",
    {
      "flag": "🇩🇪",
      "region": "DE",
      "stats": {
        "error": "Statistics are not enabled."
      },
      "type": "https",
      "uri": "https://invidious.ggc-project.de",
      "monitor": {
        "monitorId": 784310967,
        "statusClass": "success",
        "weeklyRatio": {
          "ratio": "100.00",
          "label": "success"
        },
        "name": "invidious.ggc-project.de",
        "type": "http(s)",
        "dailyRatios": [
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "100.00",
            "label": "success"
          }
        ]
      }
    }
  ],
  [
    "owxfohz4kjyv25fvlqilyxast7inivgiktls3th44jhk3ej3i7ya.b32.i2p",
    {
      "flag": null,
      "region": null,
      "stats": null,
      "type": "i2p",
      "uri": "http://owxfohz4kjyv25fvlqilyxast7inivgiktls3th44jhk3ej3i7ya.b32.i2p",
      "monitor": null
    }
  ],
  [
    "4l2dgddgsrkf2ous66i6seeyi6etzfgrue332grh2n7madpwopotugyd.onion",
    {
      "flag": null,
      "region": null,
      "stats": null,
      "type": "onion",
      "uri": "http://4l2dgddgsrkf2ous66i6seeyi6etzfgrue332grh2n7madpwopotugyd.onion",
      "monitor": null
    }
  ],
  [
    "fz253lmuao3strwbfbmx46yu7acac2jz27iwtorgmbqlkurlclmancad.onion",
    {
      "flag": null,
      "region": null,
      "stats": null,
      "type": "onion",
      "uri": "http://fz253lmuao3strwbfbmx46yu7acac2jz27iwtorgmbqlkurlclmancad.onion/",
      "monitor": null
    }
  ],
  [
    "qklhadlycap4cnod.onion",
    {
      "flag": null,
      "region": null,
      "stats": null,
      "type": "onion",
      "uri": "http://qklhadlycap4cnod.onion/",
      "monitor": null
    }
  ],
  [
    "c7hqkpkpemu6e7emz5b4vyz7idjgdvgaaa3dyimmeojqbgpea3xqjoid.onion",
    {
      "flag": null,
      "region": null,
      "stats": null,
      "type": "onion",
      "uri": "http://c7hqkpkpemu6e7emz5b4vyz7idjgdvgaaa3dyimmeojqbgpea3xqjoid.onion",
      "monitor": null
    }
  ],
  [
    "mfqczy4mysscub2s.onion",
    {
      "flag": null,
      "region": null,
      "stats": null,
      "type": "onion",
      "uri": "http://mfqczy4mysscub2s.onion/",
      "monitor": null
    }
  ],
  [
    "sb46cryrbrjsfe4kfeculaj3tpke36ynstjm5wym2yh5ng2nflgyugyd.onion",
    {
      "flag": null,
      "region": null,
      "stats": null,
      "type": "onion",
      "uri": "http://sb46cryrbrjsfe4kfeculaj3tpke36ynstjm5wym2yh5ng2nflgyugyd.onion",
      "monitor": null
    }
  ]
]

[Ambrevar]

I think by "modeline" you mean minibuffer (the interactive part).

There are several parts to this:

1. You need to intercept the request. See re

   Example handler that hi-jacks the reddit.com requests and redirects them to old.reddit.com:

(defparameter old-reddit-handler
  (url-dispatching-handler
   'old-reddit-dispatcher
   (match-host "www.reddit.com")
   (lambda (url)
     (quri:copy-uri url :host "old.reddit.com"))))

(define-configuration buffer
  ((request-resource-hook
    (reduce #'hooks:add-hook
            (list old-reddit-handler)
            :initial-value %slot-default))))

2. Once your Youtube request is intercepted, you need to replace the host by the one from Invidious.

   • You can fetch the JSON with dexador, e.g. dex:get.

   • You can parse the json with cl-json:decode-json-from-string.

Let me know if you need more pointers.

[Ambrevar]

By the way, I think it's a great idea! In your experience, is it more stable than Youtube.com on Nyxt?

Maybe we could include this handler upstream as a workaround (improvement?) for the instability of Youtube.

[80aff123-9163-4f3e-a93d-4a2f35af9be1]

@Ambrevar Thank you so much for your generous response. Yes, I meant the minibuffer. Thank you for pointing that out.

By the way, I think it's a great idea! In your experience, is it more stable than Youtube.com on Nyxt?

I'm glad you like the idea! It depends on the instance but I have found that it is the only bearable way I can watch and browse youtube content.

Overall I've found it more stable and a much faster and responsive experience than browsing on youtube.com

For me, the most important thing is that invidious is free software and protects the user. Here are the features of invidious taken from the project's README:

Invidious is an alternative front-end to YouTube

• Copylefted libre software (AGPLv3+ licensed)
• Audio-only mode (and no need to keep window open on mobile)
• Lightweight (the homepage is ~4 KB compressed)
• Tools for managing subscriptions:
  • Only show unseen videos
  • Only show latest (or latest unseen) video from each channel
  • Delivers notifications from all subscribed channels
  • Automatically redirect homepage to feed
  • Import subscriptions from YouTube
• Dark mode
• Embed support
• Set default player options (speed, quality, autoplay, loop)
• Support for Reddit comments in place of YouTube comments
• Import/Export subscriptions, watch history, preferences
• Developer API
• Does not use any of the official YouTube APIs
• Does not require JavaScript to play videos
• No need to create a Google account to save subscriptions
• No ads
• No CoC
• No CLA

Maybe we could include this handler upstream as a workaround (improvement?) for the instability of Youtube.

I will try to study and implement what you suggested but I'm not sure how long it will take me to get familiar with the common lisp ecosystem given the other projects I'm currently working on. I will try nonetheless, as I would really like to have this feature. If I'm able to come up with a solution I promise to open a pull request and share it here for code review.

I had a little keybinded function/hack in qutebrowser that would redirect me to my preferred invidious instance. Alas, I am not using qutebrowser anymore at the moment.

Feel free to try your own solution and contribute it if you get to it first. It would be amazing to have this in nyxt in any capacity.

[Ambrevar]

Good luck!

[kssytsrk]

Hi. I'm looking into this now as I would love to have a feature like that, too, but I don't seem to be able to call read-from-minibuffer from url-dispatching-handler. Here's my code (stripped of the suggestion function, function that gets the json, etc. - I don't think it's relevant to this problem):

(defparameter youtube-2-invidious-handler
  (url-dispatching-handler
   'youtube-2-invidious-dispatcher
   (match-host "www.youtube.com")
   (lambda (url)
     (with-result (instance (read-from-minibuffer
                             (make-minibuffer
                              :input-prompt "Choose an instance: ")))
                  (quri:copy-uri url :host instance)))))

(define-configuration buffer
  ((request-resource-hook
    (reduce #'hooks:add-hook
            (list youtube-2-invidious-handler)
            :initial-value %slot-default))))

This just freezes up Nyxt (without opening the minibuffer) when going on youtube.com. Error message in console is as follows:

<INFO> [12:22:41] Applied YOUTUBE-2-INVIDIOUS-DISPATCHER 
URL-dispatcher on https://www.youtube.com/ and got -1

debugger invoked on a SIMPLE-TYPE-ERROR in thread
#<THREAD cl-cffi-gtk main thread RUNNING {10083DF513}>:
The value of QURI::URI2 is -1, which is not of type QURI.URI:URI.

Should I call the minibuffer from some other function? Or in some other way? I'd really appreciate it if anyone pointed me in the right direction.

[jmercouris]

You've made quite a bit of progress! Nice!

Here's what I believe the problem to be (without deeply investigating). Hopefully this will give you some ideas!

READ-FROM-MINIBUFFER is non-blocking/asynchronous. It will immediately return. When the user has finally inputted some data, the minibuffer will then call a callback.

In your case, since url-dispaching-handler is expecting you to return an alternate URL, this will not work. I cannot think of a way to block until the query is complete- UNLESS you use your own semaphores or blocking.

As an extra note: we have a pull request with synchronous minibuffers pending (though it still requires quite some work!).

I hope that helps!

[kssytsrk]

Thanks for the reply!

Guess this issue will probably have to wait for synchronous minibuffer support, then. The furthest I got is this code below that spawns the minibuffer after loading an empty url, and after setting the instance in the minibuffer it redirects fine on all the next youtube.com visits:

(defvar *preferred-invidious-instance* "")

(defparameter youtube-2-invidious-handler
  (url-dispatching-handler
   'youtube-2-invidious-dispatcher
   (match-host "www.youtube.com")
   (lambda (url)
     (let ((queue (lparallel.queue:make-queue)))
       (lparallel.queue:push-queue
        (if (equal *preferred-invidious-instance* "")
            (with-result (instance (read-from-minibuffer
                                       (make-minibuffer
                                        :input-prompt "Choose an instance: ")))
                 (setf *preferred-invidious-instance* instance)))
        queue)
       (lparallel.queue:push-queue
        (quri:copy-uri url
                       :host
                       *preferred-invidious-instance*)
        queue)
       (lparallel.queue:pop-queue queue)
       (lparallel.queue:pop-queue queue)))))

(define-configuration buffer
  ((request-resource-hook
    (reduce #'hooks:add-hook
            (list youtube-2-invidious-handler)
            :initial-value %slot-default))))

[jmercouris]

I think you are right about that. It will be significantly simpler to implement with a synchronous minibuffer. For now, this is a good stopgap. Thank you for implementing a solution :-)

[Ambrevar]

The problem is not really with the minibuffer but with the request-resource-hook which does not handle interactivity yet.

The problem is that if we wait for user interactions, we are essentially blocking the network request, which can be problematic for various reasons.

We had discussed it with @aartaka (can you remember where?).

Do we really need interactivity here however? It seems that we can auto-select the instance based on some heuristic. This would bypass the interactivity issue.

[kssytsrk]

Should there just be a command for explicitly setting a preferred instance, then? And if preferred-invidious-instance is not set, the "healthiest" (based on the json) instance would be selected automatically.

[Ambrevar]

Yup, something along those lines would be fine.

[kssytsrk]

I'm trying to implement this now, but I don't seem to be able to make :suggestions-display tag work in the suggestion-filter function for invidious instances. It always shows me only the real-values as suggestions. To make it more easier to understand, here's a pic

What I'm expecting is for it to show the instance's url together with its health in parentheses (as I tried to implement in #'pretty-print-invidious-instance below). My code for that is:

(defvar *preferred-invidious-instance* nil)

(defun get-invidious-instances (&key with-health)
  (mapcar (if with-health
              #'(lambda (json)
                  (list (first json)
                        ;; this here is kind of ugly;
                        ;; it gets the daily ratio (health) of an instance, and
                        ;; i have to dive deep into sublists to get it
                        ;; should i indent it in some other way, perhaps?
                        (rest (assoc ':ratio
                                     (rest
                                      (assoc ':weekly-ratio
                                             (rest
                                              (assoc ':monitor
                                                     (first
                                                      (rest json))))))))))
              #'(lambda (json)
                  (first json)))
          (cl-json:with-decoder-simple-list-semantics
              (cl-json:decode-json-from-string
               (dex:get "https://instances.invidio.us/instances.json?sort_by=health")))))

(defun pretty-print-invidious-instance (instance)
  (format nil
          "~a (health: ~:[unknown~;~a~])"
          (first instance)
          (second instance)
          (second instance)))

(defun invidious-instance-suggestion-filter ()
  (let* ((instances (get-invidious-instances))
         (pretty-instances
          (mapcar #'pretty-print-invidious-instance
                  (get-invidious-instances :with-health t))))
    (lambda (minibuffer)
      (fuzzy-match (input-buffer minibuffer)
                   instances
                   :suggestions-display pretty-instances))))

(define-command set-preferred-invidious-instance ()
  (with-result (instance (read-from-minibuffer
                          (make-minibuffer
                           :input-prompt "Choose an instance"
                           :suggestion-function (invidious-instance-suggestion-filter))))
    (setf *preferred-invidious-instance* instance)))

When I run (mapcar #'pretty-print-invidious-instance (get-invidious-instances :with-health t)) in SLIME it seems to work with no problems. Would appreciate it if someone pointed me in the right direction on how to make this filter function work properly.

[jmercouris]

I have a feeling it has something to do with the let binding. Can you try printing the pretty-instances just before you return the lambda in invidious-instance-suggestion-filter?

[kssytsrk]

Done, here's the result:

[jmercouris]

Hm, that's interesting. Can you recreate a minimal example that shows the suggestions-display working?

[jmercouris]

I can also try to do so tomorrow and see if I can figure it out.

[kssytsrk]

I didn't manage to get it working (tried changing the type of real values to just symbols and the type of display values to just symbols, changing the length of a list, nothing worked, things displayed are still the "real values", not "display values"), but here's what I discovered: it does fuzzy-match the input to :suggestions-display values and not to the real values. i.e. with this code:

(defun test-suggestion-filter ()
  (let* ((instances '("a" "aaa" "aaaa"))
         (pretty-instances '("b" "c" "d")))
    (log:info pretty-instances)
    (lambda (minibuffer)
      (fuzzy-match (input-buffer minibuffer)
                   instances
                   :suggestions-display pretty-instances))))

(define-command set-preferred-invidious-instance ()
  (with-result (instance (read-from-minibuffer
                          (make-minibuffer
                           :input-prompt "Choose an instance"
                           :suggestion-function (test-suggestion-filter))))
    (setf *preferred-invidious-instance* instance)))

when inputted "c" the "aaa" option will come out on top. but it still will display itself as the "aaa" option.

There aren't many examples of using FUZZY-MATCH with :suggestions-display keyword in the codebase, I found only one in command-suggestion-filter in command-commands.lisp file, and there it seems to work fine (and I tried to do everything similarly from the start, so, I don't really get where might the issue be). Any ideas?

[jmercouris]

I have another approach that you may wish to take. The simplest example of it in the codebase I can think of is in autofill.lisp.

We make a struct (autofill), we make an object-string, and we make an object-display. The object-string is used for fuzzy matching, and the object-display is used for rendering.

You can do the same thing in your case. Make a invidious-instance struct, give it some fields that are you are interested and populate it accordingly.

Everything will work the same for you except that you will have to pass a bunch of invidious-instance structs in to your suggestion function, and your result from your minibuffer will have to process a invidious-instance struct instead.

Does that help?

[Ambrevar]

We make a struct (autofill), we make an object-string, and we make an object-display. The object-string is used for fuzzy matching, and the object-display is used for rendering.

Correction: object-display is used both for fuzzy-matching and rendering. object-string is used for textual manipulation such as tab-insert of clipboard copy.

[jmercouris]

In that case, can we make fuzzy-match just accept a #'key function as in other CL functions?

[Ambrevar]

fuzzy-match already takes a suggestions-display key argument. I don't understand what you are trying to do, can you provide an example?

[jmercouris]

I'm saying, instead of passing in a suggestions-display, one passes in a #'key function to get the string to fuzzy match against. Then, we can continue to use object-display for the rendering.

[Ambrevar]

But what's the purpose here?

[jmercouris]

So that we don't have to pass a suggestions-display

[Ambrevar]

You don't need to pass a suggestions-display, it's there only for performance reasons (only one minibuffer needs it if I'm not mistaken).

Can you give a concrete example of what you are trying to achieve?

[jmercouris]

Please read earlier in this thread for what the user kssytsrk is trying to achieve.

[kssytsrk]

Can you give a concrete example of what you are trying to achieve?

In the minibuffer, I'm trying to display as suggestions something different from the value that will be returned from the minibuffer. I want the url and health of an instance to be displayed as a suggestion (e.g. "invidiou.site (health: 100)"), but when user chooses it, just "invidiou.site" should be returned. Does that make it any more clear?

Make a invidious-instance struct, give it some fields that are you are interested and populate it accordingly.

Thanks! I will try that approach.

[Ambrevar]

kssytsrk notifications@github.com writes:

Can you give a concrete example of what you are trying to achieve?

In the minibuffer, I'm trying to display as suggestions something different from the value that will be returned from the minibuffer. I want the url and health of an instance to be displayed as a suggestion (e.g. "invidiou.site (health: 100)"), but when user chooses it, just "invidiou.site" should be returned. Does that make it any more clear?

Yes, thanks. Make an `invidious-entry' class/struct with a health slot, then define the object-display method to return

(format nil "~s (health: ~a)" (name obj) (health obj))

Finally, feed a list of `invidious-entry's to the minibuffer.

[jmercouris]

That's what I said :-D

[aartaka]

The problem is not really with the minibuffer but with the request-resource-hook which does not handle interactivity yet. The problem is that if we wait for user interactions, we are essentially blocking the network request, which can be problematic for various reasons. We had discussed it with @aartaka (can you remember where?).

It was in #868, where we needed to stop request if the user doesn't want the presented modes to be enabled for a given domain.

[Ambrevar]

That's what I said :-D

Initially, yes, but I think there was a confusion with my correction that object-string is not used as a return value.

Indeed, the original object is returned.

[kssytsrk]

Yay, it works! Thanks everyone. Now, do I open a pull request with all this implemented in a separate (new) file or should I add it to some file that already exists?

[jmercouris]

Nice work!

I would say a new file, or maybe even a new repository, this could be our first "extension", I don't know if this belongs in Nyxt core.

[Ambrevar]

Good point. I suggest the following policy: anything that's specific to a website should be packaged as a separate package (an extension). Thoughts?

[jmercouris]

I agree with that idea.

[80aff123-9163-4f3e-a93d-4a2f35af9be1]

Could the extension be added to a separate atlas-engineer git repository?

User contributed extensions could be kept at the authors personal project page but linked to this centralized atlas-engineer repo.

I'm thinking of the way that mycroft does this with user contributed mycroft-skills that are linked back to the authors personal github repo but collected on the official project page.

See here:

mycroft skills pointing to user's github repos from main repo:

https://github.com/MycroftAI/mycroft-skills

See here for a example mycroft skill contributed from a user and included in their repo:

https://github.com/Arc676/Crystal-Ball-Mycroft-Skill

User contributed skill in the mycroft market (this probably does not apply here):

https://market.mycroft.ai/skills/5deaa7d6-b439-4bd6-82ff-22d45d7835cd

Congrats @kssytsrk! Those screenshots look great! I can't wait to try this feature out.

Any thoughts?

[Ambrevar]

Having a repo with submodules could be one way to go indeed. This would not give us much metadata though.

A specialized Quickdist is probably a better approach: this would also be a repository @ atlas-engineer which would collect all the user-maintained repositories, with metadata.

[kssytsrk]

A Quickdist sounds nice. Either way, I'd love to contribute anywhere.

[Ambrevar]

For now, I suggest you create your own repository (and spread the word!), we are working on a Quickdist these days so it should not take too long before we can officially reference your package.

[kssytsrk]

Okay. But what would be the optimal/proper way to "load" the package? In StumpWM (just an example), there's the LOAD-MODULE function, does Nyxt have anything like that? I also looked into just ASDF:LOAD-SYSTEM, but it seems to need a symlink to the system's .asd file somewhere in ASDF:*CENTRAL-REGISTRY* (or at least that's what I've seen in a couple of tutorials) and that doesn't seem very convenient.

[Ambrevar]

For now you can just drop your extension in ~/common-lisp, it will be found by asdf:load-system.

We should add a Nyxt-specific path though. Maybe ~/.local/share/nyxt/packages?

[jmercouris]

I would call them systems, not packages- but otherwise agree!

[Ambrevar]

Agreed that "packages" is a poor term because it conflicts with Lisp' definition.

However, "systems" is not a familiar term. What about "extensions" instead?

[kssytsrk]

I'd vote "extensions", but won't that make some confusion when/if WebExtensions are implemented?

Also, I don't seem to be able to use the commands (defined with DEFINE-COMMAND) from the loaded system. I can't access the variables with DESCRIBE-VARIABLE command, too. When I wrap a loaded command in something like

(define-command settest ()
  (set-preferred-invidious-instance))

it werks perfectly. Any ideas on how to fix/work around that? (also, does this conversation still belong in this issue? is there any issue about "extensions" we could take this to?)

[kssytsrk]

Here's the newly created repo, by the way.

[jmercouris]

We will always have some unfamiliar terms, buffer for example. We cannot eliminate all friction. I think in this case system is the best term because it is consistent with Lisp, and cannot be confused with web Extensions.

[jmercouris]

I'm not aware of another issue where we talk about this :-D

[jmercouris]

As per why that command is not visible, it is because our minibuffer by default only shows commands available for the current modes.

What you may do to expose commands without necessitating a mode is the following:

(in-package :nyxt)

and then put whatever commands you want to be invokable regardless of mode.

[jmercouris]

Or maybe we could call them extension-systems?

[Ambrevar]

John Mercouris notifications@github.com writes:

Or maybe we could call them extension-systems?

It's clear, I like it, but maybe a bit longish... I expect our community to quickly switch to an abbreviation :)