80aff123-9163-4f3e-a93d-4a2f35af9be1 commented 4 years ago

How could I add a function that with a single keyboard shortcut opens the modeline and redirects me from youtube.com to choose amongst one of several online invidious instances?

I would like the modeline to sort the instances by current health of the site. See here or the dailyRatios and weeklyRatio json fields below for an example of what I mean about site health.

Any help or suggestions are greatly appreciated! A version of this program for online peertube instances would be cool too.

JSON code showing Invidious Instances as of Today:

[
  [
    "invidious.snopyta.org",
    {
      "flag": "🇫🇮",
      "region": "FI",
      "stats": {
        "version": "2.0",
        "software": {
          "name": "invidious",
          "version": "0.20.1-13f58d6",
          "branch": "master"
        },
        "openRegistrations": true,
        "usage": {
          "users": {
            "total": 3056,
            "activeHalfyear": 2546,
            "activeMonth": 1729
          }
        },
        "metadata": {
          "updatedAt": 1599289689,
          "lastChannelRefreshedAt": 1599289686
        }
      },
      "type": "https",
      "uri": "https://invidious.snopyta.org/",
      "monitor": {
        "monitorId": 782758961,
        "statusClass": "success",
        "weeklyRatio": {
          "ratio": "99.367",
          "label": "success"
        },
        "name": "invidious.snopyta.org",
        "type": "http(s)",
        "dailyRatios": [
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "95.569",
            "label": "warning"
          },
          {
            "ratio": "100.00",
            "label": "success"
          }
        ]
      }
    }
  ],
  [
    "yewtu.be",
    {
      "flag": "🇫🇷",
      "region": "FR",
      "stats": {
        "version": "2.0",
        "software": {
          "name": "invidious",
          "version": "0.20.1-4a6e920",
          "branch": "master"
        },
        "openRegistrations": true,
        "usage": {
          "users": {
            "total": 1548,
            "activeHalfyear": 1548,
            "activeMonth": 1053
          }
        },
        "metadata": {
          "updatedAt": 1599289694,
          "lastChannelRefreshedAt": 1599289694
        }
      },
      "type": "https",
      "uri": "https://yewtu.be",
      "monitor": {
        "monitorId": 784257752,
        "statusClass": "success",
        "weeklyRatio": {
          "ratio": "99.010",
          "label": "success"
        },
        "name": "yewtu.be",
        "type": "http(s)",
        "dailyRatios": [
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "97.237",
            "label": "warning"
          },
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "95.832",
            "label": "warning"
          }
        ]
      }
    }
  ],
  [
    "invidious.tube",
    {
      "flag": "🇳🇱",
      "region": "NL",
      "stats": {
        "version": "2.0",
        "software": {
          "name": "invidious",
          "version": "0.20.1-4a6e920",
          "branch": "master"
        },
        "openRegistrations": true,
        "usage": {
          "users": {
            "total": 1081,
            "activeHalfyear": 192,
            "activeMonth": 192
          }
        },
        "metadata": {
          "updatedAt": 1599289698,
          "lastChannelRefreshedAt": 1599289694
        }
      },
      "type": "https",
      "uri": "https://invidious.tube/",
      "monitor": null
    }
  ],
  [
    "invidious.xyz",
    {
      "flag": "🇺🇸",
      "region": "US",
      "stats": {
        "version": "2.0",
        "software": {
          "name": "invidious",
          "version": "0.20.1-45fda59",
          "branch": "master"
        },
        "openRegistrations": true,
        "usage": {
          "users": {
            "total": 78,
            "activeHalfyear": 78,
            "activeMonth": 78
          }
        },
        "metadata": {
          "updatedAt": 1599289670,
          "lastChannelRefreshedAt": 1599287429
        }
      },
      "type": "https",
      "uri": "https://invidious.xyz",
      "monitor": null
    }
  ],
  [
    "invidious.site",
    {
      "flag": "🇳🇱",
      "region": "NL",
      "stats": {
        "error": "Statistics are not enabled."
      },
      "type": "https",
      "uri": "https://invidious.site/",
      "monitor": null
    }
  ],
  [
    "vid.mint.lgbt",
    {
      "flag": "🇺🇸",
      "region": "US",
      "stats": {
        "error": "Statistics are not enabled."
      },
      "type": "https",
      "uri": "https://vid.mint.lgbt/",
      "monitor": {
        "monitorId": 785715379,
        "statusClass": "success",
        "weeklyRatio": {
          "ratio": "99.963",
          "label": "success"
        },
        "name": "vid.mint.lgbt",
        "type": "http(s)",
        "dailyRatios": [
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "99.744",
            "label": "success"
          },
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "100.00",
            "label": "success"
          }
        ]
      }
    }
  ],
  [
    "invidiou.site",
    {
      "flag": "🇺🇸",
      "region": "US",
      "stats": {
        "error": "Statistics are not enabled."
      },
      "type": "https",
      "uri": "https://invidiou.site/",
      "monitor": {
        "monitorId": 785715375,
        "statusClass": "success",
        "weeklyRatio": {
          "ratio": "100.00",
          "label": "success"
        },
        "name": "invidiou.site",
        "type": "http(s)",
        "dailyRatios": [
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "100.00",
            "label": "success"
          }
        ]
      }
    }
  ],
  [
    "invidious.fdn.fr",
    {
      "flag": null,
      "region": null,
      "stats": {
        "error": "Statistics are not enabled."
      },
      "type": "https",
      "uri": "https://invidious.fdn.fr/",
      "monitor": {
        "monitorId": 785419052,
        "statusClass": "success",
        "weeklyRatio": {
          "ratio": "99.531",
          "label": "success"
        },
        "name": "invidious.fdn.fr",
        "type": "http(s)",
        "dailyRatios": [
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "99.343",
            "label": "success"
          },
          {
            "ratio": "98.686",
            "label": "success"
          },
          {
            "ratio": "98.997",
            "label": "success"
          },
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "100.00",
            "label": "success"
          }
        ]
      }
    }
  ],
  [
    "invidious.toot.koeln",
    {
      "flag": "🇩🇪",
      "region": "DE",
      "stats": {
        "error": "Statistics are not enabled."
      },
      "type": "https",
      "uri": "https://invidious.toot.koeln",
      "monitor": {
        "monitorId": 784257754,
        "statusClass": "success",
        "weeklyRatio": {
          "ratio": "96.214",
          "label": "warning"
        },
        "name": "invidious.toot.koeln",
        "type": "http(s)",
        "dailyRatios": [
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "74.451",
            "label": "warning"
          },
          {
            "ratio": "99.045",
            "label": "success"
          },
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "100.00",
            "label": "success"
          }
        ]
      }
    }
  ],
  [
    "invidious.ggc-project.de",
    {
      "flag": "🇩🇪",
      "region": "DE",
      "stats": {
        "error": "Statistics are not enabled."
      },
      "type": "https",
      "uri": "https://invidious.ggc-project.de",
      "monitor": {
        "monitorId": 784310967,
        "statusClass": "success",
        "weeklyRatio": {
          "ratio": "100.00",
          "label": "success"
        },
        "name": "invidious.ggc-project.de",
        "type": "http(s)",
        "dailyRatios": [
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "100.00",
            "label": "success"
          },
          {
            "ratio": "100.00",
            "label": "success"
          }
        ]
      }
    }
  ],
  [
    "owxfohz4kjyv25fvlqilyxast7inivgiktls3th44jhk3ej3i7ya.b32.i2p",
    {
      "flag": null,
      "region": null,
      "stats": null,
      "type": "i2p",
      "uri": "http://owxfohz4kjyv25fvlqilyxast7inivgiktls3th44jhk3ej3i7ya.b32.i2p",
      "monitor": null
    }
  ],
  [
    "4l2dgddgsrkf2ous66i6seeyi6etzfgrue332grh2n7madpwopotugyd.onion",
    {
      "flag": null,
      "region": null,
      "stats": null,
      "type": "onion",
      "uri": "http://4l2dgddgsrkf2ous66i6seeyi6etzfgrue332grh2n7madpwopotugyd.onion",
      "monitor": null
    }
  ],
  [
    "fz253lmuao3strwbfbmx46yu7acac2jz27iwtorgmbqlkurlclmancad.onion",
    {
      "flag": null,
      "region": null,
      "stats": null,
      "type": "onion",
      "uri": "http://fz253lmuao3strwbfbmx46yu7acac2jz27iwtorgmbqlkurlclmancad.onion/",
      "monitor": null
    }
  ],
  [
    "qklhadlycap4cnod.onion",
    {
      "flag": null,
      "region": null,
      "stats": null,
      "type": "onion",
      "uri": "http://qklhadlycap4cnod.onion/",
      "monitor": null
    }
  ],
  [
    "c7hqkpkpemu6e7emz5b4vyz7idjgdvgaaa3dyimmeojqbgpea3xqjoid.onion",
    {
      "flag": null,
      "region": null,
      "stats": null,
      "type": "onion",
      "uri": "http://c7hqkpkpemu6e7emz5b4vyz7idjgdvgaaa3dyimmeojqbgpea3xqjoid.onion",
      "monitor": null
    }
  ],
  [
    "mfqczy4mysscub2s.onion",
    {
      "flag": null,
      "region": null,
      "stats": null,
      "type": "onion",
      "uri": "http://mfqczy4mysscub2s.onion/",
      "monitor": null
    }
  ],
  [
    "sb46cryrbrjsfe4kfeculaj3tpke36ynstjm5wym2yh5ng2nflgyugyd.onion",
    {
      "flag": null,
      "region": null,
      "stats": null,
      "type": "onion",
      "uri": "http://sb46cryrbrjsfe4kfeculaj3tpke36ynstjm5wym2yh5ng2nflgyugyd.onion",
      "monitor": null
    }
  ]
]

kssytsrk commented 4 years ago

Thanks, (in-package :nyxt) helped and everything seems fine now. @ccao001 please try out the new extension-system, I hope it works the way you wanted! If something's the bother, open an issue in the repo.

I suppose this issue should be closed now?

jmercouris commented 4 years ago

Almost, I would suggest we find a way to reference your repository in the interim. I liked the idea of utilizing a repository with submodules for this purpose. @Ambrevar shall it not be an OK stopgap solution until our quickdist is ready?

Ambrevar commented 4 years ago

That or the wiki we talked about recently.

Ambrevar commented 4 years ago

@kssytsrk I've sent a fix there: https://github.com/kssytsrk/invidious-handler/issues/1

I also suggest we namespace the Nyxt extensions. For instance, instead of the invidious-handler package we would have nyxt/ext/invidious-handler.

Thoughts?

jmercouris commented 4 years ago

Why would we like to namespace things?

Ambrevar commented 4 years ago

This would help the user or the hacker to tell what's a Nyxt extension from what's not just by looking at the symbol package.

In the user configuration, it makes it easy to tell what depends on something external or not.

Take Emacs: When a package is "require"d, it imports a bunch of symbols. After this point, it's hard to tell what's vanilla Emacs from what is not.

Even with Common Lisp packages, we can't obviously tell what's vanilla Nyxt unless you know the default Nyxt packages by heart (chanl, bordeaux-threads, etc.).

kssytsrk commented 4 years ago

Sorry for the slow reply, was kinda busy. I don't have a lot of experience with asdf and packaging, so I don't seem to understand how to namespace things and haven't really found any docs on that (aside from "The Complete Idiot's Guide to Common Lisp Packages", but it doesn't mention namespacing). This is what I am doing:

(invidious-handler.asd)

(asdf:defsystem nyxt/ext/invidious-handler
  ...)

(package.lisp)

(in-package :nyxt)

(uiop:define-package :ext/invidious-handler
  ...)

(invidious-handler.lisp)

(in-package :nyxt/ext/invidious-handler)
...

In my init file, I try to load it with (asdf:load-system :nyxt/ext/invidious-handler), but I get "Could not load the init file: Component :EXT/INVIDIOUS-HANDLER not found" when starting Nyxt. I sense that something is really wrong with my definitions of system and package, but can't really pinpoint what. Could you provide any pointers on this?

Ambrevar commented 4 years ago

kssytsrk notifications@github.com writes:

(package.lisp)

(in-package :nyxt)

(uiop:define-package :ext/invidious-handler
  ...)

The keyword passed to define-package (or defpackage) must be "absolute", so if you want to define the :nyxt/ext/invidious-handler package, you need to specify it completely:

(uiop:define-package :nyxt/ext/invidious-handler ...)

kssytsrk commented 4 years ago

Thank you. But for some reason I still get "Could not load the init file: Component :EXT/INVIDIOUS-HANDLER not found" even though I've checked and I don't use :EXT/INVIDIOUS-HANDLER anywhere.

Ambrevar commented 4 years ago

In the .asd maybe?

kssytsrk commented 4 years ago

Nope, my code for .asd is as follows:

;;;; invidious-handler.asd

(asdf:defsystem nyxt/ext/invidious-handler
  :serial t
  :version "1.0.0"
  :description "A redirector from Youtube to Invidious."
  :author "kssytsrk"
  :license "GNU General Public License v3.0"
  :depends-on (:nyxt
               :dexador
               :cl-json)
  :components ((:file "package")
               (:file "invidious-handler")))

I'm not sure, does Lisp recompile systems when starting Nyxt? If it didn't everything would make sense

Ambrevar commented 4 years ago

It recompiles it if it changed. How do you build Nyxt?

kssytsrk commented 4 years ago

Well, I'm running it from a compiled binary, but the extension is just loaded from the init file so I don't suppose I have to recompile the whole browser to make changes to an extension?

Ambrevar commented 4 years ago

Try not using nyxt/ext but nyxt-ext instead.

kssytsrk commented 4 years ago

Still doesn't work. For some reason any package that has slashes in system/package name won't load, I even thought it was because of asdf expecting the .asd file here "~/common-lisp/nyxt-ext/invidious-handler/nyxt-ext/invidious.handler.asd" but that didn't work either.

Ambrevar commented 4 years ago

The .asd can be left at the root.

Can you push your attempt on a non-master branch so that I can investigate?

kssytsrk commented 4 years ago

Okay, I just pushed kssytsrk/invidious-handler@f12aefa.

Ambrevar commented 4 years ago

Sorry for the late answer.

We've just created the load-after-system helper which should help with loading extensions. https://github.com/atlas-engineer/nyxt/commit/0c32d100e545200d666207e4db2a37e44c7394ba

I'll looked into your issue as soon as possible.

kssytsrk commented 4 years ago

Thanks, just updated the readme with load-after-system.

jmercouris commented 4 years ago

@Ambrevar

I make a file called extension.org where we can list all community extensions for now, OK?

Ambrevar commented 4 years ago

Yes!

jmercouris commented 4 years ago

Voila!

https://github.com/atlas-engineer/nyxt/blob/master/documents/EXTENSIONS.org

Ambrevar commented 4 years ago

Thanks! Let's keep this to a separate issue, we've polluted this one for way too long already ;)

https://github.com/atlas-engineer/nyxt/issues/1011

Ambrevar commented 4 years ago

Actually forget #1011, see #966 instead.

Ambrevar commented 4 years ago

@kssytsrk I've finally taken the time to look at the issue! (Sorry for the delay...)

You forgot to rename a package prefix:

- (setf invidious-handler:*preferred-invidious-instance* (nyxt-ext/invidious-handler::object-string instance))

+ (setf nyxt-ext/invidious-handler:*preferred-invidious-instance* (nyxt-ext/invidious-handler::object-string instance))

The .asd file must have the name of the main system, here "nyxt-ext". So it must be named "nyxt-ext.asd".
You can't define a subsystem (when the name contains a "/") without defining the main system. So if you define the dummy system:

(asdf:defsystem :nyxt-ext
  :description "Forward system declaration for Nyxt extensions.")

it will work.

The problem is that this nyxt-ext system will clash with that of other extensions. So it should be defined in Nyxt, not in invidious-handler.

Shall we include the nyxt-ext system in our ASD then?

There is an alternative: instead of prefixing the system, we could leave it as it is (here invidious-handler) and only prefix the package name. It's a bit simpler.

The main drawback is that when the user loads a system they won't be able to immediately tell whether a Nyxt extension or not.

Compare:

(load-after-system :nyxt-ext/invidious-handler)

with

(load-after-system :invidious-handler)

The package symbols remain properly prefixed with nyxt-ext/invidious-handler however.

jmercouris commented 4 years ago

Let's include nyxt-ext in our ASD, yes :-)

People typically also expect the package and system naming to be consistent, so we will also benefit in that way.

Ambrevar commented 4 years ago

OK!

Ambrevar commented 4 years ago

Done!

kssytsrk commented 4 years ago

Okay, thanks for checking the code. Just pushed kssytsrk/invidious-handler@66c0115, but it still doesn't really work (errors out with Could not load the init file: Component "nyxt-ext" not found, required by #<SYSTEM "nyxt-ext/invidious-handler"> even though the version of Nyxt I use was pulled yesterday and I can see the system definition in the source). When I define nyxt-ext in invidious-handler's nyxt-ext.asd, it works fine though.

Ambrevar commented 4 years ago

Indeed, this can't work. The main system needs to be defined in the same .asd as the extension. Oh well...

So instead of dealing with subsystems, I suggest we follow the convention of prefixing system names with, say, "nx-". This is what Stumpwm does:

https://github.com/stumpwm/stumpwm-contrib https://github.com/njkli/stumpwm-weather

A "/" in the package name still works though, but do we want it if we don't use it in the system name? Out of consistency, maybe not.

So what about "nx-invidious-handler" both for the package name and the system name?

We also need to think about extensions that may use multiple packages, e.g. multiple modes. These extensions would use a slash in their names, so that would look like

nx-foo/first-mode nx-foo/second-mode

Thoughts? @jmercouris

jmercouris commented 4 years ago

I think that is a good set-up. Let's do it!

danrobi11 commented 3 years ago

This is cool. I've seen something similar few days ago. Makes me wonder if instead of an Invidio instance only handler a FreeStance handler? Twitter(Nitter), Instagram(Bibliogram), Reddit(Teddit) and ofcourse Invidio instead of YouTube. I've no coding skills, im just throwing that around.

Ambrevar commented 3 years ago

This is great! Thanks for suggesting this, didn't know about it! Indeed, looks like Nyxt's invidious extension could be generalized.

Teddit is really cool in particular! :)

kssytsrk commented 3 years ago

Oh, I thought about doing something similar a few days ago. After looking into it for a bit, though, I'm not sure what's the best way to implement it. Nitter, Bibliogram and Teddit don't have an API that returns the list of instances, neither do they document their "health". Anyway, all of them have official instances (Nitter - nitter.net, Bibliogram - bibliogram.art, Teddit - teddit.net) that seem to work fine, should the handler just redirect to them? Another option would be hardcoding the lists of instances and choosing a random one, I guess.

Also, should I rename invidious-handler to nx-invidious-handler now? @Ambrevar

jmercouris commented 3 years ago

Yes, the name should be nx-invidious-handler. We think a consistent naming scheme will aid in discoverability, much like how emacs plugins are often 'el-xyz'.

danrobi11 commented 3 years ago

I thought you guys would probably like that ;) Then let me share this as well.(Not an instance, im just sharing) Available since yesterday. RAMBLE is a Reddit-like accessible over the clearnet, Tor, I2P and Yggdrasil. Enjoy!

kssytsrk commented 3 years ago

It's all done! I renamed the repo and the package to nx-freestance-handler and added the redirectors to Teddit, Bibliogram and Nitter. @danrobi11 hope you find it useful!

The configs shouldn't break much aside from the fact that everyone has to replace "invidious-handler" package name with "nx-freestance-handler".

jmercouris commented 3 years ago

Thank you for the update! I think it is OK to close this issue for now, if something comes up again, please feel free to reopen!

danrobi11 commented 3 years ago

hope you find it useful!

@kssytsrk Thank you much!

atlas-engineer / nyxt

Feature Request: Invidious Public Instances Redirect From YouTube #930

JSON code showing Invidious Instances as of Today: