search

minetest / minetest

Luanti (formerly Minetest) is an open source voxel game-creation platform with easy modding and game creation
https://www.minetest.net/
Other
10.85k stars 2.04k forks source link

Support minetest:// custom browser protocol #7400

Open tacotexmex opened 6 years ago

tacotexmex commented 6 years ago
Issue type
Summary

For the sake of ease-of-use, speed and interoperatibility between desktop and the web, Minetest can have its own custom browser protocol interpreter.

Protocol URLs will start with minetest:// followed by a agreed upon scheme. Useful URLs I can think of:

I’m sure there are more examples to be found.

  1. Enable instant server joining from the web (community web sites, forums, IRC, Discord etc)

  2. Make Content DB a tightly integrated component of the Minetest ecosystem. Users can install content from the convenience of a regular browser instead of, or in addition to, the formspec based browser.

Task list

Find out how to register protocols on

nerzhul commented 6 years ago

interesting idea, you expect the client to be launched connecting to this server then, but currently we need authentication, an intermediare form is required to set the authentication on the client

tacotexmex commented 6 years ago

You're right, most servers will need password. The client saves the previously used user name so that should be covered. I figure that at least the URL can boot the client (if not open) and enter the server details in the "Join game" tab for easy access.

The bigger win imo is the ability to use Content DB for installing mods and other content, not just browse them. (The in-game browser is horrible, sorry. The formspec system is at fault though, no one else.)

rubenwardy commented 6 years ago

This is something I'd like to add for contentdb

ClobberXD commented 6 years ago

An issue might arise if people have multiple versions of Minetest installed side-by-side, or even if they have one portable version, since the browser might not know what or where Minetest actually is...

tacotexmex commented 6 years ago

At least Firefox is able to display multiple application options on click, I imagine the other browsers work the same way.

rubenwardy commented 6 years ago

It should be implemented using JavaScript to detect whether the protocol is available

tacotexmex commented 6 years ago

Not sure that’s possible. Either way detection is not really needed, right?

It seems possible after all.

ClobberXD commented 6 years ago

As an aside: If the MT installer for Windows is still a thing, it could also be used to create a protocol handler during install time

tacotexmex commented 6 years ago

For macOS one simply needs to edit the app bundle’s Info.plist in this manner.

red-001 commented 6 years ago

previously discussed in #3527.

tacotexmex commented 6 years ago

3527 was before Content DB and the in-app mod Content downloader, it’s usefulness just got a lot clearer thereby.

ghost commented 5 years ago

I think a better way to do minetest handling of games, mods, and texture packs would be to use a mime-type or file extension instead of using URLs in that way. That way any website that wants to offer a minetest mod can just serve up the usual .zip file but using Content-Type: application/x-minetest-mod which can then be associated with a mod installation script.

Calinou commented 5 years ago

@kkabrams As far as I know, one can't override the Content-Type from GitHub Releases or ZIP repository downloads. Lots of mods serve downloads from GitHub, so it holds an important role in mod distribution. Therefore, it sounds difficult to use in practice.

ghost commented 5 years ago

Then how about a manifest file that can contain whether the zip file is a mod, texture pack, etc? I'm looking into if there's a standard format for manifest files inside zips atm. (There doesn't appear to be so far.)

rubenwardy commented 5 years ago

Then how about a manifest file that can contain whether the zip file is a mod, texture pack, etc? I'm looking into if there's a standard format for manifest files inside zips atm. (There doesn't appear to be so far.)

The current standard is this:

mercorii commented 4 years ago

What's the situation with this issue?

It would be nice to be able to pass information to servers via shareable links.

For example, if you want to gather up with friends in a selected place, on a specific server, you could create a link that would tell which server to join, and the link could contain additional parameters to be passed over to server, telling server to do server implementation specific things. such as spawn you at the coordinates given as parameters.

Use case

Bob wants to hold a meetup with friends, on a specific server at a specific location. Some of the friends have never really played minetest - they have the game installed, but only couple of minutes logged in. To make the gathering easy. Bob shares a special minetest url with his friends via IRC. Using the link everyone can easily join the same server at the same location (given that the server would support passing spawn coordinates, and then would do something with that information).

The link would then need to contain following parts:

Url in full would then be:

minetest:join?server=<ip or domain>&location=<x>,<y>,<z>

This way, the application minetest would be given instructions to "join" a particular server.

Additionally given that the idea of this specific link would be to join a specific server, the domain could be given in the authority part, making minetest act similar to a browser:

minetest://<ip or domain>?location=<x>,<y>,<z>

The above example urls are constructed based on apple's instructions and the wikipedia page on URI. I just read them now the first time, so there's a possibility that I'm misinterpreting some parts of URL construction.

ghost commented 4 years ago

I've been using my own handler script for minetest links for a couple years now. I use links like: minetest://user@server:port/. The port is optional if there's a SRV record or the server uses port 30000. The user is optional because it defaults to the username of the user running the script. Maybe I should build a windows version as an exe that only handles minetest links? (Something you can point your webbrowser at when it asks "what should we run minetest links with?")

rubenwardy commented 4 years ago

In terms of the user-facing side, I suggest using a form like https://play.minetest.net/join/123.12.31.212:30000/. Android has the ability to automatically open certain http URLs in an app when an app is installed. On other platforms, that page would contain JavaScript to open the minetest:// schema if it is supported. Otherwise, it would show a help page.

The benefit of this is the ability to show an informational page to users that don't have the app installed. I believe this is how smart links tend to work, I'd need to look into them more

mercorii commented 4 years ago

I'm wondering if minetest: schema could be used for improving login process in the future? Perhaps it could be used for creating something like a slack-style magic link that carries an auth token in it.

Such a magic link would look like: https://www.mysite.com/magic_link_signup?token=abcd1234.

If servers can do an external authentication check, minetest: scheme would allow a way for passing the token, without the player being required to fill forms inside minetest. benrob0329 had a related proposal in his comment on another issue: https://github.com/minetest/minetest/issues/8138#issuecomment-573215104, but the problem there, if i understood correctly, was that formspec was not available for use before authentication.

appgurueu commented 4 years ago

For Unix, it works like this

mk-pmb commented 3 years ago

Below, how I register my URL handlers on Ubuntu (trusty to focal) with Xfce. To become effective, the MIME applications cache has to be rebuilt. I do this by restarting my X session, but there should be less intrusive ways. This website sounds like my approach shoud work for all FreeDesktop/XDG environments: https://edoceo.com/sys/xfce-custom-uri-handler Exec= and Icon= may use absolute paths. Icon= supports PNG and SVG, if a suitable SVG library is installed.

$ which example-url-handler
/home/ubuntu/bin/example-url-handler
$ cat ~/.local/share/applications/example_url_handler.desktop
[Desktop Entry]
Version=1.0
Type=Application
Encoding=UTF-8
Name=URL Handler Example
Comment=
GenericName=
Exec=example-url-handler %u
Icon=utilities-terminal
StartupNotify=true
Terminal=false
Categories=GNOME;GTK;Network;WebBrowser;
MimeType=x-scheme-handler/pop3;x-scheme-handler/pop3s;x-scheme-handler/smtp;

That %u seems to be quoted already. I just tested with this dummy input that only vaguely resembles a URL: http://áccènt@example.test/?'"$PATH=${PATH}/"' > /dev/null&½¥=Ω At first I had only one double quote character, and xdg-open complained about unpaired quotes. When I added the 2nd double quote character, my script was invoked with the input verbatim, except all double quote characters were removed.

mk-pmb commented 3 years ago

Just tested on Windows 8.1 Pro, 64 bit:

REGEDIT4

[HKEY_CLASSES_ROOT\minetest]
@="MineTest Protocol"
"FriendlyTypeName"="MineTest URL"
"URL Protocol"=""

[HKEY_CLASSES_ROOT\minetest\DefaultIcon]
@="notify.exe,0"

[HKEY_CLASSES_ROOT\minetest\shell]
@="open"

[HKEY_CLASSES_ROOT\minetest\shell\open]

[HKEY_CLASSES_ROOT\minetest\shell\open\command]
@="c:\\path\\to\\notify.exe MineTest-URL \"%1\""

; This comment guards the blank line above, which might be mandatory on Win XP.

Make sure the .reg file is on a drive that the admin user can access, e.g. not a virtual drive aliased by a user via the "subst" command. The association becomes effective immediately. You can verify this by creating a new shortcut on the desktop, for a URL as the target. Also I have no idea why only the icon program is looked up in PATH, but not the command (it has to be an absolute path). Before testing, I was convinced that Windows would resolve the command path as well.

mk-pmb commented 3 years ago

What's left to do here? Can I help make progress?

sfan5 commented 3 years ago

Regarding the first point, I think supporting only minetest://connect/$ip_or_domain/$port for now is sufficient. Upon opening, the client should show a prompt asking the user if they want to connect to the given server and also an username/password field.

mk-pmb commented 3 years ago

I'd like to use this as an easy interface to login managers, so I'd prefer we use minetest://[user[:pswd]@]host[:port]/connect/ instead.

rubenwardy commented 3 years ago

you can use existing shortcuts or popen/etc for that

mk-pmb commented 3 years ago

Also I'd like to skip the connect confirmation if the server is saved as a favorite, and that confirmation dialog should have an option like "Add this server to favorites, so in the future you can connect without this question".

sfan5 commented 3 years ago

Would it just use whatever name was last entered into the mainmenu? That sounds a bit inconvenient at times.

mk-pmb commented 3 years ago

@sfan5 good catch. It should probably auto-connect only if the URL contains a username and a password (even if empty). otherwise, URLs to /connect should just pre-fill the join game tab.

mk-pmb commented 3 years ago

I'll try and make a prototype as a separate project, and if you like it, we can later package it together with MT.

Emojigit commented 3 years ago

I suggest:

appgurueu commented 3 years ago

I suggest:

* `minetest-connect://FooBar@example.com:30000` for servers

* `minetest-cdb://content.minetest.net/kay27/mineclone5` for CDB (MCL5 for example, this can also be used to download texture packs and mods)

Why use two schemes if only one is needed?

ghost commented 3 years ago

I suggest:

* `minetest-connect://FooBar@example.com:30000` for servers

* `minetest-cdb://content.minetest.net/kay27/mineclone5` for CDB (MCL5 for example, this can also be used to download texture packs and mods)

minetest-cdb: seems like a bad idea. I'd much rather have content-db stuff use a specific file extension or content-type instead of trying to put that metadata into the URI scheme. maybe .mtz for [m]ine[t]est[z]ip and then peoples' OSs can pass all the .mtz files to minetest, no matter if they got it from http, ftp, or off of a CD or USB drive. maybe content-types of something like, application/minetest-mod, application/minetest-texture-pack, application/minetest-game. I've heard the argument before that this isn't a practical solution because most places that people host these zips don't allow custom filename extensions, or content-types. Well... those places suck in their inflexibility. I dunno.

rubenwardy commented 3 years ago

I'd prefer minetest://content.minetest.net/packages/kay27/mineclone5/. To begin with, it should be based on the hostname - it should match contentdb_url in the config. As Minetest will need to load the package list and other API requests, there isn't much flexibility on changing the ContentDB URL

ghost commented 2 years ago

Necroing because I have an important point in case this continues:

I've spotted a potential issue with the idea of having the database address in the URI; namely, it allows people to create links that cause MT to download and install anything from anywhere. This is a problem for obvious reasons, and it's especially an issue for new users not understanding the safety issue and installing malicious code from somewhere.

This could be partially mitigated with a warning when connecting to a new database, however it's still a major problem.

Also, I would suggest using minetest://content/<data> and minetest://connect/<data>, where data is a comma-separated list of keyword parameters such as address=0.0.0.0. This makes it less likely to be mistaken for a normal URL, and means that the parameters and their positions aren't hardcoded.

mk-pmb commented 2 years ago

it allows people to create links that cause MT to download and install anything from anywhere.

I think that's what @rubenwardy meant with

it should match contentdb_url in the config.

I understood it as that MT will deny, or at least drastically warn about, any content URLs that do not start with the configured contentdb_url.

This makes it less likely to be mistaken for a normal URL,

What is a "normal" URL? The closest thing I can find in RFC 3986 would be the normalizer, but that doesn't seem to fit.

and means that the parameters and their positions aren't hardcoded.

I seem to have missed something here. Can you give an example of what would be confusing and what to better write instead?

ghost commented 2 years ago

I understood it as that MT will deny, or at least drastically warn about, any content URLs that do not start with the configured contentdb_url.

Then why have the DB address in the URI at all, if it will be denied?

What is a "normal" URL?

Basically a website URI, like https://github.com or example. "Normal" doesn't mean anything other than what most people think of when they hear "URI".

I seem to have missed something here. Can you give an example of what would be confusing and what to better write instead?

For example, the minetest-connect://FooBar@example.com:30000 suggestion has a fixed format; username, then hostname, then port. This reduces expandability and expressiveness for little to no benefit. minetest-connect://username=FooBar,address=example.com,port=30000 is how I would prefer it be formatted.

mk-pmb commented 2 years ago

Then why have the DB address in the URL at all, if it will be denied?

So pro users, or future versions of default MT, can arrange for smarter alternatives than just deny.

has a fixed format; username, then hostname, then port.

Yes. Those are defined by the URL schema. We're trying to make a URL handler. We should really stick to the URL schema as strictly as we can, or we will run into all kinds of compatibility issues because operating systems (or desktop environments, mail and messenger programs, etc.) will have certain expectation about what a URL is. If we fail to match them, the OS (or app) won't make the URL clickable, won't apply our URL handler, or will try to "fix" the broken URL.

For the kind of arbitrary named pairs that you suggest, URLs have a query string part.

ghost commented 2 years ago

Yes. Those are defined by the URL schema...

The format of everything coming after the minetest:// part is completely up to the handler. Most URI schemes don't comply at all with that format (take data:text/html,, or the steam URI format, or anything other than the most basic URI schemes for example). The URI scheme used by minetest won't be recognized by almost anything anyway; short of being able to write markdown, bbcode, or HTML, most sites only consider http:// and https:// as valid URLs. The worst case scenario is really that the URI looks like plain text.

So pro users, or future versions of default MT, can arrange for smarter alternatives than just deny.

Is there an actual usecase for this though? Forks are likely to have their own CDB, and I don't see a practical application for allowing links to nonstandard databases that's worth the security risk. At most, a "database ID" should be definable which tells MT that the requested package isn't on the selected contentdb_url.

ghost commented 2 years ago

Should probably mention the difference between URL and URI here. We're talking about a URI handler, not a URL handler. A URL is one specific type of URI, which doesn't concern this discussion.

URIs are generic "links"; URLs refer specifically to a location on a computer network and the method to retrieve it.

mk-pmb commented 2 years ago

Right. However, a lot of messengers and similar apps will only make your URI (or even IRI) clickable if they resemble a URL enough.

ghost commented 2 years ago

We don't need it to be clickable on most things, to be entirely honest. This tool would be much more useful on the forums and similar than anywhere else.

rubenwardy commented 2 years ago

The reason I suggested minetest://content.minetest.net is because I assumed that the URI needed to have a valid domain in it. As this is not the case, minetest://content/ is fine.

As for portability, I'd suggest wrapping the URLs with a HTTP landing page. So instead of linking directly to minetest://server.com:123, you'd link to https://open.minetest.net/join/server.com:123/ or something. That page would then attempt to open the scheme, or show instructions and a link to download Minetest if it fails

One blocker to this issue would be #9166

For example, the minetest-connect://FooBar@example.com:30000 suggestion has a fixed format; username, then hostname, then port. This reduces expandability and expressiveness for little to no benefit. minetest-connect://username=FooBar,address=example.com,port=30000 is how I would prefer it be formatted.

you can expand it with query arguments like minetest://foobar@example.com:30000/?something=value

I don't have too much of a preference though. The benefit of using the common hostname format there is it's more consistent and familiar to users, but that doesn't matter too much - also breaks consistency with minetest://content/. End users will be using open.minetest.net instead anyway

ghost commented 2 years ago

As for portability, I'd suggest wrapping the URLs with a HTTP landing page.

Is that approach for mobile compatibility or something? I'm not sure I understand the need for a HTTP redirect page.

If it's just to provide help for people clicking on the link without minetest installed, then that seems a little unnecessary; other launchers that have custom URI schemes don't do the same because it's pretty much a given that anyone attempting to install a package or join a server already has the application installed.

(Also this just adds yet another web service that fork developers would need to maintain or remove support for, whereas a normal URI handler doesn't need any work to be adapted for a fork)

The benefit of using the common hostname format there is it's more consistent and familiar to users, but that doesn't matter too much - also breaks consistency with minetest://content/.

There's also an argument that consistency with other URI schemes is a bad thing in part, because we don't want the minetest URI format to be confused with a different scheme or a normal URL. Similarity with the particles of the format is good because it makes the URI easier to type when it follows some minor conventions (for example we don't actually have to have the // after minetest:, but it makes it easier to read and write), however it can be confusing when a URI is formatted exactly the same way as a URL but has entirely different functionality.

I'm in favor of using the ?foo=bar&baz=etc format if that's what people want, for example minetest://connect/?address=0.0.0.0&port=3000

appgurueu commented 2 years ago

As for portability, I'd suggest wrapping the URLs with a HTTP landing page.

Is that approach for mobile compatibility or something? I'm not sure I understand the need for a HTTP redirect page.

If it's just to provide help for people clicking on the link without minetest installed, then that seems a little unnecessary; other launchers that have custom URI schemes don't do the same because it's pretty much a given that anyone attempting to install a package or join a server already has the application installed.

(Also this just adds yet another web service that fork developers would need to maintain or remove support for, whereas a normal URI handler doesn't need any work to be adapted for a fork)

It's not that much effort (no "webservice" needed); GH pages probably suffices, as opening the URL can be done all clientside. It's just a convenience as HTTP links are usually handled better than custom, unrecognized protocols - and we can include the install instructions if opening the link should fail.

rubenwardy commented 2 years ago

other launchers that have custom URI schemes don't do the same because it's pretty much a given that anyone attempting to install a package or join a server already has the application installed.

Discord does this. And this assumption is incorrect - imagine a friend sharing a link to join the server they're on. Also, what if they only have a portable version of Minetest? Such a page should also show copyable address/port fields

(Also this just adds yet another web service that fork developers would need to maintain or remove support for, whereas a normal URI handler doesn't need any work to be adapted for a fork)

Not our concern really. But as said, it can just be a GH pages website (which would require JS)

It's not that much effort (no "webservice" needed); GH pages probably suffices, as opening the URL can be done all clientside. It's just a convenience as HTTP links are usually handled better than custom, unrecognized protocols - and we can include the install instructions if opening the link should fail.

Exactly this. Having a wrapper like this makes the links very portable

ProgrammerWhoPrograms commented 2 years ago

https://open.minetest.net/join/server.com:123/ would be a single point of failure AFAICS. I concur with @mercorii that the URI shouldn't use a '//' if there is no authority component afterwards. I.e. it should use minetest:/server/minetest.example instead of minetest://server/minetest.example, to use the example in OP.

rubenwardy commented 2 years ago

I think that the added ease-of-use makes up for it being a single point of failure. Server websites could do a similar thing themselves instead, open.minetest.net wouldn't have a special status

ghost commented 2 years ago

How about, both? minetest://user:pass@host:port/ can be used to directly connect, but a "share this server" thing will default to opening a page to some https link that gives information for people that don't have minetest installed yet, and /that/ page can then, have a "open server" button that uses a minetest://user:pass@host:port/ URI? If you want to be able to confirm that a user was able to open the link when they click on it, then have it so the minetest config has a setting to pick which server to send a "the user connected" message to.

ghost commented 2 years ago

in the case of portable installs, you can make minetest do a check to see if something already has a minetest URI handler set, if not, put a button or popup (yeah, I know, annoying) asking "would you like to set this version of minetest as the default handler for minetest links?"