Open Envian opened 11 months ago
Envian
commented
11 months ago Also, while I am fully ready to implement this completely, as you may be aware node & react is not my area of expertise. If you'd prefer to limit my contributions, I would not be offended.
joelspadin
commented
11 months ago TOS
Definitely something we should check. I hadn't looked into this yet, but I figured gists are meant for sharing code snippets, notes, etc. which isn't far off from how this would use them.
File Format
I figured pretty-printed JSON would probably be best for this. If it's possible to get the ID of a new gist before creating it, we could include the xivplan URL to that plan in the data, either as a header that we strip out before parsing, like
// https://xivplan.netlify.app/gist/<id>
{
...
}or as a field in the data like
{
"$url": "https://xivplan.netlify.app/gist/<id>",
...
}so that it remains standard JSON.
GUI Authorize with Github button will be added to both the Save & Load menus under a "GitHub Gist" tab.
This seems reasonable to me. Alternatively, we could have a sign in button at the top of the Save and Load menus and then only show the relevant tabs if you're signed in, but I'm not sure if that's easier to use or just more confusing.
Gists will be saved with the MIME type of application/x-xivplan for identification later.
I haven't checked what the specification for MIME types says, but it looks like a lot of specialized JSON formats tack +json onto the name, so maybe application/x-xivplan+json?
The load menu will show all gists with the MIME type application/x-xivplan.
Exactly what I was thinking (aside from not having decided a good way to identify which gists should be shown; I like the idea of using the MIME type).
The load menu will also include a text box "Load from URL" to load public plans*.
I'm not sure how necessary this is when you can just open the link in a new tab.
The share menu will be updated to include a Share Gist tab, which is only visible when working on a gist. This tab will have a button to allow the user to make the gist public, and when public the share URL will be shown.
Seems reasonable to me. Maybe the load menu should also display the public/private status and have a share/copy link button as well?
Link Format
That pretty much matches what I was thinking. From what I understand, non-hash-based URLs are generally preferred, though since xivplan is a static site with no other pages currently, using a hash is fine (and consistent with the share links, where I had to use a hash to work around server URL limits).
GitHub Authorization Quirks
It is possible to do OAuth authentication in a popup window. Not sure this is the best possible example of it, but I've used code similar to https://github.com/jlumbroso/react-github-login for this before.
Also note that this requires having a separate server set up to relay the OAuth messages with the app's private API key (otherwise there's no way to keep said key private). I'd need to look into whether that could be done using Netlify's serverless functions instead, which should be free (as long as I don't get more than 125k sign ins per month).
Envian
commented
11 months ago TOS
Github's API TOS doesn't explicitly forbid much. Just illegal activity, spam, advertisements. etc.
File Format
I'm a fan of the $url field. while we won't know what a file's URL will be until its created, we can create the file empty, and get the URL before posting the real content.
we could have a sign in button at the top of the Save and Load menus and then only show the relevant tabs if you're signed in, but I'm not sure if that's easier to use or just more confusing
I like your idea of having an "Authenticate with Github" option somewhere, and hiding the gist tab unless authenticated. Couple that with selecting only a single third party site to integrate with, will keep the UI more clean.
I haven't checked what the specification for MIME types says, but it looks like a lot of specialized JSON formats tack
+jsononto the name, so maybeapplication/x-xivplan+json?
Adding +json is an optional suffix we can add - and in theory we can use this to differentiate between application/x-xivplan+json and application/x-xivplan+gzip which would be the URL's format. I do like this plan.
I'm not sure how necessary this is when you can just open the link in a new tab.
The idea behind the "Open URL" input was if you had the link to someone else's gist, but didn't have the XIV Plan link. There wouldn't be an intuitive way to open that gist without constructing the url yourself. However, if we add $url to the plan JSON, that solves this problem.
Seems reasonable to me. Maybe the load menu should also display the public/private status and have a share/copy link button as well?
I did think about that, but that would clutter the load menu. I can see the use case for making a plan public, but private is much more niche. That's why I suggested making publishing a plan a one way operation.
I took a look at the source code for this project - this would work well so long as the popup is allowed. Unfortunately an in-app popup (Similar to how Open File is a "popup"), would require an IFrame, but there's a good chance that GitHub blocks IFrames for security.
Also note that this requires having a separate server set up to relay the OAuth messages with the app's private API key
Ah, I didn't realize there wasn't a serverless authentication flow. But, in hindsight, this makes sense.
With a relay, we would want to verify the API requests on the server, otherwise we could be forwarding questionable requests. This doesn't necessarily need to be a day 1 feature, but something that should be strongly considered.
joelspadin
commented
11 months ago I took a look at the source code for this project - this would work well so long as the popup is allowed. Unfortunately an in-app popup (Similar to how Open File is a "popup"), would require an IFrame, but there's a good chance that GitHub blocks IFrames for security.
In a previous project, I used an actual popup window rather than an iframe. It worked pretty well. (Code could probably be written much more cleanly though.)
Daendaralus
commented
8 months ago Heya, not sure what the state of this currently is, but I've recently forked and made some changes for myself. Among them sharing via the paste.myst.rs API. From what I can tell, this seems to comply with their TOS. Although it's realistically not nearly as future proof as something like github.
My changes are very quick and dirty (Not helped by 0 experience with TS or react) and as such various parts of the code are a bit mangled. Not particularly valuable as a contribution I believe, but maybe still of use to you.
It's currently hosted on a github pages, here's an example plan: https://daendaralus.github.io/xivplan/#/plan/ttz3wr4o
Envian
commented
7 months ago Hello fellow criterion enjoyer!
Thanks for posting this. I haven't made any notable progress on this ticket due to a change in my professional life - that and the complexity of adding authentication, session management, backend validation, and all the other processes required for secure server to server communication left me demotivated.
paste.myst.rs solves a lot of those problems though. I've reviewed your saving/loading code and the fact that the server allows for creating pastes without authentication is a game changer. I did review their site and agree that using them for storing plans is technically not against their TOS, and can serve as a great starting point for URL based sharing.
I'll see if I can find some time to add a GUI around this which would work with the main package.
Envian
commented
6 months ago Hey @joelspadin, I did some research on pastemyst and, from a technical perspective, I can see two options for implementation. I'd like your feedback on which, if either, approach you feel is more appropriate.
Authentication with the pastemyst API is a simple bearer token. Users would go to the paste.myst.rs site, connect their github account, and receive an API token. This token can then be copied into the xivplan app to allow us to fully read/write/query pastes on the user's account.
When authenticated, we could have pastemyst be an alternative to local storage, and also double as a sharing system. Plans will be set private by default, with a checkbox to toggle to public in the Share dialog. Unauthenticated users would have the ability to open plans that have been shared by other users using the unauthenticated API.
To mitigate the risk that pastemyst becomes unavailable, I can also add a local storage copy of the plan (Separate from the Local Storage) tab so plans can be restored if we lose access to pastemyst.
Pastemyst also has a public, unauthenticated API. This only allows for reading public pastes, and creating new pastes - existing pastes cannot be edited in any form, and are permanent once uploaded.
If we implement this approach, in addition to a simple "Share" functionality, I would add logic to keep track of a plan's current share link, and add checks so we only upload a new copy when necessary, and a plan is only uploaded when explicitly requested. A quick mockup of the GUI, with appropriate disclaimers, looks like this:
In both versions of the API, we are limited to 5 request per second, which is plenty to work with. Each of the API requests will be coming from the user's browser rather than a central server.
joelspadin
commented
2 months ago Sorry, I forgot about this until recently. I don't think I'm going to have time to look into this before the expansion launches (I blame From Software), but I did just finish implementing some support for a feature that I hope will make it a bit easier to share large plans.
The site can now be installed as an app, and on Chromium-based browsers (Chrome, Edge, etc.) it now supports opening and saving JSON files with an .xivplan file extension. You can now save a file and send it to someone (e.g. via Discord), and if they have the site installed as an app, they can directly open the file in the app. I plan to allow for drag and drop of .xivplan files onto the page and saving them via download to at least get some support for Firefox next.
That doesn't solve the problem of having a single location for a plan that you know is the latest version though, so I would still like to work on this eventually.
I'm currently looking into writing guides and sharing more plans, and have need of shorter share links. To serve this end, I'm looking into adding support for (one or more) third party sites. I do see Gists on your roadmap, so I would like to build that feature out.
I will add further details over time, and feel free to ask any questions.
TOS considerations
First and foremost, we need to consider the TOS's of the site(s) we want to use. Making use of Github to host our apps backend is sketchy at best, but isn't explicitly forbidden. For the initial implementation, I will start with Github gists, but other sites may be considered down the line.
File Format
The easiest file format would simply be the full plan link URL. This has the benefit of including the URL of the site within the file itself. Alternatively, a pretty printed JSON file is more likely to be acceptable by github.
Can I get your thoughts on file format?
GUI Implementation
application/x-xivplanfor identification later. Gists will be private by default.application/x-xivplan.* I've been floating the idea of a generic "Load from URL" option that simply performs a
GETto that URL and loads what it finds, but it feels like a bit too much of a power user feature, when plan codes exist.Link Format
Currently planning on only adding a hash based URL. It will look something like this:
https://xivplan.netlify.app/#/gist/github-gist-idGitHub Authorization Quirks
Github authorization is fairly straight forward - however one quirk is that we need to navigate away from the page in order to authorize, then we will be redirected back. This will clear any unsaved state, so I plan on adding a temporary save for the scene. It will be stored on local storage.