Move room data from JSON files to database

[lazerwalker]

This feels important for unlocking some scripting capabilities. It's also fraught.

Challenge 1: This suggests we need our own editing environment. Some sort of minimal web-based JSON editor gated by roles (using Firebase auth, then checking for mod status)

Challenge 2: If a runtime script fucks with data, we need a way to reset everything to "pristine" state. One way to do this is to store the 'canonical' data in JSON (we probably want this regardless) and offer an admin button to reset it. An open question is whether there's additional value in having both a 'canonical' copy and a 'live' copy in the database, separate from disk.

Challenge 3: If we have 'canonical' data on disk, and runtime data in DB, what is our process for editing on staging and then PRing changes back onto git?

[lazerwalker]

Closing #411, but copy/pasting what I wrote there:

Right now, editing room content requires committing data files to git. It'd be nice if room editing could happen in the browser, making it easier for non-programmers to contribute content to the production instance.

For the simplest version of this, a few things need to happen:

• [ ] Right now, there are TS files that contain JSON room data. This data should be moved into CosmosDB, and read from there when we need to return room data
• [ ] Build a simple web editor to manipulate those JSON blobs. This can be a standalone website (that ties into our exiting auth infrastructure) that renders a list of rooms, as well as an editable blob of JSON for a selected room with a "save" button (that calls an admin-authenticated server function that saves the new blob of data)

Potential next steps after this:

• [ ] A proper web form with discrete fields a nicer experience than hand-typing JSON
• [ ] Can we have syntax highlighting for our link syntax, and/or autocomplete for recognized rooms/etc when linking?
• [ ] Similarly, syntax highlighting for Storyboard scripting
• [ ] What is the conceptual best way to view this data? Are there helpful visualizations other than "a list of items that each shows a web form"?

This task is a large chunk of work, but it's a good standalone task for someone who wants to take on something meaty. The first step (refactoring room data to come from CosmosDB) will require touching existing server infrastructure, but the bulk of the task is building a standalone thing.

And after that:

I like the production JSON being static files.

An "edit mode" would be helpful for 2021.

Current thoughts: find a web service like Gists that let us edit JSON from the browser. A toggleable flag in an instance switches to loading JSON from an API. Might be tricky to support multiple files, but a single working file is probably ~fine.

Slightly more complex version: host our own Monaco editor, store in Redis, starting from disk files as a default. Push a button to download.

I think we need things to be in Monaco, since our priority is now "enable this to be dynamic" instead of simply "make editing simpler"

[lazerwalker]

Brain-dumping here as I hopefully sit down to do this soon:

• [x] Each room on-disk is its own file, or a logically-organized blob of JSON, that we can dump from an editor
• [x] Script that dumps disk files into Redis, with each room at the key room_${roomId}
• [x] Instead of returning a static "rooms" blob to clients on pageload, return current data for just that room when moveRoom is called (TODO: keep an eye on performance)
• [x] We have a separate 'editor' screen (maybe a separate HTML page, even?) that gates on admin
• [x] This editor screen has a list of room descriptions (run KEYS room_ in Redis to return all keys that start with room_)
• [x] Selecting a room opens its JSON in Monaco
• [x] There's a discrete save button (no auto-save for now) that dumps that back to Redis
• [x] There's a button in that UI to generate a zip file containing room JSON files, to be manually saved to disk and updated in git as a drop-in replacement for server/src/rooms/data

• [x] That editor has an 'import from disk' button that loads the version of JSON stored on-disk in repo [this already exists in Server Settings, just call same function here]

Polish tasks taking this from "proof of concept" to "production ready"

• [x] The way I'm planning to handle the editor will break both directories with JSON files and single files with multiple rooms (both of which currently exist on-disk). Confirm I'm okay with wiping those out for the sake of simplicity. Slash think through if there is an easy way we can represent hierarchy in the editor.
• [ ] Properly handle note wall data in Redis
• [ ] Client is smarter about how we handle links, when the set of all rooms may not be known, and then remove sending our full static room data on pageload. Easy check here: links within room text should not be "undefined", and we should get the "dev warning" console message
• [ ] Fix room presence indicators in map and room list (not getting data?). Also, make sure room list includes everything.
• [ ] Write a JSON Schema for our room format (ideally synced to our TS type definitions), run that in Monaco. See https://microsoft.github.io/monaco-editor/playground.html#extending-language-services-configure-json-defaults for a working playground example. Also see https://github.com/YousefED/typescript-json-schema

This then also opens up the road to dynamically changing each room at runtime

There are performance considerations here, both to waiting on getting room data to render a room, and on massively increasing our redis reads

[lazerwalker]

Worth noting: we currently can't integrate Monaco into our Parcel-based build setup. See https://github.com/microsoft/monaco-editor/issues/2966. This bug is currently tagged for fixing in the VS Code team's June 2022 milestone, but it's been bumped back from two previous monthly milestones, so it seems very likely it will get pushed back again.

[lazerwalker]

This technically works! Separate from the TODO items to look into above, a list of things I suspect we may need in the future, pending people complaining:

• [ ] What is our dev process for shared editing? Do I need to spin up a staging server?
• [ ] File-level locks to show when someone else is editing a file? https://github.com/yjs/yjs would be the tool to investigate to see if real-time collab on a single file is possible, but don't get too yak-shaved
• [ ] Switching back to Monaco when it's possible
• [ ] Is it easy to add a JSON schema to Ace?
• [ ] Can we have some custom linting or syntax highlighting for our Twine-like syntax?

[lazerwalker]

Gonna close this for now and re-open issues as they arise