ocdtrekkie
commented
4 years ago This sounds fundamentally to be Sandstorm's "drivers" concept, which is only described in the README of the roadmap.
Open zenhack opened 4 years ago
ocdtrekkie
commented
4 years ago This sounds fundamentally to be Sandstorm's "drivers" concept, which is only described in the README of the roadmap.
zenhack
commented
4 years ago I agree there is a close relationship, but there's an important thing here that's not articulated in the roadmap: From the user's standpoint these would seem to act like built-in functionality, but architecturally they'd be treated similar to grains. The roadmap talks about some drivers being built-in and some being apps, but doesn't talk about "apps" that look like built-ins.
Quoting Jacob Weisz (2020-03-12 16:21:10)
This sounds fundamentally to be Sandstorm's "drivers" concept, which is only described in the README of the roadmap.
-- You are receiving this because you authored the thread. Reply to this email directly, [1]view it on GitHub, or [2]unsubscribe.
Verweise
ocdtrekkie
commented
4 years ago It does also specifically discuss drivers handling web protocols like IMAP or IRC. I think a driver to support web publishing would fall in line with that as well.
Presumably if we were to flesh out that roadmap entry further, it would make sense to specify some drivers would be pre-provided and not appear in the grain list. (Ideally speaking, IMAP and IRC handling should be done as independent "apps" as well, but I think it'd be silly to have an IMAP protocol app appearing in the grain list of each user who wants to also use an email app.)
The discussion about @mnutt's Office Preview app also brought up the discussion that there are probably things that could be "apps" that run in containers but would be silly to maintain grain storage and show in the grain list. Would some "system grains"/"drivers" make sense to be user-specific (so that they are finely-instanced from other users' data), but hidden from display in the UI?
I assume some drivers would need a single server-wide point of entry though, and should be handled at the server level as well. (For instance, if all requests for a given protocol needed to be accepted on a given TCP port on the server.)
zenhack
commented
4 years ago Quoting Jacob Weisz (2020-03-12 16:36:01)
It does also specifically discuss drivers handling web protocols like IMAP or IRC. I think a driver to support web publishing would fall in line with that as well.
Agree, I guess this issue just fleshes out a few details.
Presumably if we were to flesh out that roadmap entry further, it would make sense to specify some drivers would be pre-provided and not appear in the grain list. (Ideally speaking, IMAP and IRC handling should be done as independent "apps" as well, but I think it'd be silly to have an IMAP protocol app appearing in the grain list of each user who wants to also use an email app.)
Agree. Probably for protocols like this you'd still want a "request from administrator" flow, like we've talked about for IpNetwork and friends.
The discussion about [1]@mnutt's Office Preview app also brought up the discussion that there are probably things that could be "apps" that run in containers but would be silly to maintain grain storage and show in the grain list. Would some "system grains"/"drivers" make sense to be user-specific (so that they are finely-instanced from other users' data), but hidden from display in the UI?
This lines up with the subordinate/owned grains discussed in the grains section of the roadmap.
I assume some drivers would need a single server-wide point of entry though, and should be handled at the server level as well. (For instance, if all requests for a given protocol needed to be accepted on a given TCP port on the server.)
In this case we could probably still have them be "system grains," with a capability for the appropriate port, but may have to compromise the document-per-grain principle. We'll likely want to do this with the static publishing example as well, since half the point of the feature is to not have to spin up a whole bunch of separate grains just for static files (but one shouldn't be a big deal, especially if we're maintaining the code and can design it for efficiency).
xet7
commented
4 years ago Would this include:
xet7
commented
4 years ago Well, I don't know would it be possible to:
zenhack
commented
4 years ago Quoting Lauri Ojansivu (2020-03-12 17:07:40)
Would this include:
I think all of those features are orthogonal to what's being discussed here.
* Organizations/Teams/Users with subgroups unlimited hierachy management like I'm doing for Wekan [1]wekan/wekan#802 So same could be managed from Sandstorm Admin Panel? * Sandstorm Admin having ability to automate all Sandstorm apps with Wekan Rules, like managing Sandstorm cron for automated tasks?What is status of Sandstorm cron?
It's implemented, though hasn't hit an actual release yet; you need to run master or wait for @kentonv to kick out the next release. There's also no UI for users to inspect scheduled jobs, but app can create them. See SandstormApi.schedule in src/sandstorm/grain.capnp.
- Scheduled publishing of static website pages?
You could probably implement this in an app just by using the scheduling and publishing APIs.
ocdtrekkie
commented
4 years ago A huge perk to this system would be if this was fundamentally Sandstorm's "plug-in" system at a server level, as presumably Sandstorm would ship with system grains, but presumably someone could choose to implement third party ones?
Another particular perk to this approach would be ideally being able to disable these drivers, so if there was a flaw in web publishing, for instance, and you wanted to turn it off, you could disable it on your server.
zenhack
commented
4 years ago Indeed. Of course there's design work to figure out how that would work from a user standpoint, but in principle yes.
Quoting Jacob Weisz (2020-03-12 17:35:10)
A huge perk to this system would be if this was fundamentally Sandstorm's "plug-in" system at a server level, as presumably Sandstorm would ship with system grains, but presumably someone could choose to implement third party ones?
Another particular perk to this approach would be ideally being able to disable these drivers, so if there was a flaw in web publishing, for instance, and you wanted to turn it off, you could disable it on your server.
-- You are receiving this because you authored the thread. Reply to this email directly, [1]view it on GitHub, or [2]unsubscribe.
Verweise
CaptainCalliope
commented
4 years ago I really like the idea of carving out a wider birth for r&d without adding risk to existing deployments. I also like how this could enable more people to get involved with building the core platform. Very strong product management and contributor organizing implications there. And it also sounds like this could open the door for competing implementations of core-ish systems which down the line would be fantastic for the maturity of the platform+ecosystem.
I think there's a slightly wider discussion here to be had about defining/discovering the best practices and patterns of building the platform. I sometimes describe sandstorm to people as a new kind of server-side operating system, and it sounds like this issue reflects lessons learned about what the architecture of this kind of capability-based operating system needs to be.
As you describe System Grains, I'm trying to imagine an architectural diagram of Sandstorm with the core comprised of cap'n proto + whatever else, and then layers on top of that comprised of drivers, system grains, grains and subordinate/owned/child grains. Is this a pretty complete picture of the core primitives involved (or am I missing any)?
How do these primitives relate, and how do we determine if a given feature is best implemented as a driver, a system grain, a grain, or part of the core?
What unimplemented features from the roadmap would make sense to do as system grains?
Also, are there features or details of the core's current implementation that might be refactored as system grains? For example, could system grains be used to abstract the shell interface such that it could be more easily swapped out? From a UX perspective I'm thinking about how Android increasingly is creating surface area for being able to swap apps out that perform core functions. Is that applicable here?
xet7
commented
4 years ago For ispiration what similar exists on other platforms, at Friend there is DOS Drivers that are modelled to be similar to Amiga DOSDrivers for various devices. There Friend DOS API docs at: https://friendos.com/resources/
For actual Sandstorm "system grains", I would think that first start at some mimimal implementation for a system grain, and then look at what to do next.
kentonv
commented
4 years ago What's being described here sounds like exactly the "drivers" concept to me. Drivers are intended to be interacted with primarily within the powerbox, where there should be little visible difference between "built-in" and app-provided functionality. The only big difference I can think of between built-in drivers and app-based drivers is in where you'd find the "admin interface" -- is it in the Sandstorm admin panel, or is it in the grain list? Though even that's something we could think about bridging... maybe there can be some way for apps to declare that they'd like their UI to show up embedded in the admin panel. Maybe indeed driver grains shouldn't be thought of as being owned by the admin, but rather by the system itself, and so would not show up in anyone's grain list, only the admin panel.
zenhack
commented
4 years ago The only big difference I can think of between built-in drivers and app-based drivers is in where you'd find the "admin interface" [...] Maybe indeed driver grains shouldn't be thought of as being owned by the admin, but rather by the system itself, and so would not show up in anyone's grain list, only the admin panel.
Yeah, I think this is the key thing; I'm basically looking for a way to use driver apps without having them need to be in a grain list somewhere.
ocdtrekkie
commented
4 years ago I think it makes sense for drivers as described to live at the server level. If it was an app that was in a user's grain list, we'd call it an app.
As we add more functionality to Sandstorm core, I'm thinking about how to keep attack surface small and reduce the potential impact of vulnerabilities. An idea that's occurred to me in the past (I think I may have mentioned it as a side note in other conversations) is that we could re-use sandstorm's sandboxing to implement core features as "system grains" -- these would act like grains in many ways, including being able to fulfill powerbox requests, and would be managed by the same supervisor implementation as regular grains, but they would differ in that:
As an example, I am thinking about re-working Sandstorm's static publishing to use the powerbox, and given the relative complexity of the API in
web-publishing.capnp, I am nervous about adding code to handle all of that into the privileged part of Sandstorm (Especially if writing in in C++ -- I would sleep better at night if more of our code had memory safety guarantees -- but even without that there are other things worth locking down). Instead, we could implement the web publishing functionality as an app that runs as a system grain. It would service powerbox requests for web publishing, and Sandstorm would shell out to it to handle requests for static domains.Thoughts?