Open ssokolow opened 2 years ago
Cargo has a command called cargo locate-project --message-format plain
which prints the root of the project to stdout, I think we can use this to find the root of the project.
That would involve spawning an additional subprocess on each invocation. Do you have evidence that doing it that way is an improvement over just walking up the filesystem ourselves to find Cargo.toml
?
After all, not all build systems that will need to be supported are going to have an equivalent command and some (eg. npm
) are horrendously slow to start up, so the machinery to walk up the filesystem looking for a file is going to need to be implemented either way.
UPDATE: ...and it would be an additional point of failure, since the command being invoked to identify the path to sandbox could itself become compromised and can't be run sandboxed.
The problem with manually finding Cargo.toml
is workspace.
In that particular situation, finding Cargo.toml
is not enough,
Edit:
After finding the first Cargo.toml
, you would also have to check whether its parent dirs has Cargo.toml
and verify that.
IMHO in the v0.1 MVP we could just invoke an external command.
I also would like the following firejail
sandbox options to be applied when building packages:
private-etc alternatives,ca-certificates,ld.so.cache,ld.so.conf,ld.so.conf.d,ld.so.preload,resolv.conf,ssl
private-dev
private-tmp
caps.drop all
nonewprivs # Make setuid binaries has no effect
noroot
dbus-system none
dbus-user none
blacklist /opt
blacklist /media
blacklist /mnt
blacklist /srv
blacklist /sys
blacklist /run
blacklist /boot
blacklist /root
blacklist /sbin
blacklist /snap
blacklist /sys
blacklist /var
noexec /bin/su
noexec /usr/bin/sudo
After finding the first Cargo.toml, you would also have to check whether its parent dirs has Cargo.toml and verify that.
That was what the root_find_outermost=true # For workspaces
in the draft config file was.
I was already taking the need to find the last match rather than the first into account.
I also would like the following firejail sandbox options to be applied:
I'm pretty paranoid about that, though some of those may have to wait until v1.1 because I'm not sure per-profile Firejail rules will make v1.0 and I suspect that private-etc
and some of those blacklist
s might break some of the non-Cargo build tools I want to support.
That was what the root_find_outermost=true # For workspaces in the draft config file was.
I was already taking the need to find the last match rather than the first into account.
Sounds good to me.
I'm pretty paranoid about that, though some of those may have to wait until v1.1 because I'm not sure per-profile Firejail rules will make v1.0 and I suspect that private-etc and some of those blacklists might break some of the non-Cargo build tools I want to support.
I think that at least some of them should be applied unconditionally:
private-dev
private-tmp
caps.drop all
nonewprivs # Make setuid binaries has no effect
noroot
noexec /bin/su
noexec /usr/bin/sudo
blacklist /sys
blacklist /boot
I would also like to decide the language to be used for this project.
IMO dynamic-typing or weak-typing languages like python and bash is not suitable for security-sensitive application like this since you can easily work around their typing system.
Since I don't have much experience in Go, IMHO using Rust in this project sounds reasonable.
It seems that firejail
already has a cargo.profile
though IMO it is actually too strict.
I think that at least some of them should be applied unconditionally:
I'm certainly hoping to do more than that for v0.1. I'm just not sure we can do all of the longer list until we have support for per-profile overrides.
Since I don't have much experience in Go, IMHO using Rust in this project sounds reasonable.
I thought using Rust was being taken as a given before we even moved off Reddit. The only reason I even mentioned Python is that, at one point, I was considering starting from a Python script I'd written for something else.
Bash is certainly not something I'd want to write something longer than a dozen lines in (I use Python for my quick "shell scripting" and Rust for my more long-term "shell scripting") and I've never used Go because I don't like how primitive it is.
It seems that
firejail
already has a cargo.profile though IMO it is actually to strict.
I generally only take ready-made Firejail profiles as advice on what I can do in my own from-scratch profiles without causing the program to fail to start.
I'm certainly hoping to do more than that for v0.1. I'm just not sure we can do all of the longer list until we have support for per-profile overrides.
Me too, I also cannot be 100% sure that all of them in the longer list is appliable.
I thought using Rust was being taken as a given before we even moved off Reddit. The only reason I even mentioned Python is that, at one point, I was considering starting from a Python script I'd written for something else.
Bash is certainly not something I'd want to write something longer than a dozen lines in (I use Python for my quick "shell scripting" and Rust for my more long-term "shell scripting") and I've never used Go because I don't like how primitive it is.
Cool.
I generally only take ready-made Firejail profiles as advice on what I can do in my own from-scratch profiles without causing the program to fail to start.
I just thought it can be used as a reference.
I just thought it can be used as a reference.
No problem. Thanks for saving me the trouble of going looking.
I just thought of another thing that'll need to be configurable. Whether to canonicalize (i.e. resolve symlinks) or just "make absolute" when ascending to find the ancestor in the path which contains root_marked_by
.
Some build tools may do one while some may do the other so, to Just Work™ without introducing confusing edge cases that may cause problems, it'll be necessary to have something like root_resolve_symlinks
to enable matching what the command to be wrapped does.
(eg. Rust has std::fs::canonicalize
but no "make absolute" in std
yet while Python calls "just make absolute" os.path.abspath
which sounds more inviting and "correct to intention" than os.path.realpath
, which is how you canonicalize.)
I just thought of another thing that'll need to be configurable. Whether to canonicalize (i.e. resolve symlinks) or just "make absolute" when ascending to find the ancestor in the path which contains
root_marked_by
.Some build tools may do one while some may do the other so, to Just Work™ without introducing confusing edge cases that may cause problems, it'll be necessary to have something like
root_resolve_symlinks
to enable matching what the command to be wrapped does.
That's indeed another thing that needs to be configured.
That said, given that Rust has no "make absolute" in std
yet, I'll need to decide how to go about it.
(i.e. For security reasons, it may be a good idea to just do a one-to-one port of the Python code until something for that shows up in the Rust standard library rather than relying on a third-party crate.)
(eg. Rust has std::fs::canonicalize but no "make absolute" in std yet while Python calls "just make absolute" os.path.abspath which sounds more inviting and "correct to intention" than os.path.realpath, which is how you canonicalize.)
According to here, std::fs::canonicalize
seems to return an absolute path.
It's a wrapper around the libc realpath(3)
function. "realpath() expands all symbolic links".
os.path.abspath
just performs string manipulation based on $PWD
and the given path.
They behave differently in a situation like this:
~/src/project1/Cargo.toml
~/src/project2/Cargo.toml
~/src/project1/src is a symlink to ~/src/project2/src
Canonicalizing will turn ~/src/project1/src/mymod
into ~/src/project2/src/mymod
before we ascend looking for Cargo.toml
while a non-canonicalizing "make absolute" function will not.
Thanks for explaining this to me.
I didn't realize that symlink handling is different.
As for how we do it, since there's no "just make absolute" function in std
yet, I may want to do a one-to-one port of the code for os.path.abspath
from Python and then do some differential fuzzing to make sure that the behaviour is identical to the mature Python implementation.
It's a little too central to the security proposition for me to feel comfortable relying on a third-party crate.
Agreed.
Of course, I'll put it in its own module and file so keeping the licenses separate and the ported code easy to replace is as simple as possible.
Another thought to consider when I have time to do some research. Maybe I should use a proper argument parser but configure it in "the first non-option implies --
(all following arguments are positional)" mode and support options like --allow-network
to make it feasible to do something like "Run nodo --allow-network npx <command>
once to install it, then run nodo npx <command>
to use it".
(The main question being whether there are other details about how commands like npx
function which would make that not the best choice. For example, how does its cache work?)
I am not familiar with npm.
AFAIK about cargo, this seems to be perfectly OK.
npx
is a command that comes with NPM that's essentially a hybrid of cargo install
and cargo run
. It'll download the package you specify, cache it to a user-global install location akin to ~/.cargo/bin
but not in PATH
, and then run the command.
Once WASI's APIs are richer and more crates support targeting it, wax
would be a Rust equivalent with built-in sandboxing for anything published to Wasmer's WAPM repository.
In fact, this project is inspired by wax
's default sandboxing behaviour. (Because WASI uses capabilities-based security, you have to pass in handles for the pieces of the filesystem you want the program to be able to see. wax
defaults to granting $PWD
when you run something with it.)
Sounds amazing, though the combination of downloading and run makes sandboxing it harder.
You can always use the wapm
command to download and run things separately. npx
and wax
are just wrappers to make things as simple as possible.
Also, WebAssembly has its own sandboxing so, if you trust wax
, it can deny the command network access without needing to itself be denied permission to access the network for updates. (Like JavaScript running in a browser with uMatrix.)
In fact, wax
is just a wrapper script around wapm execute
minus the --offline
flag. wapm install <command>
with one sandbox profile, then wapm execute --offline <command>
with another.
IMHO, it’s probably not necessary to sandbox wasm since it has builtin sandbox via its capacity based API.
So maybe we should focus on cargo and npm instead, since these two doesn’t have any builtin sandbox?
With the way I want this to operate, once we've got profiles for Cargo and npx
set up, it's just copy-pasting a dozen lines of TOML and editing the command and subcommand names to support WAPM and wax.
The most time-consuming part would be running the tests to Do It Right™ and confirm my suspicion that, being written in Rust, Wasmer's WAPM CLI tool uses std::fs::canonicalize
for making paths absolute.
That matches with my impression of this project so far.
On the other hand, where shall we put the configuration file?
I think $HOME/.config/nodo/config.toml
will be a good place for such config file.
Also, I think we can configuration file splitting v0.1.x so that users can have cargo-profile.toml
, npm-profile.toml
and etc.
Definitely something compliant with the XDG Base Directory Specification, so something starting with ${XDG_CONFIG_HOME:-${HOME}/.config}
.
I'll want to think about whether to re-implement that to protect people who install with cargo install
without --locked
though.
I'm not sure about configuration file splitting though. I've learned to follow YAGNI the hard way and I haven't yet had a solid rationale that convinces me it's worth the added maintenance burden and risk of overlooking a way it could make the design more vulnerable.
That sort of thing is more for when you want your creation to be a piece of infrastructure a project like Cargo or NPM can integrate support for, and that would divide the responsibility for ensuring the configurations are trustworthy compared to just having all the vetted ones in the same repo as the code which enforces them.
...but I will want to think about whether it's worthwhile to let users extend the default config rather than completely overriding it, given that things like adding new profiles would require them to add new alias
es, and making the rules more nuanced will probably result in a new schema version for the config file.
(I do plan to have a key like schema_version=1
at the top level which can be used to check when the user should receive a warning about outdated rules and how to revise them.)
I'm not sure about configuration file splitting though. I've learned to follow YAGNI the hard way and I haven't yet had a solid rationale that convinces me it's worth the added maintenance burden and risk of overlooking a way it could make the design more vulnerable.
Maybe we should wait for user feedback. If the configuration file is simple enough, then splitting it definitely isn't necessary.
...but I will want to think about whether it's worthwhile to let users extend the default config rather than completely overriding it, given that things like adding new profiles would require them to add new aliases, and making the rules more nuanced will probably result in a new schema version for the config file.
Agreed, we should also provide a sensible default configuration.
Agreed, we should also provide a sensible default configuration.
We should aspire to have 99% of users not needing anything but the defaults.
Just to get it noted down, the main thing I'll want to sit and think about once I've got a proto-v0.1 to experiment with is the schema for the config file.
With needing to support subcommands (eg. fetch
) and flags (eg. --offline
) as means to control whether things like network access should be granted, the draft design shouldn't be the final one.
To achieve that properly, I'll probably want to sit down and write out a sheet with all the different commands (eg. cargo
, npm
, etc.), subcommands, and flags that are relevant to the behaviour of the sandbox, and then brainstorm on the design which can describe them in the cleanest way.
...either that or just treat schema version 1 as temporary and accept that, for 99% of use-cases, it should be expressive enough, even if it's a little inelegant.
Maybe we can have a section like “[profile.cargo-fetch]” and override and extend configurations provided in “[profile.cargo]”?
I just realised today that Godbolt also uses sandbox internally.
After a quick look at the source code, I found the firejail configuration they use.
Maybe we can have a section like “[profile.cargo-fetch]” and override and extend configurations provided in “[profile.cargo]”?
I considered that briefly, but it feels like it'll get verbose and/or complicated quickly with the interactions of things like cargo
defaults (no network), overridden by fetch
and build
and run
(network), overridden by --offline
(no network) if we don't sit down and carefully plan it out.
That's why I want to research all the commands I can think of supporting first... so I can sit down and carefully plan properly.
(Remember, my goal is a sandboxing thing that Just Works™ if you stick nodo
onto the beginning of your usual commands, so it has to be expressive enough to handle cases like that.)
...also, maybe I should give it a -v
option that causes it to print something like nodo: profile=cargo +net -gui -dbus root=/path/to/detected/root
as the first line of output as part of a comprehensive plan to steer users away from just blindly dropping the nodo
if a build fails in the hope that will make it work.
Maybe we can have a section like “[profile.cargo-fetch]” and override and extend configurations provided in “[profile.cargo]”?
I considered that briefly, but it feels like it'll get verbose and/or complicated quickly with the interactions of things like
cargo
defaults (no network), overridden byfetch
andbuild
andrun
(network), overridden by--offline
(no network) if we don't sit down and carefully plan it out.That's why I want to research all the commands I can think of supporting first... so I can sit down and carefully plan properly.
(Remember, my goal is a sandboxing thing that Just Works™ if you stick
nodo
onto the beginning of your usual commands, so it has to be expressive enough to handle cases like that.)
One issue with that is that cargo can be extended…
Of course, but if you're extending Cargo, then it's perfectly reasonable for you to be expected to extend your sandbox definitions. That's one reason the syntax needs to be simple and easy to work with.
...still, I do intend to include ready-made definitions for some of the most common third-party subcommands.
...still, I do intend to include ready-made definitions for some of the most common third-party subcommands.
Yeah.
That's why I want to research all the commands I can think of supporting first... so I can sit down and carefully plan properly.
I tried to list alll commands supported by cargo's stable channel and split them into 4 categories:
Commands that requires network access and modifies $HOME/.cargo
:
install Install a Rust binary. Default location is $HOME/.cargo/bin
login Save an api token from the registry locally. If token is not specified, it will be read from stdin.
logout Remove an API token from the registry locally
search Search packages in crates.io
uninstall Remove a Rust binary
Commands that creates new project (need special handling since Cargo.toml
does not exist):
init Create a new cargo package in an existing directory
new Create a new cargo package at <path>
Commands that only needs access to the project:
verify-project Check correctness of crate manifest
version Show version information
check Check a local package and all of its dependencies for errors
clean Remove artifacts that cargo has generated in the past
clippy Checks a package to catch common mistakes and improve your Rust code.
d alias: doc
doc Build a package's documentation
fmt Formats all bin and lib files of the current crate using rustfmt.
locate-project Print a JSON representation of a Cargo.toml file's location
metadata Output the resolved dependencies of a package, the concrete used versions including overrides, in machine-readable format
pkgid Print a fully qualified package specification
read-manifest Print a JSON representation of a Cargo.toml manifest.
rustdoc Build a package's documentation, using specified custom flags.
Commands that requires network access, access to the project and $HOME/.cargo
:
b alias: build
bench Execute all benchmarks of a local package
build Compile a local package and all of its dependencies
c alias: check
fetch Fetch dependencies of a package from the network
fix Automatically fix lint warnings reported by rustc
generate-lockfile Generate the lockfile for a package
package Assemble the local package into a distributable tarball
owner Manage the owners of a crate on the registry
publish Upload a package to the registry
r alias: run
run Run a binary or example of the local package
rustc Compile a package, and pass extra options to the compiler
t alias: test
test Execute all unit and integration tests and build examples of a local package
tree Display a tree visualization of a dependency graph
yank Remove a pushed crate from the index
update Update dependencies as recorded in the local lock file
vendor Vendor all dependencies for a project locally
Since the commands above all build the project, they also might need to download the dependencies or build.rs
might access the network.
Also, some users might use sccache
(yeah, I use it) and it can be configured to distribute the compilation jobs and use cache stored in external machines.
Here are some cargo extensions that I know:
Commands that requires network access, access to the project and $HOME/.cargo
:
cargo-criterion
cargo-expand
cargo-flamegraph
cargo-tarpaulin
They are need to compile the project, so they would also need to download dependencies, which requires network.
Would you mind running through the commands that require network access to check how many follow the convention of having an --offline
flag which the sandbox should recognize as a signal to not allow network?
Would you mind running through the commands that require network access to check how many follow the convention of having an
--offline
flag which the sandbox should recognize as a signal to not allow network?
It seems that all the builtin commands of cargo
that requires network access has --offline
in their help page.
And then I checked cargo --help
and found that --offline
is one of the standard flags that every cargo
subcommand has.
For the cargo extensions I mentioned:
cargo-expand
cargo-flamegraph
does not have --offline
.
In that case, I suppose the question is whether we should offer a "Just Works, but less secure" mode of operation where any unrecognized subcommand gets network access (unless --offline
is seen) and whatever solution we come up with for $HOME/.cargo
.
(eg. I'm thinking that we'll want to force $HOME/.cargo/bin
to read-only access for every subcommand that has $HOME/.cargo
access except install
and uninstall
.)
eg. I'm thinking that we'll want to force $HOME/.cargo/bin to read-only access for every subcommand that has $HOME/.cargo access except install and uninstall.
I will add that most commands except for cargo publish
does not access $HOME/.cargo/credentials
.
And we should also make $HOME/.cargo/config.toml
read-only.
In that case, I suppose the question is whether we should offer a "Just Works, but less secure" mode of operation where any unrecognized subcommand gets network access (unless --offline is seen) and whatever solution we come up with for $HOME/.cargo.
I think we can implement this in v0.1 MVP, then improve on it in the subsequent revisions.
Yeah.
Also, for a later v0.1.x, it'd probably be a good idea to do a survey of Cargo subcommands to identify what inside the project root could be forced to read-only access for which commands.
(eg. I don't imagine most commands will execute anything like build.rs
that might need permission to modify the contents of src
, and we can whitelist the things allowed to modify Cargo.toml
.)
Yeah.
Also, for a later v0.1.x, it'd probably be a good idea to do a survey of Cargo subcommands to identify what inside the project root could be forced to read-only access for which commands.
(eg. I don't imagine most commands will execute anything like
build.rs
that might need permission to modify the contents ofsrc
, and we can whitelist the things allowed to modifyCargo.toml
.)
Totally agreed.
Modifying the project other than target
should be forbidden.
Honestly, most of the build.rs
don't modify the source.
Binding generator like bindgen
usually outputs the generated binding to target
, and the file will be included in a predefined module in src/
using something like include_file
(can't remember the name)include!(concat!(env!("OUT_DIR"), "/binding.rs"));
Though we also need to take into account that user can configure cargo
to use a custom "target" using --target-dir
.
Another thing I want to add to the configuration is whitelist_path
.
Some users like me who use sccache
would like the cargo build
to be able to access the cache.
That's why I'm not going for full whitelisting until at least 0.2.x. You wind up deep in the weeds reinventing parsing every command's arguments if you try for that.
Well down the curve of diminishing returns. For the most part, I'm just thinking about this stuff now because it will help to reduce how many different revisions the config schema will have to go through to keep up with demands.
As a "Just Works™ baseline sandboxing wrapper for any build automation", what features does this project need for a minimum viable product, and what features should be agreed on as definitely suitable for a later release?
Current ideas for the design:
So far I'm thinking:
-h
or--help
, display help.argv[0]
), unmodified, as a command inside Firejail.--help
for more details." error and exit.alias nargo=nodo cargo
so that, if the alias is missing, they'll get an error message rather than an un-sandboxed execution....and a configuration file design which provides this sort of functionality, but not necessarily with a schema anything like this:
Requirements for v0.1 MVP
nodo
(Like "sudo" but "no") is a placeholder I'm hoping to replace because there are too many Google results for it already.root_find_outermost
for workspaces support for v0.1 or defer it to v0.2cwd_to_root
to allow tools likemake
to handle being invoked from subdirectoriesblacklist
config key to allow blacklisting things like.git
so malicious packages can't hide their changes by amending un-pushed commitsroot_marked_by
for automatically determining where to place the sandboxing boundary if something likenodo cargo
ornodo just
gets run in a subdirectory.projectless_subcommands
to avoid forcing users to bypass the wrapper forcargo new
andcargo init
. (May need to redesign the actual config key for v0.2 to support things akin tonpm install -g
where it's not the subcommand that decides local vs. global)deny_subcommands
to support transparently allowingnpm install
but refusing with a "please bypass the wrapper" message fornpm install -g
.root_marked_by
file" situations. (Maybe stop and refuse if we hit$HOME
, the home directory as identified bygetpwnam
/getpwuid
, or a directory we lack write or execute/search permissions on the way up the tree before we find such a file.)v0.1.1
allow_network
andallow_network_subcommands
(Just allowing network for all commands is an acceptable v0.1 as long as it still constrains filesystem access and blocks off D-Bus and the like)root_resolve_symlinks=false
to allow using a differentially fuzzed port of Python'sos.path.abspath
instead ofstd::fs::canonicalize
. (Needed to ensure we can haveroot_marked_by
match the behaviour of all types of build automation if there's a symlink between$PWD
and the root directory it finds.)allow_dbus_subcommands
andallow_gui_subcommands
as a design for allowing things likecargo run
access to D-Bus and X11 and the like.v0.1.x
v2.0
cargo fetch
,cargo build
, etc.)