aurae-runtime / aurae

Distributed systems runtime daemon written in Rust.
https://aurae.io
Apache License 2.0
1.84k stars 89 forks source link

Introduce devcontainer as option for dev environment #431

Closed future-highway closed 1 year ago

future-highway commented 1 year ago

Trying to get a dev environment working using devcontainer extension for VS Code.

So far, I am able to build (though it is much slower than using a VM), and some tests are passing.

When running tests that need to allocate a cgroup, I get a failure that is similar to:

---- cells::cell_service::cells::cells::tests::test_allocate stdout ----
thread 'cells::cell_service::cells::cells::tests::test_allocate' panicked at 'allocate: AbortedAllocateCell { cell_name: CellName("ae-test-7da5253e-416c-485e-951b-f9026b4f6afc"), source: AddTaskToCgroup { cell_name: CellName("ae-test-7da5253e-416c-485e-951b-f9026b4f6afc"), source: failed to write +io to "/sys/fs/cgroup/cgroup.subtree_control"

Caused by:
    Not supported (os error 95) } }', auraed/src/cells/cell_service/cells/cells.rs:300:57

However, io is listed in cgroup.controllers:

root ➜ /workspaces/aurae (devcontainer) $ cat /sys/fs/cgroup/cgroup.controllers 
cpuset cpu io memory hugetlb pids rdma

And the contents of cgroup.subtree_control:

root ➜ /workspaces/aurae (devcontainer) $ cat /sys/fs/cgroup/cgroup.subtree_control 
cpuset cpu
krisnova commented 1 year ago

Hey @future-highway really glad you are adding this to the project. Couple of thoughts:

  1. Do we want to maintain a unique dockerfile for the development sandbox? Is this notably different than the docker files we have today?

  2. I am fairly confident we won't be able to run privilege containers in GitHub codespaces. Note: Codespaces works just bad reading from the .devcontainer directory. We might want to add some docs here?

future-highway commented 1 year ago

Let me start by saying that this is very much a WIP that I hope will work, but I'm don't know if it will. I opened the PR in part to see if anyone would have insight on getting passed the current error, especially as I knew I wouldn't have time to work on it today.

It is possible that we can specify enough of specs in the devcontainer file instead of the dockerfile and maybe eventually eliminate it, if that is preferred.

There are less blunt options beside privileged, that may be able to be tried. By default the file system in the container was read only, so making cgroups was erroring. I was more aiming for a local dev environment though, mainly as an alternative to recommending the use of Parallel, on apple silicon, since that is a paid product that some may not want to pay for. I think it should still work on different architecture, but I don't have a device to try it on. I'm sure someone can check once things are working.

Re docs, yes, a readme with docs will be included.

future-highway commented 1 year ago

This PR is ready; I'm just waiting to rebase on #438

future-highway commented 1 year ago

Caught a couple of typos.

Also, the changed index.md files are autogenerated from make docs, so those shouldn't have any changes from what is already deployed.

StephenWeber commented 1 year ago

"works on my machine" - I was able to follow the README, and linked guides, to set this up and build and run auraed and things seem to work well.

One thing I couldn't figure out from the guides, that we straightened out in Discord, was that to "run" this devcontainer the command I needed was "Reload in devcontainer" (since it was freshly added to the project) -- sounds like once this is merged that detail won't be needed / won't often be needed.

I have no input on the other questions, such as if this could be accomplished with existing dockerfiles or the like. But for my purposes the current PR would address my needs on aarch64

future-highway commented 1 year ago

While I didn't get too much time to work on this PR this week, I'm definitely feeling stuck here.

future-highway commented 1 year ago

Since this PR was started, we have had an additional alternate dev environment guide added. Given that and the fact that the tests aren't passing in the devcontainer, I'm leaning towards closing this PR.

future-highway commented 1 year ago

Closing this PR. If you are looking for dev environment options, see https://aurae.io/development-environments/