Open ionutnechita opened 2 years ago
@ionutnechita Should work, but i dont think you wont profit much on it. Since the scheduler is mostly thought to provide a low latency. But you ca try it.
I tried today, but the docker containers don't start.
Maybe tomorrow I'll give you more details.
@ionutnechita
Do you have autogroup disabled ?
I use this options: CONFIG_TT_SCHED=y CONFIG_TT_ACCOUNTING_STATS=y
And AUTOGROUP is disabled.
Patch set: tt-5.15.patch - 0.3.7 high-hz.patch
@ionutnechita
Sorry, i was wrong. It does not work, since autogroup is disabled at default.
hi @ionutnechita
could you please upload the .config
docker sometimes works without autogroup and sched_group
I'm sending you 2 .config, one without TT and one with TT.
Without TT: config-5.15.9-lowlatency-non-tt.txt
With TT: config-5.15.9-lowlatency-with-tt.txt
My scenario:
./kind create cluster --config test.yml -v 1 Without TT, working correctly: Creating cluster "kind" ... DEBUG: docker/images.go:58] Image: kindest/node:v1.21.1@sha256:69860bda5563ac81e3c0057d654b5253219618a22ec3a346306239bba8cfa1a6 present locally â Ensuring node image (kindest/node:v1.21.1) đŧ â Preparing nodes đĻ đĻ đĻ đĻ đĻ đĻ â Configuring the external load balancer âī¸ â Writing configuration đ â Starting control-plane đšī¸ â Installing CNI đ â Installing StorageClass đž â Joining more control-plane nodes đŽ â Joining worker nodes đ Set kubectl context to "kind-kind" You can now use your cluster with:
kubectl cluster-info --context kind-kind
Have a nice day! đ
./kind create cluster --config test.yml -v 1 With TT, not working correctly: Creating cluster "kind" ... DEBUG: docker/images.go:58] Image: kindest/node:v1.21.1@sha256:69860bda5563ac81e3c0057d654b5253219618a22ec3a346306239bba8cfa1a6 present locally â Ensuring node image (kindest/node:v1.21.1) đŧ â Preparing nodes đĻ đĻ đĻ đĻ đĻ đĻ â Configuring the external load balancer âī¸ â Writing configuration đ ERROR: failed to create cluster: failed to generate kubeadm config content: failed to get kubernetes version from node: failed to get file: command "docker exec --privileged kind-worker3 cat /kind/version" failed with error: exit status 1 Command Output: Error response from daemon: Container b411eec646c160287d2c770eefa0fffc3498d30ecb9debdd838b4966c309094f is not running Stack Trace: sigs.k8s.io/kind/pkg/errors.WithStack sigs.k8s.io/kind/pkg/errors/errors.go:59 sigs.k8s.io/kind/pkg/exec.(LocalCmd).Run sigs.k8s.io/kind/pkg/exec/local.go:124 sigs.k8s.io/kind/pkg/cluster/internal/providers/docker.(nodeCmd).Run sigs.k8s.io/kind/pkg/cluster/internal/providers/docker/node.go:146 sigs.k8s.io/kind/pkg/exec.OutputLines sigs.k8s.io/kind/pkg/exec/helpers.go:81 sigs.k8s.io/kind/pkg/cluster/nodeutils.KubeVersion sigs.k8s.io/kind/pkg/cluster/nodeutils/util.go:35 sigs.k8s.io/kind/pkg/cluster/internal/create/actions/config.getKubeadmConfig sigs.k8s.io/kind/pkg/cluster/internal/create/actions/config/config.go:208 sigs.k8s.io/kind/pkg/cluster/internal/create/actions/config.(*Action).Execute.func1.1 sigs.k8s.io/kind/pkg/cluster/internal/create/actions/config/config.go:90 sigs.k8s.io/kind/pkg/errors.UntilErrorConcurrent.func1 sigs.k8s.io/kind/pkg/errors/concurrent.go:30 runtime.goexit runtime/asm_amd64.s:1371
A good day. Ionut.
@ionutnechita
There was also some time ago a cfs release, maybe that is a try worth. Also you could use hamad's old scheduler, the cacule scheduler.
I also run that on my buildserver, the RDB version at it runs great.
Here you find the patches for 5.15 and 5.16.
Hi @ionutnechita
Could you please try with
UCLAMP_TASK
enabled and also with HZ=300 or 1000
Hi @hamadmarri
I try with UCLAMP_TASK enabled and also with HZ=300. But not working.
config-5.15.9-lowlatency-with-tt-uclamp.txt
And i notice this in journalctl: journalctl -u docker ... level=warning msg="Unable to find cpu cgroup in mounts"
I will test the next few days. Thank you for help.
Happy New Year.
@ionutnechita
You might also try the BORE scheduler https://github.com/firelzrd/bore-scheduler
Hi @hamadmarri
I tried with BORE scheduler and working fine. I think in the future I will use BORE or Cacule scheduler.
I don't think TT design supports that full docker support. But this TT scheduler is very powerful.
Thanks for support.
A good day.
Hi Hamad Al Marri,
Is it possible to have the TT in kernel and the docker server working properly? Is there a problem with CGROUPs?
// level=warning msg="Unable to find cpu cgroup in mounts"
A good day. Ionut.