search

vmware-archive / vctl-docs

VMware vctl Docs
MIT License
125 stars 15 forks source link

`vctl build` can't build large projects. #22

Open david9991 opened 4 years ago

david9991 commented 4 years ago

I cannot build large project, it always says no space left on device, even set --builder-mem 8192, and I tried to enlarge it by --builder-mem 10240, then it gave me an error:

ERROR error trying to run container vctl-builder-1499b092: failed when running container vctl-builder-1499b092: failed to start the container, please review containerd logs: error code:exit status 255, message:2020-06-12T09:16:27.150| ServiceImpl_Opener: PID 50548
Error: The operation was canceled
david9991 commented 4 years ago

Seems vctl build will put the whole volume of vctl-builder on a RAM disk?

Screen Shot 2020-06-12 at 11 00 17 AM
david9991 commented 4 years ago

And I can see an vctl-builder persistent storage, but seems builder doesn't use it.

david9991 commented 4 years ago

And more, if builder unexpected exit while building, the vm will not stop properly, and if run vctl system stop error message 

Stop virtual machine vctl-builder-b7cce09d ...
ERROR failed to run command:/Applications/VMware Fusion.app/Contents/Public/vmrun -T fusion stop /Users/david/.vctl/.r/vms/vctl-builder-b7cce09d/vctl-builder-b7cce09d.vmx hard, err:exit status 255  module=libcrx package=vmrun
ERROR failed to stop a sandbox, err:error code:exit status 255, message:Error: The virtual machine is not powered on: /Users/david/.vctl/.r/vms/vctl-builder-b7cce09d/vctl-builder-b7cce09d.vmx  module=libcrx package=sandbox
Virtual machine vctl-builder-b7cce09d is stopped
Stop virtual machine vctl-builder-b93a0c4d ...
ERROR failed to run command:/Applications/VMware Fusion.app/Contents/Public/vmrun -T fusion stop /Users/david/.vctl/.r/vms/vctl-builder-b93a0c4d/vctl-builder-b93a0c4d.vmx hard, err:exit status 255  module=libcrx package=vmrun
ERROR failed to stop a sandbox, err:error code:exit status 255, message:Error: The virtual machine is not powered on: /Users/david/.vctl/.r/vms/vctl-builder-b93a0c4d/vctl-builder-b93a0c4d.vmx  module=libcrx package=sandbox
Virtual machine vctl-builder-b93a0c4d is stopped

will be given.

mikeroySoft commented 4 years ago

IIRC, the builder is put into memory for performance reasons.

How large is your project?

david9991 commented 4 years ago

@mikeroySoft Actually I am writing a compiler with LLVM, LLVM is big you know.

wswfc commented 4 years ago

@david9991 Is the base image public so we can have a try ourselves? Or is there similar public image we can access?

vctl does have some config to use the on-disk storage instead of ram, but that may introduce even worse performance impact. So before we try that approach, I'd like to get a hand on the base image and see if there is anything we can take in the ram approach.

david9991 commented 4 years ago

@wswfc Sorry I can't give you the base Image, but https://hub.docker.com/r/nwtgck/llvm-clang/tags maybe you can try. It has 9.17G compressed size.

Samnan commented 3 years ago

EDIT: For cleaning up the aborted build as described in https://github.com/VMwareFusion/vctl-docs/issues/22#issuecomment-643042938

vctl system stop

then delete the following folders manually /Users/\/.vctl/.r/containers/\ /Users/\/.vctl/.r/sandboxes/\

stefanhenseler commented 3 years ago

@wswfc I understand that vctl uses a RAM disk for performance reasons and that we can increase this value by specifying --builder-mem which defaults to "4g". My build also requires more space (10g). Is there a way to set the default using the ~/.vctl/config.toml file? Is there a reference for the config? I couldn't find anything in the docs.