[Ubuntu 18.04] SecGen fails to build project. (Solution in comments.)

I'm using VirtualBox 5.2 and manually installed the latest version of Vagrant (since the version installed as per the SecGen instructions was out of date). When running ruby secgen.rb run everything seems to work up to the point of building the project. Then I get this error:

Building project: /home/torhackr/SecGen/projects/SecGen20180418_1621
Loading vagrant (up ) in /home/torhackr/SecGen/projects/SecGen20180418_1621
Bringing machine 'escalation' up with 'virtualbox' provider...
==> escalation: Box 'modules_bases_debian_puppet_32' could not be found. Attempting to find and install...
    escalation: Box Provider: virtualbox
    escalation: Box Version: >= 0
==> escalation: Box file was not detected as metadata. Adding it directly...
==> escalation: Adding box 'modules_bases_debian_puppet_32' (v0) for provider: virtualbox
    escalation: Downloading: https://app.vagrantup.com/secgen/boxes/debian_wheezy_puppet/versions/1.0.0/providers/virtualbox.box
    escalation: Progress: 0Error provisioning VMs, destroying VMs and exiting SecGen.
Loading vagrant (destroy -f) in /home/torhackr/SecGen/projects/SecGen20180418_1621
==> escalation: VM not created. Moving on...

It appears that perhaps Vagrant failed to download the box 'modules_bases_debian_puppet_32' for the VirtualBox provider. I can download this file manually, but where do I put it? How do I set it up so the build process will work?

UPDATE: I manually downloaded the file in the link (https://app.vagrantup.com/secgen/boxes/debian_wheezy_puppet/versions/1.0.0/providers/virtualbox.box), renamed it to virtualbox.box, then (from the SecGen root directory) ran the command vagrant box add ./virtualbox.box --name 'modules_bases_debian_puppet_32'. After doing this, I tried ruby secgen.rb run. It seems to have made some progress, but I get the following error:

Building project: /home/torhackr/SecGen/projects/SecGen20180418_1648
Loading vagrant (up ) in /home/torhackr/SecGen/projects/SecGen20180418_1648
Bringing machine 'escalation' up with 'virtualbox' provider...
==> escalation: Importing base box 'modules_bases_debian_puppet_32'...
Progress: 10Error provisioning VMs, destroying VMs and exiting SecGen.
Loading vagrant (destroy -f) in /home/torhackr/SecGen/projects/SecGen20180418_1648
==> escalation: VM not created. Moving on...

Tried running ruby secgen.rb build-vms --project projects/SecGen20180418_1702 and got the following result:

torhackr@DevBox:~/SecGen$ ruby secgen.rb build-vms --project projects/SecGen20180418_1702
WARNING: Nokogiri was built against LibXML version 2.9.7, but has dynamically loaded 2.9.4
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
SecGen - Creates virtualised security scenarios
            Licensed GPLv3 2014-18
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Building project: projects/SecGen20180418_1702
Loading vagrant (up ) in projects/SecGen20180418_1702
Bringing machine 'escalation' up with 'virtualbox' provider...
==> escalation: Importing base box 'modules_bases_debian_puppet_32'...
Progress: 10Error provisioning VMs, destroying VMs and exiting SecGen.
Loading vagrant (destroy -f) in projects/SecGen20180418_1702
Traceback (most recent call last):
    3: from secgen.rb:458:in `<main>'
    2: from secgen.rb:200:in `build_vms'
    1: from /home/torhackr/SecGen/lib/helpers/gem_exec.rb:39:in `exe'
/home/torhackr/SecGen/lib/helpers/gem_exec.rb:39:in `chdir': No such file or directory @ dir_s_chdir - projects/SecGen20180418_1702 (Errno::ENOENT)

It seems the problem is with importing the box. When I've run vagrant up modules_bases_debian_puppet_32 it has successfully brought up the 'default' box without any issues. It seems there might be a problem in the configuration. I'm not familiar enough yet to detect what's wrong.

I noticed the 'No such file or directory' error in the last execution, so I tried using an absolute path.

torhackr@DevBox:~/SecGen$ ruby secgen.rb build-vms --project ~/SecGen/projects/SecGen20180418_1702/
WARNING: Nokogiri was built against LibXML version 2.9.7, but has dynamically loaded 2.9.4
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
SecGen - Creates virtualised security scenarios
            Licensed GPLv3 2014-18
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Building project: /home/torhackr/SecGen/projects/SecGen20180418_1702/
Loading vagrant (up ) in /home/torhackr/SecGen/projects/SecGen20180418_1702/
Bringing machine 'escalation' up with 'virtualbox' provider...
==> escalation: Importing base box 'modules_bases_debian_puppet_32'...
Progress: 10Error provisioning VMs, destroying VMs and exiting SecGen.
Loading vagrant (destroy -f) in /home/torhackr/SecGen/projects/SecGen20180418_1702/
==> escalation: VM not created. Moving on...

So this time, it didn't give the 'No such file or directory' error, but it still failed to provision.

This time I changed directories into the project directory and tried vagrant up. Here's the result:

torhackr@DevBox:~/SecGen/projects/SecGen20180418_1702$ vagrant up
Bringing machine 'escalation' up with 'virtualbox' provider...
==> escalation: Importing base box 'modules_bases_debian_puppet_32'...
==> escalation: Matching MAC address for NAT networking...
==> escalation: Setting the name of the VM: SecGen20180418_1702_escalation_1524090674152_12594
==> escalation: Clearing any previously set network interfaces...
==> escalation: Found default DHCP server from initial VirtualBox install. Cleaning it up...
==> escalation: Preparing network interfaces based on configuration...
    escalation: Adapter 1: nat
    escalation: Adapter 2: hostonly
==> escalation: Forwarding ports...
    escalation: 22 (guest) => 2222 (host) (adapter 1)
==> escalation: Running 'pre-boot' VM customizations...
==> escalation: Booting VM...
==> escalation: Waiting for machine to boot. This may take a few minutes...
    escalation: SSH address: 127.0.0.1:2222
    escalation: SSH username: vagrant
    escalation: SSH auth method: private key
    escalation: Warning: Remote connection disconnect. Retrying...
    escalation: Warning: Connection reset. Retrying...
    escalation: 
    escalation: Vagrant insecure key detected. Vagrant will automatically replace
    escalation: this with a newly generated keypair for better security.
    escalation: 
    escalation: Inserting generated public key within guest...
    escalation: Removing insecure key from the guest if it's present...
    escalation: Key inserted! Disconnecting and reconnecting using new SSH key...
==> escalation: Machine booted and ready!
==> escalation: Checking for guest additions in VM...
    escalation: The guest additions on this VM do not match the installed version of
    escalation: VirtualBox! In most cases this is fine, but in rare cases it can
    escalation: prevent things such as shared folders from working properly. If you see
    escalation: shared folder errors, please make sure the guest additions within the
    escalation: virtual machine match the version of VirtualBox you have installed on
    escalation: your host and reload your VM.
    escalation: 
    escalation: Guest Additions Version: 5.0.16
    escalation: VirtualBox Version: 5.2
==> escalation: Rsyncing folder: /home/torhackr/SecGen/projects/SecGen20180418_1702/environments/ => /tmp/vagrant-puppet/environments
==> escalation: Rsyncing folder: /home/torhackr/SecGen/projects/SecGen20180418_1702/puppet/escalation/modules/distcc_exec/ => /tmp/vagrant-puppet/manifests-d93c80d59c4865b51489c7519267279c
==> escalation: Rsyncing folder: /home/torhackr/SecGen/projects/SecGen20180418_1702/puppet/escalation/modules/ => /tmp/vagrant-puppet/modules-a7508fc313b465bc17662a3958086331
==> escalation: Rsyncing folder: /home/torhackr/SecGen/projects/SecGen20180418_1702/puppet/escalation/modules/chkrootkit/ => /tmp/vagrant-puppet/manifests-b95a280f20d6e84cb75fd4cfb572459e
==> escalation: Rsyncing folder: /home/torhackr/SecGen/projects/SecGen20180418_1702/puppet/escalation/modules/unix_update/ => /tmp/vagrant-puppet/manifests-8d5990089b7146bd7f3c837ba0ca351f
==> escalation: Rsyncing folder: /home/torhackr/SecGen/projects/SecGen20180418_1702/puppet/escalation/modules/popa3d/ => /tmp/vagrant-puppet/manifests-e6f4a1b40ff3ded38d3acba3866d9062
==> escalation: Mounting shared folders...
    escalation: /vagrant => /home/torhackr/SecGen/projects/SecGen20180418_1702
==> escalation: Running provisioner: puppet...
==> escalation: Running Puppet with environment production...
==> escalation: Notice: Compiled catalog for localhost in environment production in 0.19 seconds
==> escalation: Notice: /Stage[main]/Distcc_exec::Install/Package[distcc]/ensure: created
==> escalation: Notice: /Stage[main]/Distcc_exec::Config/File[/etc/default/distcc]/content: content changed '{md5}c2b24cc9ae328c4c07c6334fff7219a7' to '{md5}f5f2adbf51357b89baec03f9dcec0e2a'
==> escalation: Notice: /Stage[main]/Distcc_exec::Config/File[/etc/default/distcc]/mode: mode changed '0644' to '0777'
==> escalation: Notice: /Stage[main]/Distcc_exec::Config/File[/home/distccd/]/ensure: created
==> escalation: Notice: /Stage[main]/Distcc_exec::Config/Exec[change-home-dir]/returns: executed successfully
==> escalation: Notice: /Stage[main]/Distcc_exec::Service/Service[distcc]/ensure: ensure changed 'stopped' to 'running'
==> escalation: Notice: /Stage[main]/Distcc_exec::Config/Secgen_functions::Leak_files[distcc_exec-file-leak]/Secgen_functions::Leak_file[distcc_exec-dolorum.docx-0]/Exec[-/home/distccd/dolorum.docx]/returns: executed successfully
==> escalation: Notice: /Stage[main]/Distcc_exec::Config/Secgen_functions::Leak_files[distcc_exec-file-leak]/Secgen_functions::Leak_file[distcc_exec-dolorum.docx-0]/File[/home/distccd/dolorum.docx]/ensure: defined content as '{md5}76777fd1c59c295ddf7677cc2a467cab'
==> escalation: Notice: /Stage[main]/Distcc_exec::Config/Secgen_functions::Leak_files[distcc_exec-file-leak]/Secgen_functions::Leak_file[distcc_exec-ex.ods-1]/Exec[-/home/distccd/ex.ods]/returns: executed successfully
==> escalation: Notice: /Stage[main]/Distcc_exec::Config/Secgen_functions::Leak_files[distcc_exec-file-leak]/Secgen_functions::Leak_file[distcc_exec-ex.ods-1]/File[/home/distccd/ex.ods]/ensure: defined content as '{md5}40c56b92ee62610ef286e616741e8643'
==> escalation: Notice: Applied catalog in 9.52 seconds
==> escalation: Running provisioner: puppet...
==> escalation: Running Puppet with environment production...
==> escalation: Notice: Compiled catalog for localhost in environment production in 0.22 seconds
==> escalation: Notice: /Stage[main]/Chkrootkit::Install/File[/usr/local/chkrootkit-0.49.tar.gz]/ensure: defined content as '{md5}304d840d52840689e0ab0af56d6d3a18'
==> escalation: Notice: /Stage[main]/Chkrootkit::Install/Exec[unpack-chkrootkit]/returns: executed successfully
==> escalation: Notice: /Stage[main]/Chkrootkit::Install/Exec[make-chkrootkit]/returns: executed successfully
==> escalation: Notice: /Stage[main]/Chkrootkit::Install/File[/usr/sbin/chkrootkit]/ensure: created
==> escalation: Notice: /Stage[main]/Chkrootkit::Install/Exec[remove-chkrootkit-0.49.tar.gz]/returns: executed successfully
==> escalation: Notice: /Stage[main]/Chkrootkit::Configure/Cron[chkrootkit]/ensure: created
==> escalation: Notice: /Stage[main]/Chkrootkit::Install/Secgen_functions::Leak_files[chkrootkit-file-leak]/Secgen_functions::Leak_file[chkrootkit_vuln-qui.odp-0]/Exec[-/root/qui.odp]/returns: executed successfully
==> escalation: Notice: /Stage[main]/Chkrootkit::Install/Secgen_functions::Leak_files[chkrootkit-file-leak]/Secgen_functions::Leak_file[chkrootkit_vuln-qui.odp-0]/File[/root/qui.odp]/ensure: defined content as '{md5}c8c46d01825edb3710ce8cd99765abaf'
==> escalation: Notice: Applied catalog in 1.91 seconds
==> escalation: Running provisioner: puppet...
==> escalation: Running Puppet with environment production...
==> escalation: Notice: Compiled catalog for localhost in environment production in 0.08 seconds
==> escalation: Notice: /Stage[main]/Unix_update::Unix/Exec[update]/returns: executed successfully
==> escalation: Notice: Applied catalog in 6.31 seconds
==> escalation: Running provisioner: puppet...
==> escalation: Running Puppet with environment production...
==> escalation: Notice: Compiled catalog for localhost in environment production in 0.05 seconds
==> escalation: Notice: /Stage[main]/Popa3d::Install/Package[popa3d]/ensure: created
==> escalation: Notice: /Stage[main]/Popa3d::Config/Service[popa3d]/ensure: ensure changed 'stopped' to 'running'
==> escalation: Notice: Applied catalog in 15.29 seconds
torhackr@DevBox:~/SecGen/projects/SecGen20180418_1702$ vagrant status
Current machine states:

escalation                running (virtualbox)

The VM is running. To stop this VM, you can run `vagrant halt` to
shut it down forcefully, or you can run `vagrant suspend` to simply
suspend the virtual machine. In either case, to restart it again,
simply run `vagrant up`.
torhackr@DevBox:~/SecGen/projects/SecGen20180418_1702$

It seems to have worked without any problems when I do the manual build by hand.

~~Now my only question is... How do I package this up for distribution?~~ Derp. I opened VirtualBox, and there it was, just waiting for an export to ova. :smile:

Here are the steps I took to solve the problem:

Download the box from the link provided (https://app.vagrantup.com/secgen/boxes/debian_wheezy_puppet/versions/1.0.0/providers/virtualbox.box)
Save it in the SecGen root directory.
Run vagrant box add ./virtualbox.box --name 'modules_bases_debian_puppet_32'
Run ruby secgen.rb build-project.
Once the project is built, cd ./projects/[new secgen project].
Run vagrant up.
When it's up, run vagrant halt.
Open VirtualBox and select the appropriate VM. Tweak settings if necessary (there was a warning about video memory) and export the appliance.
Voila!

I have not yet tested the exported VM to see if it the output is what was expected, but everything seems to work from here.

NOTE: After exporting and attempting to re-import the appliance, I had an error regarding the virtual disk being missing. However, this is unrelated to this particular bug, so I'm going to leave it alone.

I'm going to leave this one open as others may find the manual basebox installation steps you have kindly provided useful.

Thanks again!

one more problem i faced above is when running the command ruby secgen.rb build-project even after saving the virtualbox.box in SecGen root directory sometimes gives previous errors. in my case i used rm -rf ./projects/* and now ruby secgen.rb build-project worked fine.

there were around 5 projects saved in ./projects/ directory from previous tries, and they were causing problems somehow.

cliffe / SecGen

[Ubuntu 18.04] SecGen fails to build project. (Solution in comments.) #114