Include OpenVAS appliance in 15.x release (feature request)

[Dude4Linux]

@JedMeister If possible, please include the OpenVAS appliance, https://github.com/Dude4Linux/openvas, in the 14.2 release. An ISO image is available at https://sourceforge.net/projects/turnkeylinuxcom/files/turnkey-openvas/turnkey-openvas-14.2-jessie-amd64.iso/download If any other steps are needed to review or vet new appliance candidates, please let me know.

[JedMeister]

@Dude4Linux - I'm just building this for v14.2 release now! :smile:

Good work with this one, it looks like an awesome appliance. Thanks for sorting out the artwork too (lots of people forget that!)

FWIW I've also created a TKLBAM profile for it: https://github.com/turnkeylinux/tklbam-profiles/blob/master/openvas I'm pretty sure that should cover everything that is required.

[JedMeister]

Hi @Dude4Linux - we're going to have to push OpenVAS to v15.0. :cry: It's just pulling in too much of Kali. I think that we'll have to install from source.

[Dude4Linux]

@JedMeister I'm disappointed, but I understand. I used the Kali repo's because that's what I had been testing with and there are no OpenVAS packages for Jessie. Fortunately, OpenVAS has been added to Stretch so that will be an easy change. It didn't occur to me until just now, that I could use the same trick to pull OpenVAS from Stretch instead of Kali, but I doubt that would change the decision. Speaking of the 15.0 release, can I encourage you to prioritize the v15.0 version of TKLdev? Preferably it should be released along with the v15.0 core so that those of us who don't have access to the build system, can begin work on updating our pet appliances. The more people you can get to contribute to the 15.0 release, the better. Where would you like us to post thoughts or suggestions re: v15.0? On the tracker? Whiteboard?

[JedMeister]

@Dude4Linux - Just to clarify, it worked really well for me. So there was no immediate issue (hence why I built it to ISO ready to release). And I'm sure it would have continued to work ok for quite a while. But because it pulled in so much of Kali, it wouldn't have taken long before the system would drift so far from Debian stable that it would be more Kali than Debian. As Kali is a rolling release, this would almost certainly start to cause issues within the lifetime of the appliance.

FWIW, I run a rolling install of Debian Sid on my desktop and it works really well. But there are a few things that you need to be wary of when running a rolling release. Firstly you need to upgrade every few weeks (I recall reading a recommendation for every 1-3 weeks; a month or 2 at the longest). And secondly, ideally you need to remember to not run apt-get update, unless you are planning to do an apt-get dist-upgrade straight after. Generally it should work ok, but some edge cases can cause it to break things. Also ideally, the upgrade should be done within init 1 (i.e. single user mode) so as little stuff as possible is running.

It probably wouldn't immediately cause issues and even when it did, for people like you and I, we'd be able to work it out. But most TurnKey users don't know much about Linux, let alone some of the intricacies. So I think releasing this appliance as is for v14.2 could potentially end up being a support blackhole and a poor user experience.

It's great OpenVAS is included in Stretch, although I do note that it's quite an old version (v6 vs current release v9). I don't know much about OpenVAS, how quick their development cycle is or how much difference there is between versions, but if that is good enough, we'll be good to go.

Otherwise, Sid currently has v9 but it's blocked for migration to buster because of package policy violation. If the maintainer resolves that by the time we're getting close to release (and it's in buster/testing), we can easily get it backported (to stretch-backports) for our purposes.

Pulling OpenVAS v6 from Stretch probably would have been better as I would imagine that it would have needed to pull fewer dependencies from Stretch (Kali is based on Sid). But it still wouldn't have been ideal. I think that a v15.0 OpenVAS appliance will be tons better than a v14.2 app would have ever been.

---

Re v15.0 TKLDev, it's not a bad suggestion. And actually I was thinking of at least releasing a few other appliances as v15.0rc1 (e.g. at least LAMP and perhaps a couple of the more popular appliances such as WordPress; i.e. not just Core as we have done previously).

Alon's rationale for only providing Core in the past, is that all other appliances are based on Core, so if Core works, then the base OS is good. However, during v14.2 I noticed that a number of bugs didn't appear until we had produced enough different appliances that general users started using them and reporting issues.

As for TKLDev though, we don't actually need TKLDev v15.0 to produce v15.0 appliances. Just as we did all the v14.2 development on v14.1, you can do v15.0 development on v14.x. All we need is an updated bootstrap (for Stretch), and we'll need to provide Stretch packages for our custom software, then we should be good to go. Once that's happened, you could even build your own TKLDev v15.0! :smile:

As soon as we have the Stretch transition working, I'm hoping to document how to set that up yourself for those that are eager to get going with v15.0 development. Obviously we'll still block official release, but there would be nothing stopping you or others from updating appliances and getting them ready.

---

Regarding thoughts/suggestions for v15.0, let's just do it here on the tracker. I suggest that you open new issues for each thought/suggestion. As soon as I see it, I'll tag it appropriately.

[JedMeister]

Hey @Dude4Linux - Any thoughts on OpenVAS for v15.0?

I haven't had a look at it recently, so can't provide much insight. If it's pretty much "ready to go" then I'm more than happy to have another look at it and consider including it if we can.

It seems that no one has backported it from Sid/Buster so we'd either need to just stick with the old (v6.x) version in Stretch, or possibly pin it to Buster repo (which I'm not that keen to do). Installing from source (or better still, perhaps compiling the Debian Buster source package if the decencies aren't too dramatic?) is another option?

FWIW, I found a blog post which suggests that it still works ok with the old version in Stretch. Although I note that that post itself is from last year. OpenVAS themselves note that even the newer v8.x is nearly EOL.

[Dude4Linux]

Hi @JedMeister - I haven't had much luck with the OpenVAS appliance. Something blew up the Kali installation by consuming all available disk space on the VM. I haven't been able to determine the source of the problem. I've been using Lynis as a security scanner instead of OpenVAS and I highly recommend you use it as a tool to 'vet' the 15.0 appliances. I've already run it against core and incorporated some of the recommended changes for hardening to common. However that doesn't catch mis-configured apps like apache, nginx, etc. I believe you said the first batch of 15.0 appliances were ready for release, but last I checked they weren't on the turnkey mirrors. If you would allow me early access, I could try writing an Ansible playbook to install and run Lynis on each appliance and send feedback about any detected problems. Better yet would be to include the Lynis scanning as part of your workflow for releasing the appliances. Lynis is designed to be standalone in that you can create a custom tar file, copy the file to the appliance, untar and run it, then grab the report file and look for warnings. The customization is basically telling Lynis which tests to skip.

EDIT: I do see some of the 15.0 apps on the mirror server now.

[JedMeister]

Hi @Dude4Linux - Thanks for the tip re Lynis

I've already run it against core and incorporated some of the recommended changes for hardening to common.

I assume you mean the ones we've already added?

However that doesn't catch mis-configured apps like apache, nginx, etc.

Yeah ok, good point.

I believe you said the first batch of 15.0 appliances were ready for release, but last I checked they weren't on the turnkey mirrors.

Yep, I announced stage 1 about a week ago - 47 apps; albeit ISOs only. The other builds are done and should be on the mirror as of earlier this week, but I haven't announced them yet. Probably do that early next week (I'm trying to get AMI up and going so as not to burn my lead too much).

[then I saw your EDIT note... :smile: ]

But all is not lost as I hope to do much more rapid releases now that we're on Stretch. We'll see how that works out in practice, but if we find any glaring holes or issues, then we can certainly do individual v15.1 releases of specific appliances (as I'm pretty sure I noted elsewhere previously).

I could try writing an Ansible playbook to install and run Lynis on each appliance and send feedback about any detected problems.

That would be awesome if you could. Please let me know how you go with that.

Better yet would be to include the Lynis scanning as part of your workflow for releasing the appliances.

Yes that would be ideal. I'll keep it in mind.

[Dude4Linux]

@JedMeister

I assume you mean the ones we've already added?

Yes, those that were added to common.

Yep, I announced stage 1 about a week ago - 47 apps; albeit ISOs only.

Aah! I just did a quick check and noticed the ISOs were on the mirrors, and just assumed that the proxmox containers were also available. Since I've been doing all of my Ansible work with containers, I'd prefer having the proxmox images for testing. There is now a ProxMox module for Ansible, so I could try doing the testing in a VM instead. I'm away from home ATM, but I'll take a look in about a week.

[JedMeister]

@Dude4Linux

Yes, those that were added to common.

Great, well they're already in then, so that's a good start.

Aah! I just did a quick check and noticed the ISOs were on the mirrors, and just assumed that the proxmox containers were also available. Since I've been doing all of my Ansible work with containers, I'd prefer having the proxmox images for testing. There is now a ProxMox module for Ansible, so I could try doing the testing in a VM instead. I'm away from home ATM, but I'll take a look in about a week.

FWIW, we always build the ISOs first. We usually wait 24hr+ after ISOs are published to ensure that they're propagated throughout the mirror network. Then we process the ISOs with buildtasks to create all the other builds. Sometimes altogether (i.e. bt-optimized) sometimes separate builds depending on circumstance. As they're all built from the ISO, it means that they are consistent (not withstanding the changes for each separate build). It also means that once the ISO is built, we don't need to chase our tails because something changes/breaks/etc.

Syncing all the other builds to the mirror network was really slow this time. I'm not 100% sure why that was. Although, the images have grown a bit for the Stretch based release, plus the VM builds are quite a bit bigger because we've included the kernel headers and VMtools DKMS package (makes it better for users when the kernel gets updated). So I'm guessing all that added up...

[JedMeister]

In the hope of adding this to the v15.0 release, I updated it to install from source in this branch. However, I never did get it all running properly, so it's still incomplete... Figured that I may as well note it here though.

[OnGle]

Once again we've failed to add all the new appliances, let's try again on 17.0!