hackoregon / getting-started

Documents, links, software install scripts and other resources for getting started contributing to Hack Oregon
10 stars 6 forks source link

Priorities / votes for miscellaneous enhancements #9

Closed znmeb closed 7 years ago

znmeb commented 7 years ago

I've got a few enhancements I want to make - opening this issue to get an idea who else would use them.

  1. Running the OVA in Windows Hyper-V: I am probably going to do this for myself even if nobody else uses it since my Windows partition is set up with Hyper-V. If you have Windows 8, 8.1 or 10 Pro you probably have the Hyper-V capability. You need a 64-bit machine with the virtualization bits on the chip enabled in the BIOS. If you have those, look in Windows' "Add / Remove Windows Features" for Hyper-V.

    There are two pieces to it. There's a Hyper-V manager that's like the VirtualBox GUI and there's the services part. There are two downsides to Hyper-V. First, you can't have Hyper-V and VirtualBox both running. And it's two reboots to switch Hyper-V on or off.

    Second, you need two network connections for your guest to talk to the outside world. That is, the guest can't share the WiFi with the host. For our purposes a virtual switch inside the host that doesn't talk to the public internet is probably all we need.

    The guest can share a wired connection though, and might be able to share a USB wireless NIC. I have a wireless device that plugs into the LAN port on my laptop and when I run Hyper-V I put the host on the WiFi and the guest on the LAN and magic happens. ;-) I also have a USB wireless NIC I can test.

    My laptop has Hyper-V and there's no way I'm going back to VirtualBox. ;-)

  2. Running the OVA in Linux' Virtual Machine Manager / libvirt: again, this is probably something I'm going to do for myself since my workstation and VirtualBox don't play together at all well. Unfortunately, there's no tested / documented way to go the other way ... build a virtual machine in Virtual Machine Manager and export it to VirtualBox / OVA.

  3. Other Linux distros for native work: I've had one request for Ubuntu 16.10 "Yakkety Yak", and I know there's only one minor issue with RStudio that would affect that. And Fedora 25 is coming out in a week or two - I'll probably do that for myself because they've been very responsive to my bug reports over the years and because I like their OpenShift Platform-as-a-Service. The nice thing about Linux is that you can run what you like on the desktop and anything else in a virtual machine. ;-)

  4. Automating the Ubuntu install process in the OVA creation: I found the instructions for this here: http://searchitchannel.techtarget.com/feature/Performing-an-automated-Ubuntu-install-using-preseeding. It looks straightforward but very detailed and time-consuming.

    I doubt seriously if we're going to be using OVAs for anything beyond desktop data science and test environments. There are far better ways to deploy production servers; in fact, once we start to do that, there are fully automatable ways to grab deployed server images and run them in VirtualBox!

jimtyhurst commented 7 years ago

I vote for item (4), because I want to use a development environment running in VirtualBox on a Mac. I have the HackOregonBase.ova image running after last night's installation workshop, but I would like to know more about how it was created, so that I can make modifications or at least suggest modifications in the future. My guess is that different teams will use different, but overlapping, sets of tools. One approach is to create an image that has all possible tools that any team would want. That requires a process by which people can request additions to the one canonical image. Another approach is to have a few customized images derived from a common base. I think your work or somebody's work on (4) would enable that second approach.

znmeb commented 7 years ago

One alternative would be to replace Ubuntu server with OSGeo Live. (https://live.osgeo.org/en/download.html). That already has a VMDK virtual disk prebuilt which all the virtualizers - Hyper-V, VirtualBox and Virtual Machine Manager can all run it, and at least VirtualBox and Virtual Machine Manager can be scripted to construct the virtual machines automatically.

It also includes complete, though somewhat ratty, Linux desktop. The only things it doesn't have are Miniconda and RStudio, but that's an easy install - the scripts I have now will work on it!

I'm philosophically leery of kitchen sink distros but for Hack Oregon OSGeo Live makes a lot of sense to me. When I started building the virtual machines I looked at using it as the base but I was frustrated by the ratty desktop, especially the terminal app "lxterm". And it's not very hard to start with a desktop you like and add software, so I went with Linux Mint instead.

OSGeo Live is very well supported so that's not a concern of mine. I'll spin up a virtual machine from the VMDK on Hyper-V and Virtual Machine Manager today and verify that my install scripts work and that it can be "vagrantized" with the VirtualBox provider. I really don't want Hack Oregon to be maintaining a detailed chunk of code that's a pre-seed / kickstarter script unless there's no better way.

znmeb commented 7 years ago

Well ... I spoke too soon on OSGeo Live ... Hyper-V doesn't directly use VMDK format virtual disks, and there's no supported tool for importing an OVA into Hyper-V. There is, however, a Windows port of the open source qemu-img utility to convert between the various virtual disk formats. And it can be scripted.

I'm not sure which takes longer ... converting the VMDK to the native Windows virtual disk format or building a new virtual machine by booting the ISO. I'm going with the ISO for now until I can automate the conversion

znmeb commented 7 years ago

If you're trying this at home, you have to create a Generation 1 virtual machine for the ISO to work. Generation 2 will not boot the ISO with any of the security options enabled / disabled. I'm going down the VMDK -> VHDX path next. Stay tuned.

znmeb commented 7 years ago

Latest notes:

  1. I did a few preliminary tests with Fedora 25 beta. The default Vagrant environment there is a libvirt / kvm virtualizer, so I will probably do a Vagrant libvirt build. I'm pretty sure this will work on all modern Linux desktops that are supported by the vendor / community, so there's little need for a port of the scripts ... worst case is we'd need a 32-bit Vagrant guest on a 32-bit VirtualBox host.

    A Vagrant / libvirt build is also more automatable than the VirtualBox one I have now. There are base Ubuntu server images and hooks for all the installs already ... it's just a matter of coding "glue logic".

  2. I'm still looking at Hyper-V - it's somewhat harder to find supported strategies since it only exists on Windows 8 / 8.1 / 10 Pro and not on other Windows desktops. I'd rather make the native Windows install as painless as possible. ;-)
znmeb commented 7 years ago

Another potential enhancement - PhpPgAdmin inside the virtual machines. This is a web-based GUI for managing PostgreSQL databases in a server. I don't personally use it - I use PgAdmin3 and will be migrating to PgAdmin4, which can also serve. My inclination is to go with PgAdmin4 since it's Python-based.

znmeb commented 7 years ago

OK ... I'm going to close this. Open an issue if there's something you want and I'll round it up.