Local and Secure Deployment of Broadsea (and related Deployments)

[haydenbspence]

Purpose

This is a general issue to house the discussion of deploying Broadsea in local, secure, and similar environments, whether that is by adjusting Docker or modeling separate technologies after Broadsea's Docker implementation to deploy the suite.

Problem Statement

Docker is often not able to be deployed by default in some environments. IT departments may have default environment settings that do not allow for Docker. In some settings, this may involve a lengthy process of paperwork and approvals to change, which increases the barriers to using Broadsea. In other settings, the deployment may not be feasible at all due to policy.

Types of Problems

Security: The permissions provided to a user may not allow for some of the aspects of Docker or the technologies in docker-compose. However, it would be possible using different tooling that the same functionality could be possible under default user permissions.

Firewalls: The ports necessary may not be able to be opened, even locally, depending on the security settings. This may impact the Broadsea database deployment and WebAPI in particular.

Virtualization Performance/Security: The secure environment is hosted on a server, which is frequently a virtualization itself. For both security and performance reasons IT Admins may not approve of virtualization on top of virtualization.

Detailed Analysis

The following are considerations that IT offices often include in their analysis. This may be helpful when evaluating Docker alternatives. However, organization policies also factor into permissibility.

• Known Common Vulnerabilities and Exposures (CVEs) associated with this technology. You can search for those using the National Vulnerability Database
• Cybersecurity Alerts & Advisories from CISA
• Nested virtualization scenarios are not guaranteed to work
• There are more Federal IT policy publicly available through the Government Service Administration

Initial Ideas

• Ansible to orchestrate local installations
• Integrating an option to use DuckDB over Postgres as the Broadsea database

Forum Post

[leeevans]

Here are some additional ideas you could investigate which may enable some secure sites to run the Broadsea container images:

Podman - doesn’t run as root, and no Docker service running on server: https://podman.io/

Kubernetes - may be a good option if a site is already using it for other applications

AWS Elastic Container Service (ECS) or other cloud container service equivalent

This AWS Whitepaper - Architecting for HIPAA Security and Compliance on AWS may be a useful reference for secure cloud services approach: https://docs.aws.amazon.com/pdfs/whitepapers/latest/architecting-hipaa-security-and-compliance-on-aws/architecting-hipaa-security-and-compliance-on-aws.pdf#architecting-hipaa-security-and-compliance-on-aws

Note. A local site install of PostgreSQL or cloud database service like AWS RDS is recommended for Broadsea production use instead of the demo broadsea-atladb container image.

There is also an OHDSI-in-a-box virtual server image that doesn’t use Docker here (Windows or Linux OS): https://github.com/OHDSI/OHDSI-in-a-Box

[haydenbspence]

Containerd

Containerd is a container runtime that uses Open Container Initiative (OCI) standards and can be used with both Linux and Windows daemons.

• Linux/OSX: runc
• Windows: hcsshim

Dependencies:

• Go >=1.19

Pros

• Many options for windows requires WSL2 to be enabled, this can be an additional hurdle for approval.

Cons

• Hyper-V for windows may be prohibited, and it's unclear if this impacts hcsshim. Initial reading implies that Hyper-V was influential to the development of the daemon, but not necessarily if it is a dependency.

Vagrant

Vagrant is a tool for building and managing virtual machine environments. This can be paired with a hypervisor of choice.

Dependencies:

• A hypervisor or cloud provider (VirtualBox, Docker, VMWare, Hyper-V have built-in support, others can be found through provider plugins)

Pros

• This allows for IaC similar to docker-compose, but allows users to select the hypervisor/provider, which makes the process easier if IT has one provider but not another approved.

Cons

• Vagrant can be a bit more complicated and doesn't have a GUI
• Vagrant is arguably slower running locally compared to Docker

Nix and NixOS

Nix is a tool for package management and creating reproducible, declarative and reliable systems.

Dependencies:

• Linux or WSL.
• NixOS requires a hypervisor or bare metal.

Pros

• IT may be more comfortable setting up an OS and securing it rather than allowing daemons in established systems or hypervisors on top of hypervisors. This would be a use case for NixOS
• If a Linux box or WSL is permissible, Nix should work well without interfering with the underlying system.

Cons

• No support for Windows (must use WSL or Type2 hypervisor with linux distro)

[alondhe]

Ideally, we could just leverage the same compose setup. Do any of these work with docker compose files? I see podman can (mostly).

[haydenbspence]

nerdctl is a drop-in replacement for docker-compose and should run the same yml format without any reformatting.

I am currently testing it and hcsshim referencing a write-up by James Sturtevant, and another by Markus Lippert.

What's promising here is Hyper-V is not necessarily needed if the same version of windows is used as the container image. If this is possible for Linux/OSX as well it may help with any situations in which virtualization permissions is the main issue.

[alondhe]

I think once we select a Docker alternative (that is compose compliant), let's test it in GitHub Actions here to gain evidence that the approach can work.

[haydenbspence]

@alondhe Broadsea on Podman is working locally for me with a OCI translated image. I should be able to submit a pull later tonight/tomorrow as a proof of concept w/ readme.

I also suggest looking into CRI-O and Windows Containers as two other runtimes to support. If Docker, Podman, CRI-O and Windows Containers OR virtualization in general are not options -- I think only a local install would be feasible.

This is out-of-scope and non-container related but there have also been some efforts to run portable Shiny apps locally (1, 1.1, 2) using Electron and R-Portable that may fit some use-cases.

[leeevans]

This is great @haydenbspence - thanks for working on this!

[haydenbspence]

My testing with Podman really only worked when removing the following:

• ohdsi-webapi-from-git
• ohdsi-atlas-from-git
• broadsea-ares

I also don't see a non-system specific way to fix the 80 / 443 issue with non-admin/root users.

OHDSI/Broadsea/tree/podman seems to be in-line with this.

[haydenbspence]

Official release notes for Podman 1.2 adds support for compose

https://podman-desktop.io/blog/podman-desktop-release-1.2

[alondhe]

Is it fair to say Podman is the most supported alternative? I'd want to stick to platforms that have long term viability if possible.

[haydenbspence]

Yes, Podman looks like a solid alternative for deployments in unique security situations that Docker doesn't cover. Here's why:

• Podman officially supports OCI Image specs and docker-compose.
• It's in RHEL by default, so we can count on it sticking around.
• It can run without a daemon or root privileges, offering more flexibility and security.

If a tech department isn't behind Docker or Podman, other choices are probably in 'shadow IT' territory, like nerdctl.

Podman does have a hitch - it needs WSL for Windows Servers and can't work with Windows Containers like Docker can. This is due to how it uses Namespaces, and might never change.

So far, I haven't found alternatives that support both Windows Containers and daemonless/rootless deployment. Though this might change (as mentioned in this article), it's too early to adopt. We're better off waiting for more support from Microsoft.

Check out Ajit's branch of Podman.

Closing the issue, reopen if Podman does not fulfill your site's needs or some magic is discovered we didn't find.