AzureSandbox

Contents

• Architecture
• Overview
• Sandbox index
• Prerequisites
• Getting started
• Next steps
• Videos
• Known issues

Architecture

Overview

This repository contains a collection of inter-dependent cloud computing configurations for implementing common Microsoft Azure services on a single subscription. Collectively these configurations provide a flexible and cost effective sandbox environment useful for experimenting with various Azure services and capabilities. Depending upon your Azure offer type and region, a fully provisioned #AzureSandbox environment costs approximately $50 USD / day. These costs can be further reduced by stopping / deallocating virtual machines when not in use, or by skipping optional configurations that you do not plan to use (Step-By-Step Video).

Disclaimer: #AzureSandbox is not intended for production use. While some best practices are used, others are intentionally not used in favor of simplicity and cost. See Known issues for more information.

#AzureSandbox is implemented using popular open source tools that are supported on Windows, macOS and Linux including:

• git for source control.
• Bash for scripting.
• Azure CLI is a command line interface for Azure.
• PowerShell
  • PowerShell 7.x
  • PowerShell 5.1 for Windows Server configuration.
• Terraform v1.9.8 for Infrastructure as Code (IaC).
  • Azure Provider (azuerrm) v4.9.0
  • AzAPI Provider (azapi) v2.0.1
  • cloud-init Provider (cloudinit) v2.3.5
  • Random Provider (random) v3.6.3
  • Time Provider (time) v0.12.1

This repo was created by Roger Doherty.

Sandbox index

#AzureSandbox features a modular design and can be deployed as a whole or incrementally depending upon your requirements.

• terraform-azurerm-vnet-shared includes the following:
  • A resource group which contains all the sandbox resources.
  • A key vault for managing secrets.
  • A log analytics workspace for log data and metrics.
  • A storage account for blob storage.
  • An automation account for configuration management.
  • A virtual network for hosting virtual machines.
  • A bastion for secure RDP and SSH access to virtual machines.
  • A Windows Server virtual machine running Active Directory Domain Services with a pre-configured domain and DNS server.
• terraform-azurerm-vnet-app includes the following:
  • A virtual network for hosting virtual machines and private endpoints implemented using PrivateLink. Virtual network peering with terraform-azurerm-vnet-shared is automatically configured.
  • A Windows Server virtual machine for use as a jumpbox.
  • A Linux virtual machine for use as a DevOps agent.
  • A PaaS SMB file share hosted in Azure Files with a private endpoint implemented using PrivateLink.
• terraform-azurerm-vm-mssql includes the following:
  • An IaaS database server virtual machine based on the SQL Server virtual machines in Azure offering.
• terraform-azurerm-msssql includes the following:
  • A PaaS database hosted in Azure SQL Database with a private endpoint implemented using PrivateLink.
• terraform-azurerm-mysql includes the following:
  • A PaaS database hosted in Azure Database for MySQL - Flexible Server with a private endpoint implemented using PrivateLink.
• terraform-azurerm-vwan includes the following:
  • A virtual wan.
  • A virtual wan hub with pre-configured hub virtual network connections with terraform-azurerm-vnet-shared and terraform-azurerm-vnet-app. The hub is also pre-configured for User VPN (point-to-site) connections.
• extras contains additional Terraform configurations and supporting resources.

Prerequisites

The following prerequisites are required in order to get started.

• Identify the Microsoft Entra ID tenant to be used for identity and access management, or create a new tenant using Quickstart: Set up a tenant.

• Identify a single Azure subscription or create a new Azure subscription. See Azure Offer Details and Associate or add an Azure subscription to your Microsoft Entra tenant for more information.

• Identify the owner of the Azure subscription to be used for #AzureSandbox. This user should have an Owner Azure RBAC role assignment on the subscription. See Steps to assign an Azure role for more information.

• Ask the subscription owner to create an Owner Azure RBAC role assignment for each sandbox user. See Steps to assign an Azure role for more information.

• Verify the subscription owner has privileges to create a Service principal name on the Microsoft Entra tenant. See Permissions required for registering an app for more information.

• Ask the subscription owner to Create an Azure service principal with Azure CLI (SPN) for sandbox users by running the following Azure CLI command in Azure Cloud Shell.

  # Replace 00000000-0000-0000-0000-000000000000 with the subscription id
  az ad sp create-for-rbac -n AzureSandboxSPN --role Owner --scopes /subscriptions/00000000-0000-0000-0000-000000000000

  Securely share the output with sandbox users, including appId and password:

  {
  "appId": "00000000-0000-0000-0000-000000000000",
  "displayName": "AzureSandboxSPN",
  "password": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
  "tenant": "00000000-0000-0000-0000-000000000000"
  }

• Some organizations may institute Azure policy which may cause some sandbox deployments to fail. This can be addressed by using custom settings which pass the policy checks, or by disabling the policies on the Azure subscription being used for the configurations.

• Some Azure subscriptions may have low quota limits for specific Azure resources which may cause sandbox deployments to fail. See Resolve errors for resource quotas for more information. Consult the following table to determine if quota increases are required to deploy the configurations using default settings:

Note: This list is not comprehensive. Quotas vary by Azure subscription offer type and environment. More than one quota may need to be increased for a single resource type, such as public ip addresses.

Getting started

Before you begin, familiarity with the following topics will be helpful when working with #AzureSandbox:

• Familiarize yourself with Terraform Input Variables
• Familiarize yourself with Terraform Output Values also referred to as Output Variables
• See Authenticating to Azure using a Service Principal and a Client Secret to understand the type of authentication used by Terraform in #AzureSandbox
• Familiarize yourself with Recommended naming and tagging conventions
• Familiarize yourself with Naming rules and restrictions for Azure resources

Configure client environment

#AzureSandbox automation scripts are written in Linux Bash and Linux PowerShell. In order to deploy #AzureSandbox you will need to configure a Linux client environment to execute these scripts. Detailed guidance is provided for users who are unfamiliar with Linux. Three different client environment options are described in this section, including:

• Windows Subsystem for Linux (preferred for completing smoke testing)
• Azure Cloud Shell (zero configuration required but not optimal for serious use)
• Linux / MacOS

Windows Subsystem for Linux

Windows users can use WSL which supports a variety of Linux distributions. Ubuntu 22.04 LTS (Jammy Jellyfish) is recommended. Please note these instructions may vary for different Linux releases and/or distributions.

• Windows prerequisites (Step-By-Step Video)

  • Install Visual Studio Code on Windows
  • Optional Windows software
  • Install SQL Server Management Studio with Azure Data Studio if you plan to complete smoke testing for either terraform-azurerm-vm-mssql or terraform-azurerm-mssql.
  • Install MySQL Workbench if you plan to complete smoke testing for terraform-azurerm-mysql
  • Install Azure VPN Client if you plan to complete smoke testing for terraform-azurerm-vwan.

• Linux prerequisites (Step-By-Step Video)

  • Install Linux on Windows with WSL. Ubuntu 24.04 LTS (Noble Numbat) is recommended.
  • Install pip3 Python library package manager and the PyJWT Python library. This is used to determine the id of the security principal for the currently signed in Azure CLI user.

  # Install the most recent PyJWT Python library
  sudo apt update
  sudo apt install python3-pip
  pip3 show pyjwt

  • Install the Azure CLI on Linux | apt (Ubuntu, Debian)
  • Install Terraform | Linux | Ubuntu/Debian. Note: it is not necessary to complete the Quick start tutorial.
  • Install PowerShell on Ubuntu

  • Once PowerShell is installed follow these steps to configure it.

    # Download and execute PowerShell configuration script
    wget https://raw.githubusercontent.com/Azure-Samples/azuresandbox/main/configure-powershell.ps1
    chmod 755 configure-powershell.ps1
    sudo ./configure-powershell.ps1

• Configure VS Code for Remote development in WSL (Step-By-Step Video)

  • Launch VS Code
  • Install WSL VS Code Extension.
  • Install the HashiCorp Terraform VS Code Extension in WSL.
  • Install the PowerShell VS Code extension in WSL.

Azure Cloud Shell

Azure Cloud Shell is a free pre-configured cloud hosted container with a full complement of tools needed to use #AzureSandbox. This option will be preferred for users who do not wish to install any software and don't mind a web based command line user experience. Review the following content to get started:

• Bash in Azure Cloud Shell
• Persist files in Azure Cloud Shell
• Using the Azure Cloud Shell editor

Warning: Cloud shell containers are ephemeral. Anything not saved in ~/clouddrive will not be retained when your cloud shell session ends. Also, cloud shell sessions expire. This can interrupt a long running process.

Linux / macOS

Linux and macOS users can deploy the configurations natively by installing the following tools:

• Azure CLI
  • Debian or Ubuntu: Install Azure CLI with apt
  • RHEL, Fedora or CentOS: Install Azure CLI with yum
  • openSUSE or SLES: Install Azure CLI with zypper
  • Install Azure CLI on macOS
  • Install Azure CLI on Linux manually
• Install Terraform
  • Refer to the Linux tab then choose the corresponding tab for your distro if installing on Linux.
  • Refer to the Homebrew on OS X if installing on macOS.
  • Note: Skip the Quick start tutorial.
• PowerShell
  • Installing PowerShell on Linux
  • Installing PowerShell on macOS
  • After installing, run configure-powershell.ps1
• VS Code
  • Linux
  • macOS
  • After installing, add the following extensions:
  • Terraform
• Miscellaneous packages
  • pip3 Python library package manager.
  • PyJWT Python library. This is used to determine the id of the security principal for the currently signed in Azure CLI user.

Note the Bash scripts used in the configurations were developed and tested using GNU bash, version 5.0.17(1)-release (x86_64-pc-linux-gnu) and have not been tested on other popular shells like zsh.

Next steps

Now that the client environment has been configured, here's how to clone a copy of this repo and start working with the latest release of code (Step-By-Step Video).

# Run this command on cloudshell clients only
cd clouddrive

# Run these commands on all clients, including cloudshell 
git clone https://github.com/Azure-Samples/azuresandbox
cd azuresandbox
latestTag=$(git describe --tags $(git rev-list --tags --max-count=1))
git checkout $latestTag

Perform default sandbox deployment

For the first deployment, the author recommends using defaults, which is ideal for speed, learning and testing. IP address ranges are expressed using CIDR notation.

Default IP address ranges

The configurations use default IP address ranges for networking components. These ranges are artificially large and contiguous for simplicity, and customized IP address ranges can be much smaller. A suggested minimum is provided to assist in making the conversion. It's a good idea to start small. Additional IP address ranges can be added to the networking configuration in the future if you need them, but you can't modify an existing IP address range to make it smaller.

Default subnet IP address prefixes

This section documents the default subnet IP address prefixes used in the configurations. Subnets enable you to segment the virtual network into one or more sub-networks and allocate a portion of the virtual network's address space to each subnet. You can then connect network resources to a specific subnet, and control ingress and egress using network security groups.

Apply sandbox configurations

Apply the configurations in the following order:

1. terraform-azurerm-vnet-shared implements a virtual network with shared services used by all the configurations.
2. terraform-azurerm-vnet-app implements an application virtual network with pre-configured Windows Server and Linux jumpboxes.
3. terraform-azurerm-vm-mssql (optional) implements an IaaS database server virtual machine based on the SQL Server virtual machines in Azure offering.
4. terraform-azurerm-mssql (optional) implements a PaaS database hosted in Azure SQL Database with a private endpoint implemented using PrivateLink.
5. terraform-azurerm-mysql (optional) implements a PaaS database hosted in Azure Database for MySQL - Flexible Server with a private endpoint implemented using using PrivateLink.
6. terraform-azurerm-vwan (optional) connects the shared services virtual network and the application virtual network to remote users or a private network.

Destroy sandbox configurations

While a default sandbox deployment is fine for testing, it may not work with an organization's private network. The default deployment should be destroyed first before doing a custom deployment. This is accomplished by running terraform destroy on each configuration in the reverse order in which it was deployed:

1. terraform-azurerm-vwan
2. terraform-azurerm-mysql
3. terraform-azurerm-mssql
4. terraform-azurerm-vm-mssql
5. terraform-azurerm-vnet-app
6. terraform-azurerm-vnet-shared. Note: Resources provisioned by bootstrap.sh must be deleted manually.

Alternatively, for speed, simply delete rg-sandbox-01`. You can run cleanterraformtemp.sh to clean up temporary files and directories.

# Warning: This command will delete an entire resource group and should be used with great caution.
az group delete -g rg-sandbox-01

Perform custom sandbox deployment

A custom deployment will likely be required to connect the configurations to an organization's private network. This section provides guidance on how to customize the configurations.

Document private network IP address ranges (sample)

Use this section to document one or more private network IP address ranges by consulting a network professional. This is required if you want to establish a hybrid connection between an organization's private network and the configurations. The sandbox includes two IP address ranges used in a private network. The CIDR to IPv4 Conversion tool may be useful for completing this section.

A blank table is provided here for convenience. Make a copy of this table and change the TBD values to your custom values.

Customize IP address ranges (sandbox)

Use this section to customize the default IP address ranges used by the configurations to support routing on an organization's private network. The aggregate range should be determined by consulting a network professional, and will likely be allocated using a range that falls within the private network IP address ranges discussed previously, and the rest of the IP address ranges must be contained within it. The CIDR to IPv4 Conversion tool may be useful for completing this section. Note this sandbox uses the suggested minimum address ranges from the default IP address ranges described previously.

A blank table is provided here for convenience. Make a copy of this table and change the TBD values to your custom values.

Customize subnet IP address prefixes (sandbox)

Use this section to customize the default subnet IP address prefixes used by the configurations to support routing on an organization's private network. Make a copy of this table and change these sandbox values to custom values. Each address prefix must fall within the virtual network IP address ranges discussed previously. The CIDR to IPv4 Conversion tool may be useful for completing this section.

It is recommended to reserve space for future subnets. A blank table is provided here for convenience. Make a copy of this table and change the TBD values to your custom values.

Videos

Known issues

This section documents known issues with these configurations in addition to GitHub Issues.

• Client environment
  • If you are experiencing difficulties with WSL, see Troubleshooting Windows Subsystem for Linux.
  • Some users may not be able to use Windows subsystem for Linux due to lack of administrative access to their computer or other issues. In these cases consider using terraform-azurerm-rg-devops and VS Code remote development over SSH as an alternative.
  • There is a known issue installing PowerShell on Ubuntu 24.04 (Noble) using APT. See [DSR] - Ubuntu 24.04 x64 #21385 for information and workarounds.
• Configuration management
  • Terraform
  • For simplicity, these configurations store State in a local file named terraform.tfstate. For production use, state should be managed in a secure, encrypted Backend such as azurerm.
  • There is a known issue that causes Terraform plan or apply operations to fail after provisioning an Azure Files share behind a private endpoint. If this is causing plan or apply operations to fail you can either whitelist the IP address of the client environment on the storage account firewall or use Target Resources to work around it.
  • Azure Verified Modules were not available when this project was created, and may be used in future releases of #AzureSandbox.
  • Windows Server: This configuration uses Azure Automation State Configuration (DSC) for configuring the Windows Server virtual machines, which will be replaced by Azure Automanage Machine Configuration. This configuration will be updated to the new implementation in a future release.
  • configure-automation.ps1: The performance of this script could be improved by using multi-threading to run Azure Automation operations in parallel.
• Identity, Access Management and Authentication.
  • Authentication: These configurations use a service principal to authenticate with Azure which requires a client secret to be shared. Production environments should consider using managed identities instead of service principals which eliminates the need to share secrets.
  • Point-to-site VPN gateway authentication: This configuration uses self-signed certificates for simplicity. Production environments should use certificates generated from a root certificate authority.
  • Credentials: For simplicity, these configurations use a single set of user defined credentials when an administrator account is required to provision or configure resources. In production environments these credentials would be different and follow the principal of least privilege for better security. Some user defined credentials may cause failures due to differences in how various resources implement restricted administrator user names and password complexity requirements. Note that the default password expiration policy for Active Directory is 42 days which will require the password for bootstrapadmin@mysandbox.local to be changed. It is recommended that you update the related adminpassword secret in key vault when changing the password as this does not happen automatically.
  • Active Directory Domain Services: A pre-configured AD domain controller azurerm_windows_virtual_machine.vm_adds is provisioned.
  • High availability: The current design uses a single VM for AD DS which is counter to best practices as described in Deploy AD DS in an Azure virtual network which recommends a pair of VMs in an Availability Set.
  • Data integrity: The current design hosts the AD DS domain forest data on the OS Drive which is counter to best practices as described in Deploy AD DS in an Azure virtual network which recommends hosting them on a separate data dr*ive with different cache settings.
  • Role-Based Access Control (RBAC)
  • Least privilege: The current design uses a single Azure RBAC role assignment to grant the Contributor role to the currently logged in Azure CLI user and the service principal used by Terraform. Owner privileges are only required for a few prerequisites. Since the Contributor role cannot change role assignments, all dependencies on Azure RBAC roles for data plane access were intentionally avoided. As a result, key vault access policies and storage account keys are used for access control to those resources instead of Azure RBAC role assignments. Production environments should consider leveraging best practices as described in Azure role-based access control (Azure RBAC) best practices which recommends using multiple role assignments to grant the least privilege required to perform a task. Additionally, production environments should also consider leveraging Azure RBAC roles for data plane access to key vault and storage accounts.
  • ARM provider registration: As described in issue #4440, some controlled environments may not permit automatic registration of ARM resource providers by Terraform. In these cases some ARM providers may need to be registered manually. See Azure resource providers and types and the azurerm provider skip_provider_registration optional argument for more information.
• Storage
  • Azure Storage: For simplicity, this configuration uses the Authorize with Shared Key approach for Authorizing access to data in Azure Storage. For production environments, consider using shared access signatures instead.
  • There is a known issue when attempting to apply Terraform plans against Azure Storage containers that sit behind a firewall such as a private endpoint. This may prevent the ability to apply changes to configurations that contain this type of dependency, such as terraform-azurerm-vnet-app. To work around this you use Resource Targeting to avoid issues with storage containers.
  • Standard SSD vs. Premium SSD: By default, this configuration uses Standard SSD for SQL Server data and log disks instead of Premium SSD for reduced cost. Production deployments should use Premium SSD as per best practices.
• Networking
  • azurerm_subnet.vnet_shared_01_subnets["snet-adds-01"]: This subnet is protected by an NSG as per best practices described in described in Deploy AD DS in an Azure virtual network, however the network security rules permit ingress and egress from the Virtual Network on all ports to allow for flexibility in the configurations. Production implementations of this subnet should follow the guidance in How to configure a firewall for Active Directory domains and trusts.
  • azurerm_private_dns_zone_virtual_network_link.private_dns_zone_virtual_network_links_vnet_app_01[] and azurerm_private_dns_zone_virtual_network_link.private_dns_zone_virtual_network_links_vnet_shared_01[]: Ideally private dns zones should only need to be linked to the shared services virtual network, however some provisioning processes (e.g. Azure Database for MySQL), require them to be linked to the same virtual network where the service is being provisioned. For this reason all private DNS zones are linked to all virtual networks.
  • azurerm_point_to_site_vpn_gateway.point_to_site_vpn_gateway_01: Connection attempts using the Azure VPN client may fail with the message Server did not respond correctly to VPN control packets. Session state: Reset sent. Synchronizing the time on the VPN client should resolve the issue. For Windows 11 clients go to Settings > Time & Language > Date & Time > Additional settings > Sync now.