backup-ago

Backup ArcGIS Online content to a specified location.

Purpose

Download GIS and other content from ArcGIS Online Organizations to create copies for backup and historical purposes. ArcGIS Online does not support archiving and history tracking like SDE databases so this process will provide that functionality.

Requirements

• ArcGIS Online Organizational Account

• Python 3.7+ (version 3.7.10 used at time of writing) with libraries:
  -- ArcGIS API for Python
  -- dotenv

• JupyterLab (installed via Conda) -- Used for development, will potentially be changed to run in base Python -- Note: JupyterLab has a lot of requirements and makes this script heavier than it needs to be

Project Setup

Setup Creation of Backups on ArcGIS Online

Prep data on ArcGIS Online or Enterprise

• Tag layers with special tag for backups, i.e. "HFLBackupIncludeYes"
• Create folder of layers to be backed up and move layers to this folder; otherwise simply query for the tag in later steps rather than this folder
• Create a hosted notebook on AGO or Enterprise
• Copy the code from the provided template in \CreateBackupsHostedNotebook folder of this project
• Edit the query parameters for your user, folder holding layer to backup, and destination folder for backups
• Test and run the notebook, ensuring backups are created as desired
• Schedule the notebook to run as needed offset from the download process in the next section

Local Process for Downloading Backups Install Requirements

Clone (download) this project.

Create a Python environment for the project (strongly recommended).

• You can install the libraries into your base Python environment; however, it is probably better to create a separate environment for projects
• Example using Conda shown below.
• You will use this specified environment in your task scheduler.
• Launch conda command prompt then conda create --name envname which creates a new environment by default in C:\Users\username\Anaconda3\envs

Activate the new environment

• conda activate envname docs

Install ArcGIS API for Python in the new Environment

• Example, using Anaconda: -- conda install -c esri arcgis
  -- Can alternatively use Miniconda.
• version 1.9.0 used at time of writing

Install dotenv

• conda install -c conda-forge python-dotenv
• version 0.19.0 used at time of writing

Install JupyterLab (Optional)

• conda install -c conda-forge jupyterlab
• (Optional, it is included with Anaconda full install)

Configuration

This project uses a hidden environment file named ".env" in the root directory. This file is ignored (by .gitignore method) so you must create this file in the root directory and add variable features.

Create .env file

• This file will hold credentials and other environment variables in the script.
• Create a file named .env in the root directory using the template. You can rename the template to .env.
• Plug in environment varaibles for credentials

Set Environment varaibles

• credentials
• save location
• search parameters to backup

Run

Can be run as a Python file i.e. scheduled with Windows Task Scheduler or interactively in a Jupyter Lab notebooks.

Development was done in Jupyter Lab then exported to Python.

• Run Jupyter Lab, open notebook, run all cells.
  -- conda activate env-name then cd path-to-project then jupyter lab
  -- run the notebook. OR
• Run Python .py version of the file on demand or via Task Scheduler
• Be sure to use the Python environment with the installed ArcGIS and dotenv libraries (i.e. using the Ana/Miniconda distribution here).