CLI: Assess and address startup time

[jsolbrig]

Requested Update

Description

Currently, starup time for the CLI is about 0.5 seconds and is about 0.28 seconds for import geoips. Both are too long. The problem is entirely confined to imports where imports take 0.447 seconds for the CLI and 0.27 seconds for import geoips. We should investigate several things:

- [ ] Determine whether we are profiling correctly.
- [ ] Why does the CLI take twice as long to load as `import geoips`?
- [ ] Which imports are slowest?

Once slow points are identified, we should determine whether we can address them or if "it is what it is" and we should just let it go. If we decide to let it go, we should document why in this issue prior to closing.

Background and Motivation

The CLI loads slowly.

Useful tools for profiling

The script below can be used to generate a pstats file using cProfile and generate dot-graphs from that pstats file. Another nice tool is snakeviz which gives a more dynamic view of the pstats results.

#!/usr/bin/env bash

script_dir="$( cd -- "$(dirname "$0")" >/dev/null 2>&1 ; pwd -P )"
formats=("png" "svg")

echo "Profiling GeoIPS CLI"

python -m cProfile -o ${script_dir}/geoips_cli.pstats $GEOIPS_PACKAGES_DIR/geoips/geoips/commandline/commandline_interface.py

for format in ${formats[@]}; do
    echo "Producing ${format} charts"

    gprof2dot -f pstats -n 0 -e 0 \
        -p $GEOIPS_PACKAGES_DIR/geoips \
        ${script_dir}/geoips_cli.pstats \
        | dot -T${format} -v -o ${script_dir}/geoips_cli_000.${format}
    echo "Chart written to ${script_dir}/geoips_cli_000.${format}"

    gprof2dot -f pstats -n 0.005 -e 0.001 \
        -p $GEOIPS_PACKAGES_DIR/geoips \
        ${script_dir}/geoips_cli.pstats \
        | dot -T${format} -v -o ${script_dir}/geoips_cli_005.${format}
    echo "Chart written to ${script_dir}/geoips_cli_005.${format}"

    gprof2dot -f pstats -n 0.05 -e 0.01 \
        -p $GEOIPS_PACKAGES_DIR/geoips \
        ${script_dir}/geoips_cli.pstats \
        | dot -T${format} -v -o ${script_dir}/geoips_cli_050.${format}
    echo "Chart written to ${script_dir}/geoips_cli_050.${format}"

    gprof2dot -f pstats -n 0.05 -e 0.01 \
        ${script_dir}/geoips_cli.pstats \
        | dot -T${format} -v -o ${script_dir}/geoips_cli_050_no_p.${format}
    echo "Chart written to ${script_dir}/geoips_cli_050_no_p.${format}"

    gprof2dot -f pstats -n 0.5 -e 0.1 \
        -p $GEOIPS_PACKAGES_DIR/geoips \
        ${script_dir}/geoips_cli.pstats \
        | dot -T${format} -v -o ${script_dir}/geoips_cli_500.${format}
    echo "Chart written to ${script_dir}/geoips_cli_500.${format}"
done

Checklist for Completion

• [ ] Use profiling to identify which functions or imports are slow during startup.
• [ ] Document the sources of slowness on this issue.
• [ ] Determine whether any of the sources of slowness can be mitigated and create issues for mitigation.