The existing structure of the sentry infra tools relies on 2 packages. sentry-kube and libsentrykube. In an ideal scenario we should have only a single package so that the release process when a change in libsentrykube happens involves
Fix in libsentrykube and merge.
Make a release of sentry-kube package with a new verision.
Update the version used in ops.
With 2 packages, a fix in libsentrykube to land on ops would have to follow this process.
Fix in libsentrykube and merge.
Make a release of libsentrykube with a new version.
Update the version of libsentrykube used by sentry-kube package.
Make a release of sentry-kube package.
Update the version used in ops.
Testing
Make a new virtualenv
~/workspace cd sentry-infra-tools/
ok 10:10:36
direnv: loading ~/workspace/sentry-infra-tools/.envrc
warning: creating virtualenv for the first time
Collecting pip==22.2.2
Obtaining dependency information for pip==22.2.2 from https://files.pythonhosted.org/packages/1f/2c/d9626f045e7b49a6225c6b09257861f24da78f4e5f23af2ddbdf852c99b8/pip-22.2.2-py3-none-any.whl.metadata
Using cached pip-22.2.2-py3-none-any.whl.metadata (4.2 kB)
Collecting wheel==0.37.1
Obtaining dependency information for wheel==0.37.1 from https://files.pythonhosted.org/packages/27/d6/003e593296a85fd6ed616ed962795b2f87709c3eee2bca4f6d0fe55c6d00/wheel-0.37.1-py2.py3-none-any.whl.metadata
Using cached wheel-0.37.1-py2.py3-none-any.whl.metadata (2.3 kB)
Using cached pip-22.2.2-py3-none-any.whl (2.0 MB)
Using cached wheel-0.37.1-py2.py3-none-any.whl (35 kB)
Installing collected packages: wheel, pip
Attempting uninstall: pip
Found existing installation: pip 23.2.1
Uninstalling pip-23.2.1:
Successfully uninstalled pip-23.2.1
Successfully installed pip-22.2.2 wheel-0.37.1
[notice] A new release of pip is available: 22.2.2 -> 24.2
[notice] To update, run: pip install --upgrade pip
pip install -q -r requirements-dev.txt
direnv: ([direnv export zsh]) is taking a while to execute. Use CTRL-C to give up.
[notice] A new release of pip available: 22.2.2 -> 24.2
[notice] To update, run: pip install --upgrade pip
pip install -q pre-commit==2.13.0
[notice] A new release of pip available: 22.2.2 -> 24.2
[notice] To update, run: pip install --upgrade pip
pre-commit install --install-hooks
pre-commit installed at .git/hooks/pre-commit
brew bundle
==> Downloading https://formulae.brew.sh/api/formula.jws.json
########################################################################################################### 100.0%
Using kube-linter
Homebrew Bundle complete! 1 Brewfile dependency now installed.
direnv: export +PYENV_SHELL +VIRTUAL_ENV ~PATH
direnv: unloading
Verify sentry-infra-tools is not installed
~/w/sentry-infra-tools nikhars/feat/make-package ?1 sentry-kube --help
ok 28s direnv 3.11.6 py 10:11:07
zsh: command not found: sentry-kube
Install sentry-infra-tools
~/w/sentry-infra-tools nikhars/feat/make-package ?1 pip install -e .
127 err direnv 3.11.6 py 10:11:17
Obtaining file:///Users/nikharsaxena/workspace/sentry-infra-tools
Installing build dependencies ... done
Checking if build backend supports build_editable ... done
Getting requirements to build editable ... done
Preparing editable metadata (pyproject.toml) ... done
Requirement already satisfied: anyio==4.4.0 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (4.4.0)
Collecting attrs==24.2.0
Using cached attrs-24.2.0-py3-none-any.whl (63 kB)
Requirement already satisfied: bcrypt==4.1.3 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (4.1.3)
Requirement already satisfied: cachetools==5.4.0 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (5.4.0)
Requirement already satisfied: certifi==2024.7.4 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (2024.7.4)
Requirement already satisfied: cffi==1.16.0 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (1.16.0)
Requirement already satisfied: charset-normalizer==3.3.2 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (3.3.2)
Requirement already satisfied: click==8.1.7 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (8.1.7)
Requirement already satisfied: cryptography==42.0.8 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (42.0.8)
Requirement already satisfied: dictdiffer==0.9.0 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (0.9.0)
Requirement already satisfied: fastjsonschema==2.20.0 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (2.20.0)
Requirement already satisfied: google-api-core==2.19.1 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (2.19.1)
Requirement already satisfied: google-api-python-client==2.137.0 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (2.137.0)
Requirement already satisfied: google-auth==2.32.0 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (2.32.0)
Requirement already satisfied: google-auth-httplib2==0.2.0 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (0.2.0)
Requirement already satisfied: googleapis-common-protos==1.63.2 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (1.63.2)
Requirement already satisfied: h11==0.14.0 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (0.14.0)
Requirement already satisfied: httpcore==1.0.5 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (1.0.5)
Requirement already satisfied: httplib2==0.22.0 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (0.22.0)
Requirement already satisfied: httpx==0.27.0 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (0.27.0)
Requirement already satisfied: idna==3.7 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (3.7)
Requirement already satisfied: jinja2==3.1.4 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (3.1.4)
Requirement already satisfied: jsonnet==0.20.0 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (0.20.0)
Requirement already satisfied: jsonpatch==1.33 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (1.33)
Requirement already satisfied: jsonpath-ng==1.6.1 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (1.6.1)
Requirement already satisfied: jsonpointer==3.0.0 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (3.0.0)
Requirement already satisfied: jsonschema==4.23.0 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (4.23.0)
Requirement already satisfied: jsonschema-specifications==2023.12.1 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (2023.12.1)
Requirement already satisfied: kubernetes==30.1.0 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (30.1.0)
Requirement already satisfied: lark==1.1.9 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (1.1.9)
Requirement already satisfied: markupsafe==2.1.5 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (2.1.5)
Requirement already satisfied: msgpack==1.0.8 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (1.0.8)
Requirement already satisfied: oauthlib==3.2.2 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (3.2.2)
Requirement already satisfied: paramiko==3.4.0 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (3.4.0)
Requirement already satisfied: pathspec==0.12.1 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (0.12.1)
Requirement already satisfied: ply==3.11 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (3.11)
Requirement already satisfied: proto-plus==1.24.0 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (1.24.0)
Requirement already satisfied: protobuf==5.27.2 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (5.27.2)
Requirement already satisfied: pyasn1==0.6.0 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (0.6.0)
Requirement already satisfied: pyasn1-modules==0.4.0 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (0.4.0)
Requirement already satisfied: pycparser==2.22 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (2.22)
Requirement already satisfied: pynacl==1.5.0 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (1.5.0)
Requirement already satisfied: pyparsing==3.1.2 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (3.1.2)
Requirement already satisfied: python-dateutil==2.9.0.post0 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (2.9.0.post0)
Requirement already satisfied: python-hcl2==4.3.4 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (4.3.4)
Requirement already satisfied: python-rapidjson==1.8 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (1.8)
Requirement already satisfied: pyyaml==6.0.1 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (6.0.1)
Requirement already satisfied: referencing==0.35.1 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (0.35.1)
Requirement already satisfied: requests==2.32.3 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (2.32.3)
Requirement already satisfied: requests-oauthlib==2.0.0 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (2.0.0)
Requirement already satisfied: responses==0.25.3 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (0.25.3)
Collecting rpds-py==0.20.0
Using cached rpds_py-0.20.0-cp311-cp311-macosx_11_0_arm64.whl (311 kB)
Requirement already satisfied: rsa==4.9 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (4.9)
Requirement already satisfied: sentry-jsonish==0.0.2 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (0.0.2)
Requirement already satisfied: sentry-jsonnet==0.0.4 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (0.0.4)
Requirement already satisfied: sentry-kafka-schemas==0.1.103 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (0.1.103)
Requirement already satisfied: sentry-sdk==2.10.0 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (2.10.0)
Requirement already satisfied: six==1.16.0 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (1.16.0)
Requirement already satisfied: sniffio==1.3.1 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (1.3.1)
Requirement already satisfied: sshuttle==1.1.2 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (1.1.2)
Requirement already satisfied: tabulate==0.9.0 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (0.9.0)
Requirement already satisfied: types-jsonschema==4.23.0.20240712 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (4.23.0.20240712)
Requirement already satisfied: types-paramiko==3.4.0.20240423 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (3.4.0.20240423)
Requirement already satisfied: types-pyyaml==6.0.12.20240311 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (6.0.12.20240311)
Requirement already satisfied: types-requests==2.32.0.20240712 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (2.32.0.20240712)
Requirement already satisfied: types-setuptools==70.3.0.20240710 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (70.3.0.20240710)
Requirement already satisfied: types-tabulate==0.9.0.20240106 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (0.9.0.20240106)
Requirement already satisfied: typing-extensions==4.12.2 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (4.12.2)
Requirement already satisfied: uritemplate==4.1.1 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (4.1.1)
Requirement already satisfied: urllib3==2.2.2 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (2.2.2)
Requirement already satisfied: websocket-client==1.8.0 in ./.venv/lib/python3.11/site-packages (from sentry-infra-tools==0.0.1) (1.8.0)
Building wheels for collected packages: sentry-infra-tools
Building editable for sentry-infra-tools (pyproject.toml) ... done
Created wheel for sentry-infra-tools: filename=sentry_infra_tools-0.0.1-0.editable-py3-none-any.whl size=5509 sha256=570728f4ee8d9785a8818b8e9bbc34f2122283fa2b894e213a42ef0a7c1337ea
Stored in directory: /private/var/folders/tb/g9_mm4k51rq0z28tykhkr66m0000gn/T/pip-ephem-wheel-cache-iu2ryb_p/wheels/22/a5/f7/3465af790d1cebd9ac36f649efce8a3910be2bb0bb84fd9e52
Successfully built sentry-infra-tools
Installing collected packages: rpds-py, attrs, sentry-infra-tools
Attempting uninstall: rpds-py
Found existing installation: rpds-py 0.19.0
Uninstalling rpds-py-0.19.0:
Successfully uninstalled rpds-py-0.19.0
Attempting uninstall: attrs
Found existing installation: attrs 23.2.0
Uninstalling attrs-23.2.0:
Successfully uninstalled attrs-23.2.0
Successfully installed attrs-24.2.0 rpds-py-0.20.0 sentry-infra-tools-0.0.1
[notice] A new release of pip available: 22.2.2 -> 24.2
[notice] To update, run: pip install --upgrade pip
Validate sentry-kube command is available after installation
~/w/sentry-infra-tools nikhars/feat/make-package ?1 sentry-kube --help
ok 4s direnv 3.11.6 py 10:11:25
Usage: sentry-kube [OPTIONS] COMMAND [ARGS]...
__ __
/ | / |
$$ | __ __ __ $$ |____ ______
$$ | / |/ | / |$$ \ / \
$$ |_/$$/ $$ | $$ |$$$$$$$ |/$$$$$$ |
$$ $$< $$ | $$ |$$ | $$ |$$ $$ |
$$$$$$ \ $$ \__$$ |$$ |__$$ |$$$$$$$$/
$$ | $$ |$$ $$/ $$ $$/ $$ |
$$/ $$/ $$$$$$/ $$$$$$$/ $$$$$$$/
Get kubed.
Options:
-c, --cluster TEXT Cluster name to operate on. This is ignored if the
customer has one cluster
--root DIRECTORY Skip auto-detection of root directory.
-C, --customer TEXT Customer name. Does not override the customer for
`connect`.
-q, --quiet Do not output informational messages
-s, --no-sentry Disables sending errors/transactions to Sentry when
running
--help Show this message and exit.
Commands:
apply
audit Generates a list of objects that exist...
cluster
connect
datadog-log Submits events to DataDog.
datadog-log-terragrunt Submits terraform events to DataDog.
detect-drift This command runs a `sentry-kube diff` on all...
diff Render a diff between production and local...
edit-secret kubectl edit secret, but made convenient.
get-context Get the kubernetes context value for use in...
get-customers Gets the list of all avaliable customers.
k9s Start k9s (a terminal based Kubernetes UI)
kafkactl Spawn kafkactl environment pod.
kubectl Confirmation wrapper for kubectl, for when you...
render Render a service(s).
rendervalues Render the end state of variables (after...
resolve-pvc Figure out which Pod and Node a PVC is bound to.
restart Restarts deployment(s) by bumping the...
run-job Apply a service's job to production.
run-pod
scale
scp Use SCP to transfer a file.
ssh SSH into a host.
toolbox Run a toolbox pod.
tunnel IAP tunnel into a host.
validate Renders the specified service and then runs...
~/w/sentry-infra-tools nikhars/feat/make-package ?1 python
ok 28s direnv 3.11.6 py 10:12:03
Python 3.11.6 (main, Apr 20 2024, 00:34:50) [Clang 15.0.0 (clang-1500.3.9.4)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from libsentrykube.utils import ensure_libsentrykube_folder
>>> ensure_libsentrykube_folder()
PosixPath('/Users/nikharsaxena/.libsentrykube')
Background
The existing structure of the sentry infra tools relies on 2 packages. sentry-kube and libsentrykube. In an ideal scenario we should have only a single package so that the release process when a change in libsentrykube happens involves
With 2 packages, a fix in libsentrykube to land on ops would have to follow this process.
Testing
Make a new virtualenv
Verify sentry-infra-tools is not installed
Install sentry-infra-tools
Validate sentry-kube command is available after installation
Validate that libsentrykube macros are available.
The only usage of libsentrykube macro directly in ops repo I found is this https://github.com/getsentry/ops/blob/master/bin/iam-permission-in-what-roles#L5. Verify that the import still works.