Use Python 3.9, including for ingestion lambdas

[huonw]

This fixes #248 by upgrading the Python version from 3.7 to 3.9. 3.7 is going out of support soon (end of November), so this has to happen.

This also requires upgrading a few dev dependencies to newer versions that provide pre-built wheels for 3.9 for my machine. I only upgraded the ones don't build by default on my computer (ARM macOS, with various packages/compilers installed), there are likely others that should be upgraded.

I tried upgrading to even newer versions but these failed to run pytest (presumably it and/or other libs need to be upgraded too), plus, it seems the latest version of https://github.com/newrelic/aws-log-ingestion only supports 3.9.

Error on 3.10

``` Traceback (most recent call last): File "/opt/homebrew/Cellar/python@3.10/3.10.10_1/Frameworks/Python.framework/Versions/3.10/lib/python3.10/runpy.py", line 196, in _run_module_as_main return _run_code(code, main_globals, None, File "/opt/homebrew/Cellar/python@3.10/3.10.10_1/Frameworks/Python.framework/Versions/3.10/lib/python3.10/runpy.py", line 86, in _run_code exec(code, run_globals) File "/Users/huon/projects/newrelic/newrelic-lambda-cli/.venv310/lib/python3.10/site-packages/pytest/__main__.py", line 7, in raise SystemExit(pytest.console_main()) File "/Users/huon/projects/newrelic/newrelic-lambda-cli/.venv310/lib/python3.10/site-packages/_pytest/config/__init__.py", line 180, in console_main code = main() File "/Users/huon/projects/newrelic/newrelic-lambda-cli/.venv310/lib/python3.10/site-packages/_pytest/config/__init__.py", line 136, in main config = _prepareconfig(args, plugins) File "/Users/huon/projects/newrelic/newrelic-lambda-cli/.venv310/lib/python3.10/site-packages/_pytest/config/__init__.py", line 313, in _prepareconfig config = pluginmanager.hook.pytest_cmdline_parse( File "/Users/huon/projects/newrelic/newrelic-lambda-cli/.venv310/lib/python3.10/site-packages/pluggy/hooks.py", line 286, in __call__ return self._hookexec(self, self.get_hookimpls(), kwargs) File "/Users/huon/projects/newrelic/newrelic-lambda-cli/.venv310/lib/python3.10/site-packages/pluggy/manager.py", line 93, in _hookexec return self._inner_hookexec(hook, methods, kwargs) File "/Users/huon/projects/newrelic/newrelic-lambda-cli/.venv310/lib/python3.10/site-packages/pluggy/manager.py", line 84, in self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall( File "/Users/huon/projects/newrelic/newrelic-lambda-cli/.venv310/lib/python3.10/site-packages/pluggy/callers.py", line 203, in _multicall gen.send(outcome) File "/Users/huon/projects/newrelic/newrelic-lambda-cli/.venv310/lib/python3.10/site-packages/_pytest/helpconfig.py", line 99, in pytest_cmdline_parse config = outcome.get_result() # type: Config File "/Users/huon/projects/newrelic/newrelic-lambda-cli/.venv310/lib/python3.10/site-packages/pluggy/callers.py", line 80, in get_result raise ex[1].with_traceback(ex[2]) File "/Users/huon/projects/newrelic/newrelic-lambda-cli/.venv310/lib/python3.10/site-packages/pluggy/callers.py", line 187, in _multicall res = hook_impl.function(*args) File "/Users/huon/projects/newrelic/newrelic-lambda-cli/.venv310/lib/python3.10/site-packages/_pytest/config/__init__.py", line 932, in pytest_cmdline_parse self.parse(args) File "/Users/huon/projects/newrelic/newrelic-lambda-cli/.venv310/lib/python3.10/site-packages/_pytest/config/__init__.py", line 1204, in parse self._preparse(args, addopts=addopts) File "/Users/huon/projects/newrelic/newrelic-lambda-cli/.venv310/lib/python3.10/site-packages/_pytest/config/__init__.py", line 1097, in _preparse self.pluginmanager.load_setuptools_entrypoints("pytest11") File "/Users/huon/projects/newrelic/newrelic-lambda-cli/.venv310/lib/python3.10/site-packages/pluggy/manager.py", line 299, in load_setuptools_entrypoints plugin = ep.load() File "/opt/homebrew/Cellar/python@3.10/3.10.10_1/Frameworks/Python.framework/Versions/3.10/lib/python3.10/importlib/metadata/__init__.py", line 171, in load module = import_module(match.group('module')) File "/opt/homebrew/Cellar/python@3.10/3.10.10_1/Frameworks/Python.framework/Versions/3.10/lib/python3.10/importlib/__init__.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "", line 1050, in _gcd_import File "", line 1027, in _find_and_load File "", line 1006, in _find_and_load_unlocked File "", line 688, in _load_unlocked File "/Users/huon/projects/newrelic/newrelic-lambda-cli/.venv310/lib/python3.10/site-packages/_pytest/assertion/rewrite.py", line 161, in exec_module source_stat, co = _rewrite_test(fn, self.config) File "/Users/huon/projects/newrelic/newrelic-lambda-cli/.venv310/lib/python3.10/site-packages/_pytest/assertion/rewrite.py", line 360, in _rewrite_test co = compile(tree, fn_, "exec", dont_inherit=True) TypeError: required field "lineno" missing from alias ```

Error on 3.11

``` Traceback (most recent call last): File "/Users/huon/projects/newrelic/newrelic-lambda-cli/.venv/lib/python3.11/site-packages/py/_vendored_packages/apipkg.py", line 141, in __makeattr modpath, attrname = self.__map__[name] ~~~~~~~~~~~~^^^^^^ KeyError: '__spec__' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "", line 189, in _run_module_as_main File "", line 148, in _get_module_details File "", line 112, in _get_module_details File "/Users/huon/projects/newrelic/newrelic-lambda-cli/.venv/lib/python3.11/site-packages/pytest/__init__.py", line 7, in from _pytest.assertion import register_assert_rewrite File "/Users/huon/projects/newrelic/newrelic-lambda-cli/.venv/lib/python3.11/site-packages/_pytest/assertion/__init__.py", line 10, in from _pytest.assertion import rewrite File "/Users/huon/projects/newrelic/newrelic-lambda-cli/.venv/lib/python3.11/site-packages/_pytest/assertion/rewrite.py", line 30, in from _pytest.assertion import util File "/Users/huon/projects/newrelic/newrelic-lambda-cli/.venv/lib/python3.11/site-packages/_pytest/assertion/util.py", line 14, in import _pytest._code File "/Users/huon/projects/newrelic/newrelic-lambda-cli/.venv/lib/python3.11/site-packages/_pytest/_code/__init__.py", line 2, in from .code import Code File "/Users/huon/projects/newrelic/newrelic-lambda-cli/.venv/lib/python3.11/site-packages/_pytest/_code/code.py", line 53, in class Code: File "/Users/huon/projects/newrelic/newrelic-lambda-cli/.venv/lib/python3.11/site-packages/_pytest/_code/code.py", line 73, in Code def path(self) -> Union[py.path.local, str]: ^^^^^^^^^^^^^ File "/Users/huon/projects/newrelic/newrelic-lambda-cli/.venv/lib/python3.11/site-packages/py/_vendored_packages/apipkg.py", line 148, in __makeattr result = importobj(modpath, attrname) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/huon/projects/newrelic/newrelic-lambda-cli/.venv/lib/python3.11/site-packages/py/_vendored_packages/apipkg.py", line 69, in importobj module = __import__(modpath, None, None, ['__doc__']) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "", line 1176, in _find_and_load File "", line 1136, in _find_and_load_unlocked File "/Users/huon/projects/newrelic/newrelic-lambda-cli/.venv/lib/python3.11/site-packages/py/_vendored_packages/apipkg.py", line 146, in __makeattr raise AttributeError(name) AttributeError: __spec__ ```

I'm not sure how to test accurately, and wonder if further changes may be required.

[CLAassistant]

All committers have signed the CLA.

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[huonw]

I think we probably need to upgrade these code URI references too, to the latest version of https://github.com/newrelic/aws-log-ingestion, but I'm not sure how to compute them: https://github.com/newrelic/newrelic-lambda-cli/blob/3732e00485dd96bd627d298477d2f3f3a8a55d8b/newrelic_lambda_cli/templates/import-template.yaml#L62

[huonw]

Hi @umaannamalai and @TimPansino, is there a chance you could take a look at this and let me know if there's a path forward? We've deployed lambda integration via this CLI with Python 3.7 and would prefer to upgrade it using the same method before AWS's November 27 end-of-support deadline. Thank you!

[TimPansino]

FYI We fixed this, and we're going to try and get approval to release this tomorrow. We're under a code freeze and we'll have to get an exception, if it's not approved then this won't release until the 29th. Sorry for the potential inconvenience.

[huonw]

Thank you!