bug installing packages from github in 3.11 notebooks

[OwenPriceSkelly]

Noticed this while working on #436. I was sanity checking that the correct version of dill was installed on the endpoint in different python versions, and had modified containers.py so the RUN pip install garden-ai would install from my branch instead of pypi.

while an entrypoint published in a base-3.10 notebook was happy to install garden from my branch, the same entrypoint published in a base-3.11 notebook caused the pod to crash with these logs:

....
Requirement already satisfied: pycparser in /opt/conda/lib/python3.11/site-packages (from cffi>=1.12->cryptography!=3.4.0,>=3.3.1->globus-sdk->globus-compute-endpoint>=2.0) (2.21)
Building wheels for collected packages: pyzmq
  Building wheel for pyzmq (pyproject.toml): started
  Building wheel for pyzmq (pyproject.toml): finished with status 'error'
  error: subprocess-exited-with-error

  × Building wheel for pyzmq (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [174 lines of output]
      /tmp/pip-build-env-0f6nmyam/overlay/lib/python3.11/site-packages/setuptools/_distutils/dist.py:265: UserWarning: Unknown distribution option: 'cffi_modules'
        warnings.warn(msg)
      running bdist_wheel
      running build
      running build_py
      copying zmq/asyncio.py -> build/lib.linux-x86_64-cpython-311/zmq
      copying zmq/_future.py -> build/lib.linux-x86_64-cpython-311/zmq
      copying zmq/constants.py -> build/lib.linux-x86_64-cpython-311/zmq
      copying zmq/decorators.py -> build/lib.linux-x86_64-cpython-311/zmq
      copying zmq/__init__.py -> build/lib.linux-x86_64-cpython-311/zmq
      copying zmq/_typing.py -> build/lib.linux-x86_64-cpython-311/zmq
      copying zmq/error.py -> build/lib.linux-x86_64-cpython-311/zmq
      creating build/lib.linux-x86_64-cpython-311/zmq/sugar
      copying zmq/sugar/attrsettr.py -> build/lib.linux-x86_64-cpython-311/zmq/sugar
      copying zmq/sugar/poll.py -> build/lib.linux-x86_64-cpython-311/zmq/sugar
      copying zmq/sugar/context.py -> build/lib.linux-x86_64-cpython-311/zmq/sugar
      copying zmq/sugar/frame.py -> build/lib.linux-x86_64-cpython-311/zmq/sugar
      copying zmq/sugar/socket.py -> build/lib.linux-x86_64-cpython-311/zmq/sugar
      copying zmq/sugar/tracker.py -> build/lib.linux-x86_64-cpython-311/zmq/sugar
      copying zmq/sugar/version.py -> build/lib.linux-x86_64-cpython-311/zmq/sugar
      copying zmq/sugar/__init__.py -> build/lib.linux-x86_64-cpython-311/zmq/sugar
      copying zmq/sugar/stopwatch.py -> build/lib.linux-x86_64-cpython-311/zmq/sugar
      creating build/lib.linux-x86_64-cpython-311/zmq/auth
      copying zmq/auth/ioloop.py -> build/lib.linux-x86_64-cpython-311/zmq/auth
      copying zmq/auth/asyncio.py -> build/lib.linux-x86_64-cpython-311/zmq/auth
      copying zmq/auth/certs.py -> build/lib.linux-x86_64-cpython-311/zmq/auth
      copying zmq/auth/base.py -> build/lib.linux-x86_64-cpython-311/zmq/auth
      copying zmq/auth/__init__.py -> build/lib.linux-x86_64-cpython-311/zmq/auth
      copying zmq/auth/thread.py -> build/lib.linux-x86_64-cpython-311/zmq/auth
      creating build/lib.linux-x86_64-cpython-311/zmq/devices
      copying zmq/devices/monitoredqueue.py -> build/lib.linux-x86_64-cpython-311/zmq/devices
      copying zmq/devices/proxysteerabledevice.py -> build/lib.linux-x86_64-cpython-311/zmq/devices
      copying zmq/devices/proxydevice.py -> build/lib.linux-x86_64-cpython-311/zmq/devices
      copying zmq/devices/__init__.py -> build/lib.linux-x86_64-cpython-311/zmq/devices
      copying zmq/devices/monitoredqueuedevice.py -> build/lib.linux-x86_64-cpython-311/zmq/devices
      copying zmq/devices/basedevice.py -> build/lib.linux-x86_64-cpython-311/zmq/devices
      creating build/lib.linux-x86_64-cpython-311/zmq/log
      copying zmq/log/__main__.py -> build/lib.linux-x86_64-cpython-311/zmq/log
      copying zmq/log/__init__.py -> build/lib.linux-x86_64-cpython-311/zmq/log
      copying zmq/log/handlers.py -> build/lib.linux-x86_64-cpython-311/zmq/log
      creating build/lib.linux-x86_64-cpython-311/zmq/utils
      copying zmq/utils/jsonapi.py -> build/lib.linux-x86_64-cpython-311/zmq/utils
      copying zmq/utils/interop.py -> build/lib.linux-x86_64-cpython-311/zmq/utils
      copying zmq/utils/z85.py -> build/lib.linux-x86_64-cpython-311/zmq/utils
      copying zmq/utils/__init__.py -> build/lib.linux-x86_64-cpython-311/zmq/utils
      copying zmq/utils/garbage.py -> build/lib.linux-x86_64-cpython-311/zmq/utils
      copying zmq/utils/win32.py -> build/lib.linux-x86_64-cpython-311/zmq/utils
      copying zmq/utils/monitor.py -> build/lib.linux-x86_64-cpython-311/zmq/utils
      copying zmq/utils/strtypes.py -> build/lib.linux-x86_64-cpython-311/zmq/utils
      creating build/lib.linux-x86_64-cpython-311/zmq/backend
      copying zmq/backend/__init__.py -> build/lib.linux-x86_64-cpython-311/zmq/backend
      copying zmq/backend/select.py -> build/lib.linux-x86_64-cpython-311/zmq/backend
      creating build/lib.linux-x86_64-cpython-311/zmq/backend/cython
      copying zmq/backend/cython/__init__.py -> build/lib.linux-x86_64-cpython-311/zmq/backend/cython
      creating build/lib.linux-x86_64-cpython-311/zmq/backend/cffi
      copying zmq/backend/cffi/_poll.py -> build/lib.linux-x86_64-cpython-311/zmq/backend/cffi
      copying zmq/backend/cffi/context.py -> build/lib.linux-x86_64-cpython-311/zmq/backend/cffi
      copying zmq/backend/cffi/socket.py -> build/lib.linux-x86_64-cpython-311/zmq/backend/cffi
      copying zmq/backend/cffi/__init__.py -> build/lib.linux-x86_64-cpython-311/zmq/backend/cffi
      copying zmq/backend/cffi/utils.py -> build/lib.linux-x86_64-cpython-311/zmq/backend/cffi
      copying zmq/backend/cffi/devices.py -> build/lib.linux-x86_64-cpython-311/zmq/backend/cffi
      copying zmq/backend/cffi/error.py -> build/lib.linux-x86_64-cpython-311/zmq/backend/cffi
      copying zmq/backend/cffi/message.py -> build/lib.linux-x86_64-cpython-311/zmq/backend/cffi
      creating build/lib.linux-x86_64-cpython-311/zmq/eventloop
      copying zmq/eventloop/ioloop.py -> build/lib.linux-x86_64-cpython-311/zmq/eventloop
      copying zmq/eventloop/__init__.py -> build/lib.linux-x86_64-cpython-311/zmq/eventloop
      copying zmq/eventloop/future.py -> build/lib.linux-x86_64-cpython-311/zmq/eventloop
      copying zmq/eventloop/_deprecated.py -> build/lib.linux-x86_64-cpython-311/zmq/eventloop
      copying zmq/eventloop/zmqstream.py -> build/lib.linux-x86_64-cpython-311/zmq/eventloop
      creating build/lib.linux-x86_64-cpython-311/zmq/eventloop/minitornado
      copying zmq/eventloop/minitornado/ioloop.py -> build/lib.linux-x86_64-cpython-311/zmq/eventloop/minitornado
      copying zmq/eventloop/minitornado/util.py -> build/lib.linux-x86_64-cpython-311/zmq/eventloop/minitornado
      copying zmq/eventloop/minitornado/log.py -> build/lib.linux-x86_64-cpython-311/zmq/eventloop/minitornado
      copying zmq/eventloop/minitornado/stack_context.py -> build/lib.linux-x86_64-cpython-311/zmq/eventloop/minitornado
      copying zmq/eventloop/minitornado/concurrent.py -> build/lib.linux-x86_64-cpython-311/zmq/eventloop/minitornado
      copying zmq/eventloop/minitornado/__init__.py -> build/lib.linux-x86_64-cpython-311/zmq/eventloop/minitornado
      creating build/lib.linux-x86_64-cpython-311/zmq/eventloop/minitornado/platform
      copying zmq/eventloop/minitornado/platform/common.py -> build/lib.linux-x86_64-cpython-311/zmq/eventloop/minitornado/platform
      copying zmq/eventloop/minitornado/platform/windows.py -> build/lib.linux-x86_64-cpython-311/zmq/eventloop/minitornado/platform
      copying zmq/eventloop/minitornado/platform/auto.py -> build/lib.linux-x86_64-cpython-311/zmq/eventloop/minitornado/platform
      copying zmq/eventloop/minitornado/platform/__init__.py -> build/lib.linux-x86_64-cpython-311/zmq/eventloop/minitornado/platform
      copying zmq/eventloop/minitornado/platform/interface.py -> build/lib.linux-x86_64-cpython-311/zmq/eventloop/minitornado/platform
      copying zmq/eventloop/minitornado/platform/posix.py -> build/lib.linux-x86_64-cpython-311/zmq/eventloop/minitornado/platform
      creating build/lib.linux-x86_64-cpython-311/zmq/green
      copying zmq/green/device.py -> build/lib.linux-x86_64-cpython-311/zmq/green
      copying zmq/green/poll.py -> build/lib.linux-x86_64-cpython-311/zmq/green
      copying zmq/green/core.py -> build/lib.linux-x86_64-cpython-311/zmq/green
      copying zmq/green/__init__.py -> build/lib.linux-x86_64-cpython-311/zmq/green
      creating build/lib.linux-x86_64-cpython-311/zmq/green/eventloop
      copying zmq/green/eventloop/ioloop.py -> build/lib.linux-x86_64-cpython-311/zmq/green/eventloop
      copying zmq/green/eventloop/__init__.py -> build/lib.linux-x86_64-cpython-311/zmq/green/eventloop
      copying zmq/green/eventloop/zmqstream.py -> build/lib.linux-x86_64-cpython-311/zmq/green/eventloop
      creating build/lib.linux-x86_64-cpython-311/zmq/ssh
      copying zmq/ssh/tunnel.py -> build/lib.linux-x86_64-cpython-311/zmq/ssh
      copying zmq/ssh/__init__.py -> build/lib.linux-x86_64-cpython-311/zmq/ssh
      copying zmq/ssh/forward.py -> build/lib.linux-x86_64-cpython-311/zmq/ssh
      creating build/lib.linux-x86_64-cpython-311/zmq/tests
      copying zmq/tests/test_log.py -> build/lib.linux-x86_64-cpython-311/zmq/tests
      copying zmq/tests/test_pair.py -> build/lib.linux-x86_64-cpython-311/zmq/tests
      copying zmq/tests/test_mypy.py -> build/lib.linux-x86_64-cpython-311/zmq/tests
      copying zmq/tests/test_monitor.py -> build/lib.linux-x86_64-cpython-311/zmq/tests
      copying zmq/tests/test_decorators.py -> build/lib.linux-x86_64-cpython-311/zmq/tests
      copying zmq/tests/test_ext.py -> build/lib.linux-x86_64-cpython-311/zmq/tests
      copying zmq/tests/test_constants.py -> build/lib.linux-x86_64-cpython-311/zmq/tests
      copying zmq/tests/test_imports.py -> build/lib.linux-x86_64-cpython-311/zmq/tests
      copying zmq/tests/conftest.py -> build/lib.linux-x86_64-cpython-311/zmq/tests
      copying zmq/tests/test_poll.py -> build/lib.linux-x86_64-cpython-311/zmq/tests
      copying zmq/tests/test_cython.py -> build/lib.linux-x86_64-cpython-311/zmq/tests
      copying zmq/tests/test_etc.py -> build/lib.linux-x86_64-cpython-311/zmq/tests
      copying zmq/tests/test_message.py -> build/lib.linux-x86_64-cpython-311/zmq/tests
      copying zmq/tests/test_asyncio.py -> build/lib.linux-x86_64-cpython-311/zmq/tests
      copying zmq/tests/test_version.py -> build/lib.linux-x86_64-cpython-311/zmq/tests
      copying zmq/tests/test_reqrep.py -> build/lib.linux-x86_64-cpython-311/zmq/tests
      copying zmq/tests/test_security.py -> build/lib.linux-x86_64-cpython-311/zmq/tests
      copying zmq/tests/test_cffi_backend.py -> build/lib.linux-x86_64-cpython-311/zmq/tests
      copying zmq/tests/test_error.py -> build/lib.linux-x86_64-cpython-311/zmq/tests
      copying zmq/tests/test_pubsub.py -> build/lib.linux-x86_64-cpython-311/zmq/tests
      copying zmq/tests/test_z85.py -> build/lib.linux-x86_64-cpython-311/zmq/tests
      copying zmq/tests/test_includes.py -> build/lib.linux-x86_64-cpython-311/zmq/tests
      copying zmq/tests/test_monqueue.py -> build/lib.linux-x86_64-cpython-311/zmq/tests
      copying zmq/tests/test_auth.py -> build/lib.linux-x86_64-cpython-311/zmq/tests
      copying zmq/tests/__init__.py -> build/lib.linux-x86_64-cpython-311/zmq/tests
      copying zmq/tests/test_future.py -> build/lib.linux-x86_64-cpython-311/zmq/tests
      copying zmq/tests/test_device.py -> build/lib.linux-x86_64-cpython-311/zmq/tests
      copying zmq/tests/test_win32_shim.py -> build/lib.linux-x86_64-cpython-311/zmq/tests
      copying zmq/tests/test_context.py -> build/lib.linux-x86_64-cpython-311/zmq/tests
      copying zmq/tests/test_ioloop.py -> build/lib.linux-x86_64-cpython-311/zmq/tests
      copying zmq/tests/test_multipart.py -> build/lib.linux-x86_64-cpython-311/zmq/tests
      copying zmq/tests/test_ssh.py -> build/lib.linux-x86_64-cpython-311/zmq/tests
      copying zmq/tests/test_socket.py -> build/lib.linux-x86_64-cpython-311/zmq/tests
      copying zmq/tests/test_retry_eintr.py -> build/lib.linux-x86_64-cpython-311/zmq/tests
      copying zmq/tests/test_draft.py -> build/lib.linux-x86_64-cpython-311/zmq/tests
      copying zmq/tests/test_proxy_steerable.py -> build/lib.linux-x86_64-cpython-311/zmq/tests
      copying zmq/tests/test_zmqstream.py -> build/lib.linux-x86_64-cpython-311/zmq/tests
      copying zmq/__init__.pyi -> build/lib.linux-x86_64-cpython-311/zmq
      copying zmq/py.typed -> build/lib.linux-x86_64-cpython-311/zmq
      copying zmq/__init__.pxd -> build/lib.linux-x86_64-cpython-311/zmq
      copying zmq/sugar/__init__.pyi -> build/lib.linux-x86_64-cpython-311/zmq/sugar
      copying zmq/devices/monitoredqueue.pxd -> build/lib.linux-x86_64-cpython-311/zmq/devices
      copying zmq/utils/buffers.pxd -> build/lib.linux-x86_64-cpython-311/zmq/utils
      copying zmq/utils/getpid_compat.h -> build/lib.linux-x86_64-cpython-311/zmq/utils
      copying zmq/utils/pyversion_compat.h -> build/lib.linux-x86_64-cpython-311/zmq/utils
      copying zmq/utils/mutex.h -> build/lib.linux-x86_64-cpython-311/zmq/utils
      copying zmq/utils/zmq_compat.h -> build/lib.linux-x86_64-cpython-311/zmq/utils
      copying zmq/utils/ipcmaxlen.h -> build/lib.linux-x86_64-cpython-311/zmq/utils
      copying zmq/utils/config.json -> build/lib.linux-x86_64-cpython-311/zmq/utils
      copying zmq/utils/compiler.json -> build/lib.linux-x86_64-cpython-311/zmq/utils
      copying zmq/backend/__init__.pyi -> build/lib.linux-x86_64-cpython-311/zmq/backend
      copying zmq/backend/cython/context.pxd -> build/lib.linux-x86_64-cpython-311/zmq/backend/cython
      copying zmq/backend/cython/__init__.pxd -> build/lib.linux-x86_64-cpython-311/zmq/backend/cython
      copying zmq/backend/cython/message.pxd -> build/lib.linux-x86_64-cpython-311/zmq/backend/cython
      copying zmq/backend/cython/libzmq.pxd -> build/lib.linux-x86_64-cpython-311/zmq/backend/cython
      copying zmq/backend/cython/socket.pxd -> build/lib.linux-x86_64-cpython-311/zmq/backend/cython
      copying zmq/backend/cython/checkrc.pxd -> build/lib.linux-x86_64-cpython-311/zmq/backend/cython
      copying zmq/backend/cython/constant_enums.pxi -> build/lib.linux-x86_64-cpython-311/zmq/backend/cython
      copying zmq/backend/cffi/_cdefs.h -> build/lib.linux-x86_64-cpython-311/zmq/backend/cffi
      running build_ext
      running configure
      Using bundled libzmq
      already have bundled/zeromq
      already have platform.hpp
      checking for timer_create
      gcc -pthread -B /opt/conda/compiler_compat -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /opt/conda/include -fPIC -O2 -isystem /opt/conda/include -fPIC -I/opt/conda/include/python3.11 -c /tmp/timer_createermavrme.c -o tmp/timer_createermavrme.o
      no timer_create, linking librt
      building 'zmq.libzmq' extension
      creating build/temp.linux-x86_64-cpython-311/buildutils
      creating build/temp.linux-x86_64-cpython-311/bundled
      creating build/temp.linux-x86_64-cpython-311/bundled/zeromq
      creating build/temp.linux-x86_64-cpython-311/bundled/zeromq/src
      g++ -pthread -B /opt/conda/compiler_compat -std=c++11 -pthread -B /opt/conda/compiler_compat -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /opt/conda/include -fPIC -O2 -isystem /opt/conda/include -fPIC -DZMQ_HAVE_CURVE=1 -DZMQ_USE_TWEETNACL=1 -DZMQ_USE_EPOLL=1 -DZMQ_IOTHREADS_USE_EPOLL=1 -DZMQ_POLL_BASED_ON_POLL=1 -Ibundled/zeromq/include -Ibundled -I/opt/conda/include/python3.11 -c buildutils/initlibzmq.cpp -o build/temp.linux-x86_64-cpython-311/buildutils/initlibzmq.o
      ************************************************
      ************************************************
      ************************************************
      error: command 'g++' failed: No such file or directory
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for pyzmq
Failed to build pyzmq
ERROR: Could not build wheels for pyzmq, which is required to install pyproject.toml-based projects
/bin/bash: line 3: globus-compute-manager: command not found

I'm not even 100% sure that it's related to installing from github -- I'm guessing that it is because of the reference to pyproject.toml-based projects, but this needs a little more investigation. unfortunately, I don't think I can easily test this suspicion until I've cut the next release relaxing the dill requirement for 3.11+ when installed from pypi