issue with ruamel.yaml when trying to install azureml-sdk[databricks] on a Databricks cluster

[jknmsft]

I have tried installing azureml-sdk[databricks] on a Azure Databricks cluster, runtime 11.0 ML (includes Apache Spark 3.3.0, Scala 2.12), node type Standard_DS_v2 in two different ways 1) Using the UI following the instructions in the Track Azure Databricks experiments in Azure Machine Learning course (https://docs.microsoft.com/en-us/learn/modules/track-azure-databricks-experiments-azure-machine-learning/?ns-enrollment-type=learningpath&ns-enrollment-id=learn.wwl.build-operate-machine-learning-solutions-azure-databricks) and 2) With !pip install azureml-sdk[databricks]

With (1) installation never completes With (2) installation fails. The output shows several instances of INFO: pip is looking at multiple versions of azureml-train to determine which version is compatible with other requirements. This could take a while. and INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. See https://pip.pypa.io/warnings/backtracking for guidance. If you want to abort this run, press Ctrl + C.
After about 90 seconds installation fails with the following error message

            |                                                                                            |
            |                                                                                            char *
      In file included from ext/_ruamel_yaml.h:2,
                       from ext/_ruamel_yaml.c:523:
      ext/yaml.h:582:22: note: expected ‘yaml_char_t *’ {aka ‘unsigned char *’} but argument is of type ‘char *’
        582 |         yaml_char_t *value, int length,
            |         ~~~~~~~~~~~~~^~~~~
      ext/_ruamel_yaml.c:18390:71: warning: pointer targets in passing argument 2 of ‘yaml_sequence_start_event_initialize’ differ in signedness [-Wpointer-sign]
      18390 |     __pyx_t_3 = ((yaml_sequence_start_event_initialize(__pyx_v_event, __pyx_v_anchor, __pyx_v_tag, __pyx_v_implicit, __pyx_v_sequence_style) == 0) != 0);
            |                                                                       ^~~~~~~~~~~~~~
            |                                                                       |
            |                                                                       char *
      In file included from ext/_ruamel_yaml.h:2,
                       from ext/_ruamel_yaml.c:523:
      ext/yaml.h:604:22: note: expected ‘yaml_char_t *’ {aka ‘unsigned char *’} but argument is of type ‘char *’
        604 |         yaml_char_t *anchor, yaml_char_t *tag, int implicit,
            |         ~~~~~~~~~~~~~^~~~~~
      ext/_ruamel_yaml.c:18390:87: warning: pointer targets in passing argument 3 of ‘yaml_sequence_start_event_initialize’ differ in signedness [-Wpointer-sign]
      18390 |     __pyx_t_3 = ((yaml_sequence_start_event_initialize(__pyx_v_event, __pyx_v_anchor, __pyx_v_tag, __pyx_v_implicit, __pyx_v_sequence_style) == 0) != 0);
            |                                                                                       ^~~~~~~~~~~
            |                                                                                       |
            |                                                                                       char *
      In file included from ext/_ruamel_yaml.h:2,
                       from ext/_ruamel_yaml.c:523:
      ext/yaml.h:604:43: note: expected ‘yaml_char_t *’ {aka ‘unsigned char *’} but argument is of type ‘char *’
        604 |         yaml_char_t *anchor, yaml_char_t *tag, int implicit,
            |                              ~~~~~~~~~~~~~^~~
      ext/_ruamel_yaml.c:18824:70: warning: pointer targets in passing argument 2 of ‘yaml_mapping_start_event_initialize’ differ in signedness [-Wpointer-sign]
      18824 |     __pyx_t_2 = ((yaml_mapping_start_event_initialize(__pyx_v_event, __pyx_v_anchor, __pyx_v_tag, __pyx_v_implicit, __pyx_v_mapping_style) == 0) != 0);
            |                                                                      ^~~~~~~~~~~~~~
            |                                                                      |
            |                                                                      char *
      In file included from ext/_ruamel_yaml.h:2,
                       from ext/_ruamel_yaml.c:523:
      ext/yaml.h:636:22: note: expected ‘yaml_char_t *’ {aka ‘unsigned char *’} but argument is of type ‘char *’
        636 |         yaml_char_t *anchor, yaml_char_t *tag, int implicit,
            |         ~~~~~~~~~~~~~^~~~~~
      ext/_ruamel_yaml.c:18824:86: warning: pointer targets in passing argument 3 of ‘yaml_mapping_start_event_initialize’ differ in signedness [-Wpointer-sign]
      18824 |     __pyx_t_2 = ((yaml_mapping_start_event_initialize(__pyx_v_event, __pyx_v_anchor, __pyx_v_tag, __pyx_v_implicit, __pyx_v_mapping_style) == 0) != 0);
            |                                                                                      ^~~~~~~~~~~
            |                                                                                      |
            |                                                                                      char *
      In file included from ext/_ruamel_yaml.h:2,
                       from ext/_ruamel_yaml.c:523:
      ext/yaml.h:636:43: note: expected ‘yaml_char_t *’ {aka ‘unsigned char *’} but argument is of type ‘char *’
        636 |         yaml_char_t *anchor, yaml_char_t *tag, int implicit,
            |                              ~~~~~~~~~~~~~^~~
      ext/_ruamel_yaml.c: In function ‘__pyx_pf_12_ruamel_yaml_8CEmitter_12serialize’:
      ext/_ruamel_yaml.c:20303:42: warning: pointer targets in assignment from ‘char *’ to ‘yaml_char_t *’ {aka ‘unsigned char *’} differ in signedness [-Wpointer-sign]
      20303 |       __pyx_v_tag_directives_end->handle = PyString_AS_STRING(__pyx_v_handle);
            |                                          ^
      ext/_ruamel_yaml.c:20418:42: warning: pointer targets in assignment from ‘char *’ to ‘yaml_char_t *’ {aka ‘unsigned char *’} differ in signedness [-Wpointer-sign]
      20418 |       __pyx_v_tag_directives_end->prefix = PyString_AS_STRING(__pyx_v_prefix);
            |                                          ^
      ext/_ruamel_yaml.c: In function ‘__pyx_f_12_ruamel_yaml_8CEmitter__serialize_node’:
      ext/_ruamel_yaml.c:21278:65: warning: pointer targets in passing argument 2 of ‘yaml_alias_event_initialize’ differ in signedness [-Wpointer-sign]
      21278 |     __pyx_t_2 = ((yaml_alias_event_initialize((&__pyx_v_event), __pyx_v_anchor) == 0) != 0);
            |                                                                 ^~~~~~~~~~~~~~
            |                                                                 |
            |                                                                 char *
      In file included from ext/_ruamel_yaml.h:2,
                       from ext/_ruamel_yaml.c:523:
      ext/yaml.h:555:63: note: expected ‘yaml_char_t *’ {aka ‘unsigned char *’} but argument is of type ‘char *’
        555 | yaml_alias_event_initialize(yaml_event_t *event, yaml_char_t *anchor);
            |                                                  ~~~~~~~~~~~~~^~~~~~
      ext/_ruamel_yaml.c:22089:68: warning: pointer targets in passing argument 2 of ‘yaml_scalar_event_initialize’ differ in signedness [-Wpointer-sign]
      22089 |       __pyx_t_2 = ((yaml_scalar_event_initialize((&__pyx_v_event), __pyx_v_anchor, __pyx_v_tag, __pyx_v_value, __pyx_v_length, __pyx_v_plain_implicit, __pyx_v_quoted_implicit, __pyx_v_scalar_style) == 0) != 0);
            |                                                                    ^~~~~~~~~~~~~~
            |                                                                    |
            |                                                                    char *
      In file included from ext/_ruamel_yaml.h:2,
                       from ext/_ruamel_yaml.c:523:
      ext/yaml.h:581:22: note: expected ‘yaml_char_t *’ {aka ‘unsigned char *’} but argument is of type ‘char *’
        581 |         yaml_char_t *anchor, yaml_char_t *tag,
            |         ~~~~~~~~~~~~~^~~~~~
      ext/_ruamel_yaml.c:22089:84: warning: pointer targets in passing argument 3 of ‘yaml_scalar_event_initialize’ differ in signedness [-Wpointer-sign]
      22089 |       __pyx_t_2 = ((yaml_scalar_event_initialize((&__pyx_v_event), __pyx_v_anchor, __pyx_v_tag, __pyx_v_value, __pyx_v_length, __pyx_v_plain_implicit, __pyx_v_quoted_implicit, __pyx_v_scalar_style) == 0) != 0);
            |                                                                                    ^~~~~~~~~~~
            |                                                                                    |
            |                                                                                    char *
      In file included from ext/_ruamel_yaml.h:2,
                       from ext/_ruamel_yaml.c:523:
      ext/yaml.h:581:43: note: expected ‘yaml_char_t *’ {aka ‘unsigned char *’} but argument is of type ‘char *’
        581 |         yaml_char_t *anchor, yaml_char_t *tag,
            |                              ~~~~~~~~~~~~~^~~
      ext/_ruamel_yaml.c:22089:97: warning: pointer targets in passing argument 4 of ‘yaml_scalar_event_initialize’ differ in signedness [-Wpointer-sign]
      22089 |       __pyx_t_2 = ((yaml_scalar_event_initialize((&__pyx_v_event), __pyx_v_anchor, __pyx_v_tag, __pyx_v_value, __pyx_v_length, __pyx_v_plain_implicit, __pyx_v_quoted_implicit, __pyx_v_scalar_style) == 0) != 0);
            |                                                                                                 ^~~~~~~~~~~~~
            |                                                                                                 |
            |                                                                                                 char *
      In file included from ext/_ruamel_yaml.h:2,
                       from ext/_ruamel_yaml.c:523:
      ext/yaml.h:582:22: note: expected ‘yaml_char_t *’ {aka ‘unsigned char *’} but argument is of type ‘char *’
        582 |         yaml_char_t *value, int length,
            |         ~~~~~~~~~~~~~^~~~~
      ext/_ruamel_yaml.c:22476:76: warning: pointer targets in passing argument 2 of ‘yaml_sequence_start_event_initialize’ differ in signedness [-Wpointer-sign]
      22476 |       __pyx_t_2 = ((yaml_sequence_start_event_initialize((&__pyx_v_event), __pyx_v_anchor, __pyx_v_tag, __pyx_v_implicit, __pyx_v_sequence_style) == 0) != 0);
            |                                                                            ^~~~~~~~~~~~~~
            |                                                                            |
            |                                                                            char *
      In file included from ext/_ruamel_yaml.h:2,
                       from ext/_ruamel_yaml.c:523:
      ext/yaml.h:604:22: note: expected ‘yaml_char_t *’ {aka ‘unsigned char *’} but argument is of type ‘char *’
        604 |         yaml_char_t *anchor, yaml_char_t *tag, int implicit,
            |         ~~~~~~~~~~~~~^~~~~~
      ext/_ruamel_yaml.c:22476:92: warning: pointer targets in passing argument 3 of ‘yaml_sequence_start_event_initialize’ differ in signedness [-Wpointer-sign]
      22476 |       __pyx_t_2 = ((yaml_sequence_start_event_initialize((&__pyx_v_event), __pyx_v_anchor, __pyx_v_tag, __pyx_v_implicit, __pyx_v_sequence_style) == 0) != 0);
            |                                                                                            ^~~~~~~~~~~
            |                                                                                            |
            |                                                                                            char *
      In file included from ext/_ruamel_yaml.h:2,
                       from ext/_ruamel_yaml.c:523:
      ext/yaml.h:604:43: note: expected ‘yaml_char_t *’ {aka ‘unsigned char *’} but argument is of type ‘char *’
        604 |         yaml_char_t *anchor, yaml_char_t *tag, int implicit,
            |                              ~~~~~~~~~~~~~^~~
      ext/_ruamel_yaml.c:23006:75: warning: pointer targets in passing argument 2 of ‘yaml_mapping_start_event_initialize’ differ in signedness [-Wpointer-sign]
      23006 |       __pyx_t_2 = ((yaml_mapping_start_event_initialize((&__pyx_v_event), __pyx_v_anchor, __pyx_v_tag, __pyx_v_implicit, __pyx_v_mapping_style) == 0) != 0);
            |                                                                           ^~~~~~~~~~~~~~
            |                                                                           |
            |                                                                           char *
      In file included from ext/_ruamel_yaml.h:2,
                       from ext/_ruamel_yaml.c:523:
      ext/yaml.h:636:22: note: expected ‘yaml_char_t *’ {aka ‘unsigned char *’} but argument is of type ‘char *’
        636 |         yaml_char_t *anchor, yaml_char_t *tag, int implicit,
            |         ~~~~~~~~~~~~~^~~~~~
      ext/_ruamel_yaml.c:23006:91: warning: pointer targets in passing argument 3 of ‘yaml_mapping_start_event_initialize’ differ in signedness [-Wpointer-sign]
      23006 |       __pyx_t_2 = ((yaml_mapping_start_event_initialize((&__pyx_v_event), __pyx_v_anchor, __pyx_v_tag, __pyx_v_implicit, __pyx_v_mapping_style) == 0) != 0);
            |                                                                                           ^~~~~~~~~~~
            |                                                                                           |
            |                                                                                           char *
      In file included from ext/_ruamel_yaml.h:2,
                       from ext/_ruamel_yaml.c:523:
      ext/yaml.h:636:43: note: expected ‘yaml_char_t *’ {aka ‘unsigned char *’} but argument is of type ‘char *’
        636 |         yaml_char_t *anchor, yaml_char_t *tag, int implicit,
            |                              ~~~~~~~~~~~~~^~~
      ext/_ruamel_yaml.c: In function ‘PyInit__ruamel_yaml’:
      ext/_ruamel_yaml.c:25793:33: error: ‘PyTypeObject’ {aka ‘struct _typeobject’} has no member named ‘tp_print’
      25793 |   __pyx_type_12_ruamel_yaml_Mark.tp_print = 0;
            |                                 ^
      ext/_ruamel_yaml.c:25810:36: error: ‘PyTypeObject’ {aka ‘struct _typeobject’} has no member named ‘tp_print’
      25810 |   __pyx_type_12_ruamel_yaml_CParser.tp_print = 0;
            |                                    ^
      ext/_ruamel_yaml.c:25821:37: error: ‘PyTypeObject’ {aka ‘struct _typeobject’} has no member named ‘tp_print’
      25821 |   __pyx_type_12_ruamel_yaml_CEmitter.tp_print = 0;
            |                                     ^
      ext/_ruamel_yaml.c: In function ‘__Pyx_ParseOptionalKeywords’:
      ext/_ruamel_yaml.c:26908:21: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
      26908 |                     (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
            |                     ^
      In file included from /usr/include/python3.9/unicodeobject.h:1026,
                       from /usr/include/python3.9/Python.h:97,
                       from ext/_ruamel_yaml.c:4:
      /usr/include/python3.9/cpython/unicodeobject.h:446:26: note: declared here
        446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
            |                          ^~~~~~~~~~~~~~~~~~~~~~~~~~
      ext/_ruamel_yaml.c:26908:21: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations]
      26908 |                     (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
            |                     ^
      In file included from /usr/include/python3.9/unicodeobject.h:1026,
                       from /usr/include/python3.9/Python.h:97,
                       from ext/_ruamel_yaml.c:4:
      /usr/include/python3.9/cpython/unicodeobject.h:580:45: note: declared here
        580 | Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
            |                                             ^~~~~~~~~~~~~~~~~~~
      ext/_ruamel_yaml.c:26908:21: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
      26908 |                     (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
            |                     ^
      In file included from /usr/include/python3.9/unicodeobject.h:1026,
                       from /usr/include/python3.9/Python.h:97,
                       from ext/_ruamel_yaml.c:4:
      /usr/include/python3.9/cpython/unicodeobject.h:446:26: note: declared here
        446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
            |                          ^~~~~~~~~~~~~~~~~~~~~~~~~~
      ext/_ruamel_yaml.c:26908:21: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
      26908 |                     (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
            |                     ^
      In file included from /usr/include/python3.9/unicodeobject.h:1026,
                       from /usr/include/python3.9/Python.h:97,
                       from ext/_ruamel_yaml.c:4:
      /usr/include/python3.9/cpython/unicodeobject.h:446:26: note: declared here
        446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
            |                          ^~~~~~~~~~~~~~~~~~~~~~~~~~
      ext/_ruamel_yaml.c:26908:21: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations]
      26908 |                     (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
            |                     ^
      In file included from /usr/include/python3.9/unicodeobject.h:1026,
                       from /usr/include/python3.9/Python.h:97,
                       from ext/_ruamel_yaml.c:4:
      /usr/include/python3.9/cpython/unicodeobject.h:580:45: note: declared here
        580 | Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
            |                                             ^~~~~~~~~~~~~~~~~~~
      ext/_ruamel_yaml.c:26908:21: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
      26908 |                     (PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
            |                     ^
      In file included from /usr/include/python3.9/unicodeobject.h:1026,
                       from /usr/include/python3.9/Python.h:97,
                       from ext/_ruamel_yaml.c:4:
      /usr/include/python3.9/cpython/unicodeobject.h:446:26: note: declared here
        446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
            |                          ^~~~~~~~~~~~~~~~~~~~~~~~~~
      ext/_ruamel_yaml.c:26924:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
      26924 |                         (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
            |                         ^
      In file included from /usr/include/python3.9/unicodeobject.h:1026,
                       from /usr/include/python3.9/Python.h:97,
                       from ext/_ruamel_yaml.c:4:
      /usr/include/python3.9/cpython/unicodeobject.h:446:26: note: declared here
        446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
            |                          ^~~~~~~~~~~~~~~~~~~~~~~~~~
      ext/_ruamel_yaml.c:26924:25: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations]
      26924 |                         (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
            |                         ^
      In file included from /usr/include/python3.9/unicodeobject.h:1026,
                       from /usr/include/python3.9/Python.h:97,
                       from ext/_ruamel_yaml.c:4:
      /usr/include/python3.9/cpython/unicodeobject.h:580:45: note: declared here
        580 | Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
            |                                             ^~~~~~~~~~~~~~~~~~~
      ext/_ruamel_yaml.c:26924:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
      26924 |                         (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
            |                         ^
      In file included from /usr/include/python3.9/unicodeobject.h:1026,
                       from /usr/include/python3.9/Python.h:97,
                       from ext/_ruamel_yaml.c:4:
      /usr/include/python3.9/cpython/unicodeobject.h:446:26: note: declared here
        446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
            |                          ^~~~~~~~~~~~~~~~~~~~~~~~~~
      ext/_ruamel_yaml.c:26924:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
      26924 |                         (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
            |                         ^
      In file included from /usr/include/python3.9/unicodeobject.h:1026,
                       from /usr/include/python3.9/Python.h:97,
                       from ext/_ruamel_yaml.c:4:
      /usr/include/python3.9/cpython/unicodeobject.h:446:26: note: declared here
        446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
            |                          ^~~~~~~~~~~~~~~~~~~~~~~~~~
      ext/_ruamel_yaml.c:26924:25: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations]
      26924 |                         (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
            |                         ^
      In file included from /usr/include/python3.9/unicodeobject.h:1026,
                       from /usr/include/python3.9/Python.h:97,
                       from ext/_ruamel_yaml.c:4:
      /usr/include/python3.9/cpython/unicodeobject.h:580:45: note: declared here
        580 | Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
            |                                             ^~~~~~~~~~~~~~~~~~~
      ext/_ruamel_yaml.c:26924:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
      26924 |                         (PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
            |                         ^
      In file included from /usr/include/python3.9/unicodeobject.h:1026,
                       from /usr/include/python3.9/Python.h:97,
                       from ext/_ruamel_yaml.c:4:
      /usr/include/python3.9/cpython/unicodeobject.h:446:26: note: declared here
        446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
            |                          ^~~~~~~~~~~~~~~~~~~~~~~~~~
      error: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> ruamel.yaml

note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.

This happened after upgrading pip from 21.2.4 to 22.1.2.  With pip version 22.1.2 installation failed after more than 6 hours and the error message was 

ERROR: Exception:
Traceback (most recent call last):
  File "/local_disk0/.ephemeral_nfs/envs/pythonEnv-443996c2-41a0-4a83-813c-eea1c5b1d372/lib/python3.9/site-packages/pip/_internal/cli/base_command.py", line 173, in _main
    status = self.run(options, args)
  File "/local_disk0/.ephemeral_nfs/envs/pythonEnv-443996c2-41a0-4a83-813c-eea1c5b1d372/lib/python3.9/site-packages/pip/_internal/cli/req_command.py", line 203, in wrapper
    return func(self, options, args)
  File "/local_disk0/.ephemeral_nfs/envs/pythonEnv-443996c2-41a0-4a83-813c-eea1c5b1d372/lib/python3.9/site-packages/pip/_internal/commands/install.py", line 315, in run
    requirement_set = resolver.resolve(
  File "/local_disk0/.ephemeral_nfs/envs/pythonEnv-443996c2-41a0-4a83-813c-eea1c5b1d372/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 94, in resolve
    result = self._result = resolver.resolve(
  File "/local_disk0/.ephemeral_nfs/envs/pythonEnv-443996c2-41a0-4a83-813c-eea1c5b1d372/lib/python3.9/site-packages/pip/_vendor/resolvelib/resolvers.py", line 472, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
  File "/local_disk0/.ephemeral_nfs/envs/pythonEnv-443996c2-41a0-4a83-813c-eea1c5b1d372/lib/python3.9/site-packages/pip/_vendor/resolvelib/resolvers.py", line 383, in resolve
    raise ResolutionTooDeep(max_rounds)
pip._vendor.resolvelib.resolvers.ResolutionTooDeep: 2000000

Is this a know problem with a solution?  Is it necessary to specify a version of the sdk for installation to succeed?

[rwaginaire]

I had the same problem and I found out that it was because the Databricks runtime 11.0 ML is using Python 3.9. But azureml-sdk is only working for Python versions up to 3.8. So, I created a new cluster with Databricks Runtime 10.4 ML (Python 3.8) and that did the trick