search

datacontract / datacontract-cli

CLI to manage your datacontract.yaml files
https://cli.datacontract.com
Other
352 stars 60 forks source link

Installation on Databricks cluster takes approx. 8 min #262

Closed john-bunge-ed closed 2 weeks ago

john-bunge-ed commented 2 weeks ago

Currently pip install datacontract-cli==0.10.7 on a Databricks cluster takes around 8 min.

Cluster runtime is 14.3 LTS.

Here the console output:

Note: you may need to restart the kernel using dbutils.library.restartPython() to use updated packages.
Collecting datacontract-cli==0.10.7
  Using cached datacontract_cli-0.10.7-py3-none-any.whl (114 kB)
Collecting soda-core-duckdb<3.4.0,>=3.3.1
  Using cached soda_core_duckdb-3.3.5-py3-none-any.whl (7.5 kB)
Collecting typer[all]<0.13,>=0.9
  Using cached typer-0.12.3-py3-none-any.whl (47 kB)
Collecting pyyaml~=6.0.1
  Using cached PyYAML-6.0.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (705 kB)
Collecting opentelemetry-exporter-otlp-proto-http~=1.16
  Using cached opentelemetry_exporter_otlp_proto_http-1.25.0-py3-none-any.whl (16 kB)
Collecting jinja-partials>=0.2.1
  Using cached jinja_partials-0.2.1-py3-none-any.whl (4.7 kB)
Collecting avro==1.11.3
  Using cached avro-1.11.3-py2.py3-none-any.whl
Collecting requests<2.33,>=2.31
  Using cached requests-2.32.3-py3-none-any.whl (64 kB)
Collecting rdflib==7.0.0
  Using cached rdflib-7.0.0-py3-none-any.whl (531 kB)
Collecting soda-core-sqlserver<3.4.0,>=3.3.1
  Using cached soda_core_sqlserver-3.3.5-py3-none-any.whl (9.4 kB)
Collecting soda-core-spark-df<3.4.0,>=3.3.1
  Using cached soda_core_spark_df-3.3.5-py3-none-any.whl (7.3 kB)
Collecting fastapi==0.111.0
  Using cached fastapi-0.111.0-py3-none-any.whl (91 kB)
Collecting python-dotenv~=1.0.0
  Using cached python_dotenv-1.0.1-py3-none-any.whl (19 kB)
Collecting duckdb==0.10.3
  Using cached duckdb-0.10.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (18.5 MB)
Collecting snowflake-connector-python[pandas]<3.11,>=3.6
  Using cached snowflake_connector_python-3.10.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.5 MB)
Collecting soda-core-bigquery<3.4.0,>=3.3.1
  Using cached soda_core_bigquery-3.3.5-py3-none-any.whl (8.0 kB)
Requirement already satisfied: setuptools>=60 in /databricks/python3/lib/python3.10/site-packages (from datacontract-cli==0.10.7) (65.6.3)
Requirement already satisfied: fastjsonschema~=2.19.1 in /databricks/python3/lib/python3.10/site-packages (from datacontract-cli==0.10.7) (2.19.1)
Collecting python-multipart==0.0.9
  Using cached python_multipart-0.0.9-py3-none-any.whl (22 kB)
Collecting s3fs==2024.5.0
  Using cached s3fs-2024.5.0-py3-none-any.whl (29 kB)
Collecting databricks-sql-connector<3.2.0,>=3.1.2
  Using cached databricks_sql_connector-3.1.2-py3-none-any.whl (427 kB)
Collecting opentelemetry-exporter-otlp-proto-grpc~=1.16
  Using cached opentelemetry_exporter_otlp_proto_grpc-1.25.0-py3-none-any.whl (18 kB)
Collecting rich~=13.7.0
  Using cached rich-13.7.1-py3-none-any.whl (240 kB)
Collecting deltalake~=0.17.0
  Using cached deltalake-0.17.4-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (26.5 MB)
Collecting pydantic<2.8.0,>=2.5.3
  Using cached pydantic-2.7.4-py3-none-any.whl (409 kB)
Collecting soda-core-spark[databricks]<3.4.0,>=3.3.1
  Using cached soda_core_spark-3.3.5-py3-none-any.whl (9.8 kB)
Collecting boto3<1.34.114,>=1.34.41
  Using cached boto3-1.34.113-py3-none-any.whl (139 kB)
Collecting soda-core-postgres<3.4.0,>=3.3.1
  Using cached soda_core_postgres-3.3.5-py3-none-any.whl (9.4 kB)
Collecting soda-core-snowflake<3.4.0,>=3.3.1
  Using cached soda_core_snowflake-3.3.5-py3-none-any.whl (7.9 kB)
Collecting botocore<1.34.114,>=1.34.41
  Using cached botocore-1.34.113-py3-none-any.whl (12.3 MB)
Collecting fastparquet==2024.5.0
  Using cached fastparquet-2024.5.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.7 MB)
Collecting simple-ddl-parser==1.5.1
  Using cached simple_ddl_parser-1.5.1-py3-none-any.whl (93 kB)
Collecting typing-extensions>=4.8.0
  Using cached typing_extensions-4.12.2-py3-none-any.whl (37 kB)
Requirement already satisfied: ujson!=4.0.2,!=4.1.0,!=4.2.0,!=4.3.0,!=5.0.0,!=5.1.0,>=4.0.1 in /databricks/python3/lib/python3.10/site-packages (from fastapi==0.111.0->datacontract-cli==0.10.7) (5.4.0)
Collecting uvicorn[standard]>=0.12.0
  Using cached uvicorn-0.30.1-py3-none-any.whl (62 kB)
Collecting email_validator>=2.0.0
  Using cached email_validator-2.1.1-py3-none-any.whl (30 kB)
Collecting fastapi-cli>=0.0.2
  Using cached fastapi_cli-0.0.4-py3-none-any.whl (9.5 kB)
Collecting starlette<0.38.0,>=0.37.2
  Using cached starlette-0.37.2-py3-none-any.whl (71 kB)
Requirement already satisfied: jinja2>=2.11.2 in /databricks/python3/lib/python3.10/site-packages (from fastapi==0.111.0->datacontract-cli==0.10.7) (3.1.2)
Collecting httpx>=0.23.0
  Using cached httpx-0.27.0-py3-none-any.whl (75 kB)
Collecting orjson>=3.2.1
  Using cached orjson-3.10.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (142 kB)
Requirement already satisfied: packaging in /databricks/python3/lib/python3.10/site-packages (from fastparquet==2024.5.0->datacontract-cli==0.10.7) (23.2)
Collecting cramjam>=2.3
  Using cached cramjam-2.8.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.0 MB)
Requirement already satisfied: pandas>=1.5.0 in /databricks/python3/lib/python3.10/site-packages (from fastparquet==2024.5.0->datacontract-cli==0.10.7) (1.5.3)
Requirement already satisfied: numpy in /databricks/python3/lib/python3.10/site-packages (from fastparquet==2024.5.0->datacontract-cli==0.10.7) (1.23.5)
Collecting fsspec
  Using cached fsspec-2024.6.0-py3-none-any.whl (176 kB)
Collecting isodate<0.7.0,>=0.6.0
  Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Requirement already satisfied: pyparsing<4,>=2.1.0 in /databricks/python3/lib/python3.10/site-packages (from rdflib==7.0.0->datacontract-cli==0.10.7) (3.0.9)
Collecting fsspec
  Using cached fsspec-2024.5.0-py3-none-any.whl (316 kB)
Collecting aiobotocore<3.0.0,>=2.5.4
  Using cached aiobotocore-2.13.0-py3-none-any.whl (76 kB)
Collecting aiohttp!=4.0.0a0,!=4.0.0a1
  Using cached aiohttp-3.9.5-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.2 MB)
Collecting ply<4.0,>=3.11
  Using cached ply-3.11-py2.py3-none-any.whl (49 kB)
Requirement already satisfied: jmespath<2.0.0,>=0.7.1 in /databricks/python3/lib/python3.10/site-packages (from boto3<1.34.114,>=1.34.41->datacontract-cli==0.10.7) (0.10.0)
Collecting s3transfer<0.11.0,>=0.10.0
  Using cached s3transfer-0.10.1-py3-none-any.whl (82 kB)
Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in /databricks/python3/lib/python3.10/site-packages (from botocore<1.34.114,>=1.34.41->datacontract-cli==0.10.7) (2.8.2)
Requirement already satisfied: urllib3!=2.2.0,<3,>=1.25.4 in /databricks/python3/lib/python3.10/site-packages (from botocore<1.34.114,>=1.34.41->datacontract-cli==0.10.7) (1.26.14)
Collecting openpyxl<4.0.0,>=3.0.10
  Using cached openpyxl-3.1.4-py2.py3-none-any.whl (251 kB)
Requirement already satisfied: oauthlib<4.0.0,>=3.1.0 in /usr/lib/python3/dist-packages (from databricks-sql-connector<3.2.0,>=3.1.2->datacontract-cli==0.10.7) (3.2.0)
Collecting pyarrow<15.0.0,>=14.0.1
  Using cached pyarrow-14.0.2-cp310-cp310-manylinux_2_28_x86_64.whl (38.0 MB)
Collecting lz4<5.0.0,>=4.0.2
  Using cached lz4-4.3.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)
Collecting thrift<0.17.0,>=0.16.0
  Using cached thrift-0.16.0-cp310-cp310-linux_x86_64.whl
Requirement already satisfied: pyarrow-hotfix in /databricks/python3/lib/python3.10/site-packages (from deltalake~=0.17.0->datacontract-cli==0.10.7) (0.5)
Collecting deprecated>=1.2.6
  Using cached Deprecated-1.2.14-py2.py3-none-any.whl (9.6 kB)
Collecting opentelemetry-exporter-otlp-proto-common==1.25.0
  Using cached opentelemetry_exporter_otlp_proto_common-1.25.0-py3-none-any.whl (17 kB)
Requirement already satisfied: googleapis-common-protos~=1.52 in /databricks/python3/lib/python3.10/site-packages (from opentelemetry-exporter-otlp-proto-grpc~=1.16->datacontract-cli==0.10.7) (1.62.0)
Collecting opentelemetry-sdk~=1.25.0
  Using cached opentelemetry_sdk-1.25.0-py3-none-any.whl (107 kB)
Collecting opentelemetry-api~=1.15
  Using cached opentelemetry_api-1.25.0-py3-none-any.whl (59 kB)
Requirement already satisfied: grpcio<2.0.0,>=1.0.0 in /databricks/python3/lib/python3.10/site-packages (from opentelemetry-exporter-otlp-proto-grpc~=1.16->datacontract-cli==0.10.7) (1.48.2)
Collecting opentelemetry-proto==1.25.0
  Using cached opentelemetry_proto-1.25.0-py3-none-any.whl (52 kB)
Requirement already satisfied: protobuf<5.0,>=3.19 in /databricks/python3/lib/python3.10/site-packages (from opentelemetry-proto==1.25.0->opentelemetry-exporter-otlp-proto-grpc~=1.16->datacontract-cli==0.10.7) (4.24.0)
Collecting pydantic-core==2.18.4
  Using cached pydantic_core-2.18.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.0 MB)
Collecting annotated-types>=0.4.0
  Using cached annotated_types-0.7.0-py3-none-any.whl (13 kB)
Requirement already satisfied: idna<4,>=2.5 in /databricks/python3/lib/python3.10/site-packages (from requests<2.33,>=2.31->datacontract-cli==0.10.7) (3.4)
Requirement already satisfied: certifi>=2017.4.17 in /databricks/python3/lib/python3.10/site-packages (from requests<2.33,>=2.31->datacontract-cli==0.10.7) (2022.12.7)
Requirement already satisfied: charset-normalizer<4,>=2 in /databricks/python3/lib/python3.10/site-packages (from requests<2.33,>=2.31->datacontract-cli==0.10.7) (2.0.4)
Collecting pygments<3.0.0,>=2.13.0
  Using cached pygments-2.18.0-py3-none-any.whl (1.2 MB)
Collecting markdown-it-py>=2.2.0
  Using cached markdown_it_py-3.0.0-py3-none-any.whl (87 kB)
Requirement already satisfied: pytz in /databricks/python3/lib/python3.10/site-packages (from snowflake-connector-python[pandas]<3.11,>=3.6->datacontract-cli==0.10.7) (2022.7)
Collecting platformdirs<5.0.0,>=2.6.0
  Using cached platformdirs-4.2.2-py3-none-any.whl (18 kB)
Requirement already satisfied: cffi<2.0.0,>=1.9 in /databricks/python3/lib/python3.10/site-packages (from snowflake-connector-python[pandas]<3.11,>=3.6->datacontract-cli==0.10.7) (1.15.1)
Requirement already satisfied: cryptography<43.0.0,>=3.1.0 in /databricks/python3/lib/python3.10/site-packages (from snowflake-connector-python[pandas]<3.11,>=3.6->datacontract-cli==0.10.7) (39.0.1)
Collecting asn1crypto<2.0.0,>0.24.0
  Using cached asn1crypto-1.5.1-py2.py3-none-any.whl (105 kB)
Requirement already satisfied: pyjwt<3.0.0 in /usr/lib/python3/dist-packages (from snowflake-connector-python[pandas]<3.11,>=3.6->datacontract-cli==0.10.7) (2.3.0)
Requirement already satisfied: filelock<4,>=3.5 in /usr/local/lib/python3.10/dist-packages (from snowflake-connector-python[pandas]<3.11,>=3.6->datacontract-cli==0.10.7) (3.13.1)
Collecting pyOpenSSL<25.0.0,>=16.2.0
  Using cached pyOpenSSL-24.1.0-py3-none-any.whl (56 kB)
Collecting sortedcontainers>=2.4.0
  Using cached sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting tomlkit
  Using cached tomlkit-0.12.5-py3-none-any.whl (37 kB)
Collecting soda-core==3.3.5
  Using cached soda_core-3.3.5-py3-none-any.whl (193 kB)
Collecting google-cloud-bigquery<4.0,>=2.25.0
  Using cached google_cloud_bigquery-3.24.0-py2.py3-none-any.whl (238 kB)
Collecting sqlparse~=0.4
  Using cached sqlparse-0.5.0-py3-none-any.whl (43 kB)
Collecting opentelemetry-api~=1.15
  Using cached opentelemetry_api-1.22.0-py3-none-any.whl (57 kB)
Collecting opentelemetry-exporter-otlp-proto-http~=1.16
  Using cached opentelemetry_exporter_otlp_proto_http-1.22.0-py3-none-any.whl (16 kB)
Collecting antlr4-python3-runtime~=4.11.1
  Using cached antlr4_python3_runtime-4.11.1-py3-none-any.whl (144 kB)
Requirement already satisfied: markupsafe<=2.1.2,>=2.0.1 in /databricks/python3/lib/python3.10/site-packages (from soda-core==3.3.5->soda-core-bigquery<3.4.0,>=3.3.1->datacontract-cli==0.10.7) (2.1.1)
Collecting ruamel.yaml<0.18.0,>=0.17.0
  Using cached ruamel.yaml-0.17.40-py3-none-any.whl (113 kB)
Collecting inflect~=7.0
  Using cached inflect-7.2.1-py3-none-any.whl (34 kB)
Requirement already satisfied: click~=8.0 in /databricks/python3/lib/python3.10/site-packages (from soda-core==3.3.5->soda-core-bigquery<3.4.0,>=3.3.1->datacontract-cli==0.10.7) (8.0.4)
Collecting backoff<3.0.0,>=1.10.0
  Using cached backoff-2.2.1-py3-none-any.whl (15 kB)
Collecting opentelemetry-exporter-otlp-proto-http~=1.16
  Using cached opentelemetry_exporter_otlp_proto_http-1.21.0-py3-none-any.whl (16 kB)
  Using cached opentelemetry_exporter_otlp_proto_http-1.20.0-py3-none-any.whl (16 kB)
  Using cached opentelemetry_exporter_otlp_proto_http-1.19.0-py3-none-any.whl (17 kB)
  Using cached opentelemetry_exporter_otlp_proto_http-1.18.0-py3-none-any.whl (17 kB)
  Using cached opentelemetry_exporter_otlp_proto_http-1.17.0-py3-none-any.whl (21 kB)
  Using cached opentelemetry_exporter_otlp_proto_http-1.16.0-py3-none-any.whl (21 kB)
INFO: pip is looking at multiple versions of soda-core to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of soda-core-bigquery to determine which version is compatible with other requirements. This could take a while.
Collecting soda-core-bigquery<3.4.0,>=3.3.1
  Using cached soda_core_bigquery-3.3.4-py3-none-any.whl (8.0 kB)
Collecting opentelemetry-semantic-conventions==0.46b0
  Using cached opentelemetry_semantic_conventions-0.46b0-py3-none-any.whl (130 kB)
Collecting importlib-metadata<=7.1,>=6.0
  Using cached importlib_metadata-7.1.0-py3-none-any.whl (24 kB)
Collecting soda-core==3.3.4
  Using cached soda_core-3.3.4-py3-none-any.whl (193 kB)
Collecting soda-core-bigquery<3.4.0,>=3.3.1
  Using cached soda_core_bigquery-3.3.3-py3-none-any.whl (8.0 kB)
Collecting soda-core==3.3.3
  Using cached soda_core-3.3.3-py3-none-any.whl (193 kB)
Collecting soda-core-bigquery<3.4.0,>=3.3.1
  Using cached soda_core_bigquery-3.3.2-py3-none-any.whl (8.0 kB)
Collecting soda-core==3.3.2
  Using cached soda_core-3.3.2-py3-none-any.whl (193 kB)
Collecting soda-core-bigquery<3.4.0,>=3.3.1
  Using cached soda_core_bigquery-3.3.1-py3-none-any.whl (8.0 kB)
Collecting soda-core==3.3.1
  Using cached soda_core-3.3.1-py3-none-any.whl (193 kB)
INFO: pip is looking at multiple versions of opentelemetry-semantic-conventions to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of opentelemetry-api to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of opentelemetry-sdk to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of snowflake-connector-python[pandas] to determine which version is compatible with other requirements. This could take a while.
Collecting snowflake-connector-python[pandas]<3.11,>=3.6
  Using cached snowflake_connector_python-3.10.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.5 MB)
INFO: pip is looking at multiple versions of soda-core-bigquery to determine which version is compatible with other requirements. This could take a while.
  Using cached snowflake_connector_python-3.9.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.6 MB)
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. See https://pip.pypa.io/warnings/backtracking for guidance. If you want to abort this run, press Ctrl + C.
  Using cached snowflake_connector_python-3.9.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.6 MB)
  Using cached snowflake_connector_python-3.8.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.6 MB)
  Using cached snowflake_connector_python-3.7.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.6 MB)
Collecting platformdirs<4.0.0,>=2.6.0
  Using cached platformdirs-3.11.0-py3-none-any.whl (17 kB)
Collecting snowflake-connector-python[pandas]<3.11,>=3.6
  Using cached snowflake_connector_python-3.7.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.4 MB)
Collecting pyOpenSSL<24.0.0,>=16.2.0
  Using cached pyOpenSSL-23.3.0-py3-none-any.whl (58 kB)
Collecting snowflake-connector-python[pandas]<3.11,>=3.6
  Using cached snowflake_connector_python-3.6.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.5 MB)
INFO: pip is looking at multiple versions of snowflake-connector-python[pandas] to determine which version is compatible with other requirements. This could take a while.
Collecting opentelemetry-api~=1.15
  Using cached opentelemetry_api-1.24.0-py3-none-any.whl (60 kB)
Collecting importlib-metadata<=7.0,>=6.0
  Using cached importlib_metadata-7.0.0-py3-none-any.whl (23 kB)
Collecting opentelemetry-api~=1.15
  Using cached opentelemetry_api-1.23.0-py3-none-any.whl (58 kB)
Collecting importlib-metadata<7.0,>=6.0
  Using cached importlib_metadata-6.11.0-py3-none-any.whl (23 kB)
Collecting opentelemetry-api~=1.15
  Using cached opentelemetry_api-1.21.0-py3-none-any.whl (57 kB)
  Using cached opentelemetry_api-1.20.0-py3-none-any.whl (57 kB)
  Using cached opentelemetry_api-1.19.0-py3-none-any.whl (57 kB)
INFO: pip is looking at multiple versions of opentelemetry-api to determine which version is compatible with other requirements. This could take a while.
  Using cached opentelemetry_api-1.18.0-py3-none-any.whl (57 kB)
  Using cached opentelemetry_api-1.17.0-py3-none-any.whl (57 kB)
  Using cached opentelemetry_api-1.16.0-py3-none-any.whl (57 kB)
  Using cached opentelemetry_api-1.15.0-py3-none-any.whl (56 kB)
INFO: pip is looking at multiple versions of rich to determine which version is compatible with other requirements. This could take a while.
Collecting rich~=13.7.0
  Using cached rich-13.7.0-py3-none-any.whl (240 kB)
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. See https://pip.pypa.io/warnings/backtracking for guidance. If you want to abort this run, press Ctrl + C.
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. See https://pip.pypa.io/warnings/backtracking for guidance. If you want to abort this run, press Ctrl + C.
INFO: pip is looking at multiple versions of requests to determine which version is compatible with other requirements. This could take a while.
Collecting requests<2.33,>=2.31
  Using cached requests-2.32.2-py3-none-any.whl (63 kB)
  Using cached requests-2.31.0-py3-none-any.whl (62 kB)
INFO: pip is looking at multiple versions of pyyaml to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of python-dotenv to determine which version is compatible with other requirements. This could take a while.
Collecting python-dotenv~=1.0.0
  Using cached python_dotenv-1.0.0-py3-none-any.whl (19 kB)
INFO: pip is looking at multiple versions of rich to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of pydantic-core to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of pydantic to determine which version is compatible with other requirements. This could take a while.
Collecting pydantic<2.8.0,>=2.5.3
  Using cached pydantic-2.7.3-py3-none-any.whl (409 kB)
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. See https://pip.pypa.io/warnings/backtracking for guidance. If you want to abort this run, press Ctrl + C.
INFO: pip is looking at multiple versions of requests to determine which version is compatible with other requirements. This could take a while.
  Using cached pydantic-2.7.2-py3-none-any.whl (409 kB)
Collecting pydantic-core==2.18.3
  Using cached pydantic_core-2.18.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.0 MB)
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. See https://pip.pypa.io/warnings/backtracking for guidance. If you want to abort this run, press Ctrl + C.
Collecting pydantic<2.8.0,>=2.5.3
  Using cached pydantic-2.7.1-py3-none-any.whl (409 kB)
Collecting pydantic-core==2.18.2
  Using cached pydantic_core-2.18.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.1 MB)
INFO: pip is looking at multiple versions of pyyaml to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of python-dotenv to determine which version is compatible with other requirements. This could take a while.
Collecting pydantic<2.8.0,>=2.5.3
  Using cached pydantic-2.7.0-py3-none-any.whl (407 kB)
Collecting pydantic-core==2.18.1
  Using cached pydantic_core-2.18.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.1 MB)
Collecting pydantic<2.8.0,>=2.5.3
  Using cached pydantic-2.6.4-py3-none-any.whl (394 kB)
Collecting pydantic-core==2.16.3
  Using cached pydantic_core-2.16.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.2 MB)
Collecting pydantic<2.8.0,>=2.5.3
  Using cached pydantic-2.6.3-py3-none-any.whl (395 kB)
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. See https://pip.pypa.io/warnings/backtracking for guidance. If you want to abort this run, press Ctrl + C.
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. See https://pip.pypa.io/warnings/backtracking for guidance. If you want to abort this run, press Ctrl + C.
  Using cached pydantic-2.6.2-py3-none-any.whl (394 kB)
INFO: pip is looking at multiple versions of pydantic-core to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of pydantic to determine which version is compatible with other requirements. This could take a while.
  Using cached pydantic-2.6.1-py3-none-any.whl (394 kB)
Collecting pydantic-core==2.16.2
  Using cached pydantic_core-2.16.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.2 MB)
Collecting pydantic<2.8.0,>=2.5.3
  Using cached pydantic-2.6.0-py3-none-any.whl (394 kB)
Collecting pydantic-core==2.16.1
  Using cached pydantic_core-2.16.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.2 MB)
Collecting pydantic<2.8.0,>=2.5.3
  Using cached pydantic-2.5.3-py3-none-any.whl (381 kB)
Collecting pydantic-core==2.14.6
  Using cached pydantic_core-2.14.6-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.1 MB)
INFO: pip is looking at multiple versions of opentelemetry-exporter-otlp-proto-http to determine which version is compatible with other requirements. This could take a while.
Collecting opentelemetry-exporter-otlp-proto-http~=1.16
  Using cached opentelemetry_exporter_otlp_proto_http-1.24.0-py3-none-any.whl (16 kB)
  Using cached opentelemetry_exporter_otlp_proto_http-1.23.0-py3-none-any.whl (16 kB)
INFO: pip is looking at multiple versions of opentelemetry-proto to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of opentelemetry-exporter-otlp-proto-common to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of opentelemetry-exporter-otlp-proto-grpc to determine which version is compatible with other requirements. This could take a while.
Collecting opentelemetry-exporter-otlp-proto-grpc~=1.16
  Using cached opentelemetry_exporter_otlp_proto_grpc-1.24.0-py3-none-any.whl (18 kB)
Collecting opentelemetry-sdk~=1.24.0
  Using cached opentelemetry_sdk-1.24.0-py3-none-any.whl (106 kB)
Collecting opentelemetry-proto==1.24.0
  Using cached opentelemetry_proto-1.24.0-py3-none-any.whl (50 kB)
Collecting opentelemetry-exporter-otlp-proto-common==1.24.0
  Using cached opentelemetry_exporter_otlp_proto_common-1.24.0-py3-none-any.whl (17 kB)
Collecting opentelemetry-semantic-conventions==0.45b0
  Using cached opentelemetry_semantic_conventions-0.45b0-py3-none-any.whl (36 kB)
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. See https://pip.pypa.io/warnings/backtracking for guidance. If you want to abort this run, press Ctrl + C.
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. See https://pip.pypa.io/warnings/backtracking for guidance. If you want to abort this run, press Ctrl + C.
Collecting opentelemetry-exporter-otlp-proto-grpc~=1.16
  Using cached opentelemetry_exporter_otlp_proto_grpc-1.23.0-py3-none-any.whl (18 kB)
Collecting opentelemetry-sdk~=1.23.0
  Using cached opentelemetry_sdk-1.23.0-py3-none-any.whl (105 kB)
Collecting opentelemetry-proto==1.23.0
  Using cached opentelemetry_proto-1.23.0-py3-none-any.whl (50 kB)
Collecting opentelemetry-exporter-otlp-proto-common==1.23.0
  Using cached opentelemetry_exporter_otlp_proto_common-1.23.0-py3-none-any.whl (17 kB)
Collecting opentelemetry-semantic-conventions==0.44b0
  Using cached opentelemetry_semantic_conventions-0.44b0-py3-none-any.whl (36 kB)
INFO: pip is looking at multiple versions of opentelemetry-semantic-conventions to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of opentelemetry-sdk to determine which version is compatible with other requirements. This could take a while.
Collecting opentelemetry-exporter-otlp-proto-grpc~=1.16
  Using cached opentelemetry_exporter_otlp_proto_grpc-1.22.0-py3-none-any.whl (18 kB)
Collecting opentelemetry-proto==1.22.0
  Using cached opentelemetry_proto-1.22.0-py3-none-any.whl (50 kB)
Collecting opentelemetry-sdk~=1.22.0
  Using cached opentelemetry_sdk-1.22.0-py3-none-any.whl (105 kB)
Collecting opentelemetry-exporter-otlp-proto-common==1.22.0
  Using cached opentelemetry_exporter_otlp_proto_common-1.22.0-py3-none-any.whl (17 kB)
Collecting opentelemetry-semantic-conventions==0.43b0
  Using cached opentelemetry_semantic_conventions-0.43b0-py3-none-any.whl (36 kB)
Collecting psycopg2-binary<3.0,>=2.8.5
  Using cached psycopg2_binary-2.9.9-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
Collecting pyspark
  Using cached pyspark-3.5.1-py2.py3-none-any.whl
Requirement already satisfied: pyodbc in /databricks/python3/lib/python3.10/site-packages (from soda-core-sqlserver<3.4.0,>=3.3.1->datacontract-cli==0.10.7) (4.0.32)
WARNING: typer 0.12.3 does not provide the extra 'all'
Collecting shellingham>=1.3.0
  Using cached shellingham-1.5.4-py2.py3-none-any.whl (9.8 kB)
Collecting aiobotocore<3.0.0,>=2.5.4
  Using cached aiobotocore-2.12.4-py3-none-any.whl (76 kB)
  Using cached aiobotocore-2.12.3-py3-none-any.whl (76 kB)
Collecting wrapt<2.0.0,>=1.10.10
  Using cached wrapt-1.16.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (80 kB)
Collecting aioitertools<1.0.0,>=0.5.1
  Using cached aioitertools-0.11.0-py3-none-any.whl (23 kB)
Collecting aiobotocore<3.0.0,>=2.5.4
  Using cached aiobotocore-2.12.2-py3-none-any.whl (76 kB)
  Using cached aiobotocore-2.12.1-py3-none-any.whl (76 kB)
  Using cached aiobotocore-2.12.0-py3-none-any.whl (76 kB)
  Using cached aiobotocore-2.11.2-py3-none-any.whl (76 kB)
  Using cached aiobotocore-2.11.1-py3-none-any.whl (76 kB)
  Using cached aiobotocore-2.11.0-py3-none-any.whl (76 kB)
  Using cached aiobotocore-2.10.0-py3-none-any.whl (75 kB)
  Using cached aiobotocore-2.9.1-py3-none-any.whl (75 kB)
  Using cached aiobotocore-2.9.0-py3-none-any.whl (75 kB)
  Using cached aiobotocore-2.8.0-py3-none-any.whl (75 kB)
  Using cached aiobotocore-2.7.0-py3-none-any.whl (73 kB)
  Using cached aiobotocore-2.6.0-py3-none-any.whl (73 kB)
  Using cached aiobotocore-2.5.4-py3-none-any.whl (73 kB)
INFO: pip is looking at multiple versions of typer[all] to determine which version is compatible with other requirements. This could take a while.
Collecting typer[all]<0.13,>=0.9
  Using cached typer-0.12.2-py3-none-any.whl (47 kB)
INFO: pip is looking at multiple versions of soda-core-sqlserver to determine which version is compatible with other requirements. This could take a while.
Collecting soda-core-sqlserver<3.4.0,>=3.3.1
  Using cached soda_core_sqlserver-3.3.4-py3-none-any.whl (9.4 kB)
INFO: pip is looking at multiple versions of soda-core-spark[databricks] to determine which version is compatible with other requirements. This could take a while.
Collecting soda-core-spark[databricks]<3.4.0,>=3.3.1
  Using cached soda_core_spark-3.3.4-py3-none-any.whl (9.8 kB)
INFO: pip is looking at multiple versions of soda-core-spark to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of soda-core-spark-df to determine which version is compatible with other requirements. This could take a while.
Collecting soda-core-spark-df<3.4.0,>=3.3.1
  Using cached soda_core_spark_df-3.3.4-py3-none-any.whl (7.3 kB)
INFO: pip is looking at multiple versions of soda-core-snowflake to determine which version is compatible with other requirements. This could take a while.
Collecting soda-core-snowflake<3.4.0,>=3.3.1
  Using cached soda_core_snowflake-3.3.4-py3-none-any.whl (7.9 kB)
INFO: pip is looking at multiple versions of soda-core-postgres to determine which version is compatible with other requirements. This could take a while.
Collecting soda-core-postgres<3.4.0,>=3.3.1
  Using cached soda_core_postgres-3.3.4-py3-none-any.whl (9.4 kB)
INFO: pip is looking at multiple versions of soda-core-duckdb to determine which version is compatible with other requirements. This could take a while.
Collecting soda-core-duckdb<3.4.0,>=3.3.1
  Using cached soda_core_duckdb-3.3.4-py3-none-any.whl (7.5 kB)
Collecting opentelemetry-exporter-otlp-proto-grpc~=1.16
  Using cached opentelemetry_exporter_otlp_proto_grpc-1.21.0-py3-none-any.whl (18 kB)
INFO: pip is looking at multiple versions of jinja-partials to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of fastjsonschema to determine which version is compatible with other requirements. This could take a while.
Collecting fastjsonschema~=2.19.1
  Using cached fastjsonschema-2.19.1-py3-none-any.whl (23 kB)
INFO: pip is looking at multiple versions of deltalake to determine which version is compatible with other requirements. This could take a while.
Collecting deltalake~=0.17.0
  Using cached deltalake-0.17.3-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (26.4 MB)
INFO: pip is looking at multiple versions of databricks-sql-connector to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of botocore to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of boto3 to determine which version is compatible with other requirements. This could take a while.
Collecting boto3<1.34.114,>=1.34.41
  Using cached boto3-1.34.112-py3-none-any.whl (139 kB)
Collecting botocore<1.34.114,>=1.34.41
  Using cached botocore-1.34.112-py3-none-any.whl (12.3 MB)
Collecting boto3<1.34.114,>=1.34.41
  Using cached boto3-1.34.111-py3-none-any.whl (139 kB)
Collecting botocore<1.34.114,>=1.34.41
  Using cached botocore-1.34.111-py3-none-any.whl (12.3 MB)
Collecting boto3<1.34.114,>=1.34.41
  Using cached boto3-1.34.110-py3-none-any.whl (139 kB)
INFO: pip is looking at multiple versions of botocore to determine which version is compatible with other requirements. This could take a while.
Collecting botocore<1.34.114,>=1.34.41
  Using cached botocore-1.34.110-py3-none-any.whl (12.3 MB)
Collecting boto3<1.34.114,>=1.34.41
  Using cached boto3-1.34.109-py3-none-any.whl (139 kB)
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. See https://pip.pypa.io/warnings/backtracking for guidance. If you want to abort this run, press Ctrl + C.
Collecting botocore<1.34.114,>=1.34.41
  Using cached botocore-1.34.109-py3-none-any.whl (12.2 MB)
Collecting boto3<1.34.114,>=1.34.41
  Using cached boto3-1.34.108-py3-none-any.whl (139 kB)
Collecting botocore<1.34.114,>=1.34.41
  Using cached botocore-1.34.108-py3-none-any.whl (12.2 MB)
Collecting boto3<1.34.114,>=1.34.41
  Using cached boto3-1.34.107-py3-none-any.whl (139 kB)
Collecting botocore<1.34.114,>=1.34.41
  Using cached botocore-1.34.107-py3-none-any.whl (12.2 MB)
Collecting boto3<1.34.114,>=1.34.41
  Using cached boto3-1.34.106-py3-none-any.whl (139 kB)
Collecting botocore<1.34.114,>=1.34.41
  Using cached botocore-1.34.106-py3-none-any.whl (12.2 MB)
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. See https://pip.pypa.io/warnings/backtracking for guidance. If you want to abort this run, press Ctrl + C.
INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. See https://pip.pypa.io/warnings/backtracking for guidance. If you want to abort this run, press Ctrl + C.
WARNING: typer 0.12.3 does not provide the extra 'all'
Collecting yarl<2.0,>=1.0
  Using cached yarl-1.9.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (301 kB)
Collecting async-timeout<5.0,>=4.0
  Using cached async_timeout-4.0.3-py3-none-any.whl (5.7 kB)
Collecting aiosignal>=1.1.2
  Using cached aiosignal-1.3.1-py3-none-any.whl (7.6 kB)
Collecting frozenlist>=1.1.1
  Using cached frozenlist-1.4.1-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (239 kB)
Requirement already satisfied: attrs>=17.3.0 in /databricks/python3/lib/python3.10/site-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->s3fs==2024.5.0->datacontract-cli==0.10.7) (22.1.0)
Collecting multidict<7.0,>=4.5
  Using cached multidict-6.0.5-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (124 kB)
Requirement already satisfied: pycparser in /databricks/python3/lib/python3.10/site-packages (from cffi<2.0.0,>=1.9->snowflake-connector-python[pandas]<3.11,>=3.6->datacontract-cli==0.10.7) (2.21)
Collecting dnspython>=2.0.0
  Using cached dnspython-2.6.1-py3-none-any.whl (307 kB)
Collecting google-cloud-core<3.0.0dev,>=1.6.0
  Using cached google_cloud_core-2.4.1-py2.py3-none-any.whl (29 kB)
Collecting google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.10.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,!=2.8.*,!=2.9.*,<3.0.0dev,>=1.34.1
  Using cached google_api_core-2.19.0-py3-none-any.whl (139 kB)
Collecting google-resumable-media<3.0dev,>=0.6.0
  Using cached google_resumable_media-2.7.1-py2.py3-none-any.whl (81 kB)
Collecting google-auth<3.0.0dev,>=2.14.1
  Using cached google_auth-2.30.0-py2.py3-none-any.whl (193 kB)
Requirement already satisfied: six>=1.5.2 in /usr/lib/python3/dist-packages (from grpcio<2.0.0,>=1.0.0->opentelemetry-exporter-otlp-proto-grpc~=1.16->datacontract-cli==0.10.7) (1.16.0)
Collecting httpcore==1.*
  Using cached httpcore-1.0.5-py3-none-any.whl (77 kB)
Requirement already satisfied: sniffio in /databricks/python3/lib/python3.10/site-packages (from httpx>=0.23.0->fastapi==0.111.0->datacontract-cli==0.10.7) (1.2.0)
Requirement already satisfied: anyio in /databricks/python3/lib/python3.10/site-packages (from httpx>=0.23.0->fastapi==0.111.0->datacontract-cli==0.10.7) (3.5.0)
Collecting h11<0.15,>=0.13
  Using cached h11-0.14.0-py3-none-any.whl (58 kB)
Collecting mdurl~=0.1
  Using cached mdurl-0.1.2-py3-none-any.whl (10.0 kB)
Collecting et-xmlfile
  Using cached et_xmlfile-1.1.0-py3-none-any.whl (4.7 kB)
Collecting cryptography<43.0.0,>=3.1.0
  Using cached cryptography-42.0.8-cp39-abi3-manylinux_2_28_x86_64.whl (3.9 MB)
Collecting watchfiles>=0.13
  Using cached watchfiles-0.22.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.2 MB)
Collecting uvloop!=0.15.0,!=0.15.1,>=0.14.0
  Using cached uvloop-0.19.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.4 MB)
Collecting websockets>=10.4
  Using cached websockets-12.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (130 kB)
Collecting httptools>=0.5.0
  Using cached httptools-0.6.1-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (341 kB)
Collecting py4j==0.10.9.7
  Using cached py4j-0.10.9.7-py2.py3-none-any.whl (200 kB)
Collecting proto-plus<2.0.0dev,>=1.22.3
  Using cached proto_plus-1.23.0-py3-none-any.whl (48 kB)
Requirement already satisfied: grpcio-status<2.0.dev0,>=1.33.2 in /databricks/python3/lib/python3.10/site-packages (from google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.10.*,!=2.2.*,!=2.3.*,!=2.4.*,!=2.5.*,!=2.6.*,!=2.7.*,!=2.8.*,!=2.9.*,<3.0.0dev,>=1.34.1->google-cloud-bigquery<4.0,>=2.25.0->soda-core-bigquery<3.4.0,>=3.3.1->datacontract-cli==0.10.7) (1.48.1)
Collecting rsa<5,>=3.1.4
  Using cached rsa-4.9-py3-none-any.whl (34 kB)
Collecting pyasn1-modules>=0.2.1
  Using cached pyasn1_modules-0.4.0-py3-none-any.whl (181 kB)
Collecting cachetools<6.0,>=2.0.0
  Using cached cachetools-5.3.3-py3-none-any.whl (9.3 kB)
Collecting google-crc32c<2.0dev,>=1.0
  Using cached google_crc32c-1.5.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)
Requirement already satisfied: zipp>=0.5 in /usr/lib/python3/dist-packages (from importlib-metadata<7.0,>=6.0->opentelemetry-api~=1.15->opentelemetry-exporter-otlp-proto-grpc~=1.16->datacontract-cli==0.10.7) (1.0.0)
Requirement already satisfied: more-itertools in /usr/lib/python3/dist-packages (from inflect~=7.0->soda-core==3.3.5->soda-core-bigquery<3.4.0,>=3.3.1->datacontract-cli==0.10.7) (8.10.0)
Collecting typeguard>=4.0.1
  Using cached typeguard-4.3.0-py3-none-any.whl (35 kB)
Collecting ruamel.yaml.clib>=0.2.7
  Using cached ruamel.yaml.clib-0.2.8-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (526 kB)
Collecting pyasn1<0.7.0,>=0.4.6
  Using cached pyasn1-0.6.0-py2.py3-none-any.whl (85 kB)
Installing collected packages: sortedcontainers, py4j, ply, asn1crypto, antlr4-python3-runtime, wrapt, websockets, uvloop, typing-extensions, tomlkit, thrift, sqlparse, simple-ddl-parser, shellingham, ruamel.yaml.clib, requests, pyyaml, python-multipart, python-dotenv, pyspark, pygments, pyasn1, pyarrow, psycopg2-binary, proto-plus, platformdirs, orjson, opentelemetry-semantic-conventions, opentelemetry-proto, multidict, mdurl, lz4, isodate, importlib-metadata, httptools, h11, google-crc32c, fsspec, frozenlist, et-xmlfile, duckdb, dnspython, cramjam, cachetools, backoff, avro, async-timeout, annotated-types, aioitertools, yarl, watchfiles, uvicorn, typeguard, starlette, ruamel.yaml, rsa, rdflib, pydantic-core, pyasn1-modules, opentelemetry-exporter-otlp-proto-common, openpyxl, markdown-it-py, jinja-partials, httpcore, google-resumable-media, email_validator, deprecated, deltalake, cryptography, botocore, aiosignal, s3transfer, rich, pyOpenSSL, pydantic, opentelemetry-api, inflect, httpx, google-auth, fastparquet, databricks-sql-connector, aiohttp, typer, snowflake-connector-python, opentelemetry-sdk, google-api-core, boto3, aiobotocore, s3fs, opentelemetry-exporter-otlp-proto-http, opentelemetry-exporter-otlp-proto-grpc, google-cloud-core, fastapi-cli, soda-core, google-cloud-bigquery, fastapi, soda-core-sqlserver, soda-core-spark, soda-core-snowflake, soda-core-postgres, soda-core-duckdb, soda-core-bigquery, soda-core-spark-df, datacontract-cli
  Attempting uninstall: typing-extensions
    Found existing installation: typing_extensions 4.4.0
    Not uninstalling typing-extensions at /databricks/python3/lib/python3.10/site-packages, outside environment /local_disk0/.ephemeral_nfs/envs/pythonEnv-2ee81d6d-b276-48a2-b298-01d7ad6caaf8
    Can't uninstall 'typing_extensions'. No files were found to uninstall.
  Attempting uninstall: requests
    Found existing installation: requests 2.28.1
    Not uninstalling requests at /databricks/python3/lib/python3.10/site-packages, outside environment /local_disk0/.ephemeral_nfs/envs/pythonEnv-2ee81d6d-b276-48a2-b298-01d7ad6caaf8
    Can't uninstall 'requests'. No files were found to uninstall.
  Attempting uninstall: pygments
    Found existing installation: Pygments 2.11.2
    Not uninstalling pygments at /databricks/python3/lib/python3.10/site-packages, outside environment /local_disk0/.ephemeral_nfs/envs/pythonEnv-2ee81d6d-b276-48a2-b298-01d7ad6caaf8
    Can't uninstall 'Pygments'. No files were found to uninstall.
  Attempting uninstall: pyarrow
    Found existing installation: pyarrow 8.0.0
    Not uninstalling pyarrow at /databricks/python3/lib/python3.10/site-packages, outside environment /local_disk0/.ephemeral_nfs/envs/pythonEnv-2ee81d6d-b276-48a2-b298-01d7ad6caaf8
    Can't uninstall 'pyarrow'. No files were found to uninstall.
  Attempting uninstall: platformdirs
    Found existing installation: platformdirs 2.5.2
    Not uninstalling platformdirs at /databricks/python3/lib/python3.10/site-packages, outside environment /local_disk0/.ephemeral_nfs/envs/pythonEnv-2ee81d6d-b276-48a2-b298-01d7ad6caaf8
    Can't uninstall 'platformdirs'. No files were found to uninstall.
  Attempting uninstall: importlib-metadata
    Found existing installation: importlib-metadata 4.6.4
    Not uninstalling importlib-metadata at /usr/lib/python3/dist-packages, outside environment /local_disk0/.ephemeral_nfs/envs/pythonEnv-2ee81d6d-b276-48a2-b298-01d7ad6caaf8
    Can't uninstall 'importlib-metadata'. No files were found to uninstall.
  Attempting uninstall: cryptography
    Found existing installation: cryptography 39.0.1
    Not uninstalling cryptography at /databricks/python3/lib/python3.10/site-packages, outside environment /local_disk0/.ephemeral_nfs/envs/pythonEnv-2ee81d6d-b276-48a2-b298-01d7ad6caaf8
    Can't uninstall 'cryptography'. No files were found to uninstall.
  Attempting uninstall: botocore
    Found existing installation: botocore 1.27.96
    Not uninstalling botocore at /databricks/python3/lib/python3.10/site-packages, outside environment /local_disk0/.ephemeral_nfs/envs/pythonEnv-2ee81d6d-b276-48a2-b298-01d7ad6caaf8
    Can't uninstall 'botocore'. No files were found to uninstall.
  Attempting uninstall: s3transfer
    Found existing installation: s3transfer 0.6.2
    Not uninstalling s3transfer at /databricks/python3/lib/python3.10/site-packages, outside environment /local_disk0/.ephemeral_nfs/envs/pythonEnv-2ee81d6d-b276-48a2-b298-01d7ad6caaf8
    Can't uninstall 's3transfer'. No files were found to uninstall.
  Attempting uninstall: pydantic
    Found existing installation: pydantic 1.10.6
    Not uninstalling pydantic at /databricks/python3/lib/python3.10/site-packages, outside environment /local_disk0/.ephemeral_nfs/envs/pythonEnv-2ee81d6d-b276-48a2-b298-01d7ad6caaf8
    Can't uninstall 'pydantic'. No files were found to uninstall.
  Attempting uninstall: boto3
    Found existing installation: boto3 1.24.28
    Not uninstalling boto3 at /databricks/python3/lib/python3.10/site-packages, outside environment /local_disk0/.ephemeral_nfs/envs/pythonEnv-2ee81d6d-b276-48a2-b298-01d7ad6caaf8
    Can't uninstall 'boto3'. No files were found to uninstall.
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
databricks-sdk 0.1.6 requires requests<2.29.0,>=2.28.1, but you have requests 2.32.3 which is incompatible.
virtualenv 20.16.7 requires platformdirs<3,>=2.4, but you have platformdirs 4.2.2 which is incompatible.
Successfully installed aiobotocore-2.13.0 aiohttp-3.9.5 aioitertools-0.11.0 aiosignal-1.3.1 annotated-types-0.7.0 antlr4-python3-runtime-4.11.1 asn1crypto-1.5.1 async-timeout-4.0.3 avro-1.11.3 backoff-2.2.1 boto3-1.34.106 botocore-1.34.106 cachetools-5.3.3 cramjam-2.8.3 cryptography-42.0.8 databricks-sql-connector-3.1.2 datacontract-cli-0.10.7 deltalake-0.17.4 deprecated-1.2.14 dnspython-2.6.1 duckdb-0.10.3 email_validator-2.1.1 et-xmlfile-1.1.0 fastapi-0.111.0 fastapi-cli-0.0.4 fastparquet-2024.5.0 frozenlist-1.4.1 fsspec-2024.5.0 google-api-core-2.19.0 google-auth-2.30.0 google-cloud-bigquery-3.24.0 google-cloud-core-2.4.1 google-crc32c-1.5.0 google-resumable-media-2.7.1 h11-0.14.0 httpcore-1.0.5 httptools-0.6.1 httpx-0.27.0 importlib-metadata-6.11.0 inflect-7.2.1 isodate-0.6.1 jinja-partials-0.2.1 lz4-4.3.3 markdown-it-py-3.0.0 mdurl-0.1.2 multidict-6.0.5 openpyxl-3.1.4 opentelemetry-api-1.22.0 opentelemetry-exporter-otlp-proto-common-1.22.0 opentelemetry-exporter-otlp-proto-grpc-1.22.0 opentelemetry-exporter-otlp-proto-http-1.22.0 opentelemetry-proto-1.22.0 opentelemetry-sdk-1.22.0 opentelemetry-semantic-conventions-0.43b0 orjson-3.10.4 platformdirs-4.2.2 ply-3.11 proto-plus-1.23.0 psycopg2-binary-2.9.9 py4j-0.10.9.7 pyOpenSSL-24.1.0 pyarrow-14.0.2 pyasn1-0.6.0 pyasn1-modules-0.4.0 pydantic-2.7.4 pydantic-core-2.18.4 pygments-2.18.0 pyspark-3.5.1 python-dotenv-1.0.1 python-multipart-0.0.9 pyyaml-6.0.1 rdflib-7.0.0 requests-2.32.3 rich-13.7.1 rsa-4.9 ruamel.yaml-0.17.40 ruamel.yaml.clib-0.2.8 s3fs-2024.5.0 s3transfer-0.10.1 shellingham-1.5.4 simple-ddl-parser-1.5.1 snowflake-connector-python-3.10.1 soda-core-3.3.5 soda-core-bigquery-3.3.5 soda-core-duckdb-3.3.5 soda-core-postgres-3.3.5 soda-core-snowflake-3.3.5 soda-core-spark-3.3.5 soda-core-spark-df-3.3.5 soda-core-sqlserver-3.3.5 sortedcontainers-2.4.0 sqlparse-0.5.0 starlette-0.37.2 thrift-0.16.0 tomlkit-0.12.5 typeguard-4.3.0 typer-0.12.3 typing-extensions-4.12.2 uvicorn-0.30.1 uvloop-0.19.0 watchfiles-0.22.0 websockets-12.0 wrapt-1.16.0 yarl-1.9.4
Note: you may need to restart the kernel using dbutils.library.restartPython() to use updated packages.
john-bunge-ed commented 2 weeks ago

As a comparison, pip install datacontract-cli==0.10.3 on a Databricks cluster takes only 1 min 30 sec.

jochenchrist commented 2 weeks ago

213 should help here.

Also, I'll try to strict the dependencies.

john-bunge-ed commented 2 weeks ago

Thank you. I will test installation with optional dependencies.

I also realized that installation on a Databricks cluster with runtime 15.2 (which uses Python 3.11) only takes 1min 30sec.

jochenchrist commented 2 weeks ago

BTW, on Databricks, you might want to consider to install the library directly in your cluster resource:

image
john-bunge-ed commented 2 weeks ago

BTW, on Databricks, you might want to consider to install the library directly in your cluster resource:

Thanks for pointing that out. In fact this part does not make any difference re installation duration.