Adding support for MetaFrame

[matt035343]

I would like to add support for Polars without removing support for Pandas. Adding this MetaFrame module is an attempt to generalize our APIs. For now only pandas and poalrs are supported. More libraries can easily be added.

TODO:

• [x] Base implementation of MetaFrame
• [x] Refactor deltalake operations
• [x] Refactor ODBC connector
• [x] Refactor Astra connector
• [x] Refactor Trino connector
• [x] Refactor Snowflake connector
• [x] Refactor QES
• [x] Add MetaFrame codumentation
• [x] Ensure all other documentation is up to date after refactorings

[github-actions[bot]]

Coverage Report

File	Stmts	Miss	Cover	Missing
adapta/connectors/service_bus
__init__.py	1	1	0%	18
_connector.py	17	17	0%	19–61
adapta/logs
_async_logger.py	81	3	96%	55, 78–79
_base.py	55	6	89%	35, 41, 44, 94–97
_internal_logger.py	109	6	94%	276–286
adapta/logs/handlers
datadog_api_handler.py	109	31	72%	89, 106–113, 124, 136–152, 161–195, 204, 210, 238
adapta/metrics/providers
datadog_provider.py	43	43	0%	19–147
adapta/ml
__init__.py	1	1	0%	19
_model.py	10	10	0%	17–42
adapta/ml/mlflow
__init__.py	2	2	0%	17–18
_client.py	42	42	0%	19–142
_functions.py	42	42	0%	17–111
adapta/process_communication
_models.py	33	6	82%	90–96
adapta/security/clients
__init__.py	26	12	54%	27–28, 34–35, 41–42, 48–49, 53–54, 58–59
_azure_client.py	72	49	32%	42, 55–64, 74–77, 80, 83–85, 94–152, 155, 158–196
adapta/security/clients/aws
_aws_client.py	37	16	57%	37–39, 46, 56, 62, 74, 99–107, 113–116
_aws_credentials.py	73	30	59%	60–79, 83, 87, 91, 95, 99, 108–112, 116, 120, 124, 128, 132
adapta/security/clients/hashicorp_vault
hashicorp_vault_client.py	30	4	87%	46, 86, 90, 94
kubernetes_client.py	21	5	76%	45–48, 67–68
oidc_client.py	45	20	56%	33–62, 80–83, 92
token_client.py	17	7	59%	42–45, 52–53, 56, 59
adapta/storage/blob
azure_storage_client.py	120	52	57%	71–78, 88, 95–96, 99–105, 127–128, 131–157, 160, 176–189, 197–201, 220, 241–243, 252, 268–272, 282–284, 287–306, 313
local_storage_client.py	45	5	89%	39, 64, 90, 93, 96
s3_storage_client.py	119	84	29%	56–66, 75–88, 97–102, 123–131, 139–140, 154–171, 187–191, 209–218, 228–249, 260–281
adapta/storage/cache
redis_cache.py	37	37	0%	19–107
adapta/storage/database/v2
azure_sql.py	34	21	38%	57–66, 78, 92–130, 140
odbc.py	73	54	26%	62–78, 81–111, 114–115, 121, 131–135, 145–159, 179–219
snowflake_sql.py	69	51	26%	51–64, 71–85, 100–102, 113–122, 127–149, 177–228
trino_sql.py	39	23	41%	66–98, 101–111, 114–115, 127
adapta/storage/database/v3
azure_sql.py	32	20	38%	55, 70, 84–122, 132
odbc.py	72	11	85%	94–104, 114, 125–126, 140, 149–155, 182
snowflake_sql.py	67	19	72%	63–77, 92–94, 105–114, 141
trino_sql.py	38	38	0%	20–119
adapta/storage/delta_lake/v2
_functions.py	63	40	37%	71–99, 146–161, 203–287
adapta/storage/delta_lake/v3
_functions.py	61	11	82%	148, 153, 155, 157, 213, 223, 233–243, 249, 284
adapta/storage/distributed_object_store/v2/datastax_astra
_models.py	19	9	53%	54–59, 62, 65, 68
astra_client.py	196	140	29%	40–43, 113–137, 143–184, 190–191, 197–198, 201, 211, 223–224, 233, 273–361, 369, 393–520, 530, 541–556, 579–594, 616–632, 656–673
adapta/storage/distributed_object_store/v3/datastax_astra
_models.py	19	9	53%	53–58, 61, 64, 67
astra_client.py	194	118	39%	38–41, 137–178, 184–185, 191–192, 195, 205, 217–218, 227, 271–362, 370, 396–523, 533, 544–559, 582–597, 619–635, 659–676
adapta/storage/models
astra.py	35	10	71%	37, 40, 44, 59–61, 64, 70–73
aws.py	36	11	69%	36–39, 46–49, 57–59, 82, 91
azure.py	60	17	72%	32, 36–40, 47, 67, 70–71, 74–75, 89–93, 100, 113, 120, 123–124, 127
filter_expression.py	125	5	96%	55, 183–184, 238, 329
hive.py	57	25	56%	36, 40, 43, 91–98, 110, 113–114, 123–163, 166
local.py	21	4	81%	31, 35, 38, 50
adapta/storage/query_enabled_store
_models.py	61	10	84%	76, 138–141, 147–150, 156–157, 163
_qes_astra.py	57	13	77%	65–66, 78, 83–96, 106–109
_qes_delta.py	32	4	88%	28, 50, 62, 70
adapta/storage/secrets
azure_secret_client.py	20	20	0%	19–66
adapta/utils
_common.py	95	15	84%	36–37, 70–83, 94, 122, 142, 162, 245
concurrent_task_runner.py	27	1	96%	109
adapta/utils/data_structures
_functions.py	34	1	97%	134
adapta/utils/decorators
_logging.py	41	1	98%	32
_rate_limit.py	25	1	96%	58
adapta/utils/python_typing
_functions.py	11	1	91%	24
tests
test_filtering_api.py	32	2	94%	197–198
test_utils.py	166	1	99%	371
test_vault_client.py	80	18	78%	33–35, 40–42, 47–51, 56–57, 62–66
TOTAL	4276	1255	71%

Tests	Skipped	Failures	Errors	Time
198	5 :zzz:	0 :x:	0 :fire:	1m 0s :stopwatch:

[github-actions[bot]]

Coverage Report

File	Stmts	Miss	Cover	Missing
adapta/connectors/service_bus
__init__.py	1	1	0%	18
_connector.py	17	17	0%	19–61
adapta/logs
_async_logger.py	81	3	96%	55, 78–79
_base.py	55	6	89%	35, 41, 44, 94–97
_internal_logger.py	109	6	94%	276–286
adapta/logs/handlers
datadog_api_handler.py	109	31	72%	89, 106–113, 124, 136–152, 161–195, 204, 210, 238
adapta/metrics/providers
datadog_provider.py	43	43	0%	19–147
adapta/ml
__init__.py	1	1	0%	19
_model.py	10	10	0%	17–42
adapta/ml/mlflow
__init__.py	2	2	0%	17–18
_client.py	42	42	0%	19–142
_functions.py	42	42	0%	17–111
adapta/process_communication
_models.py	33	6	82%	90–96
adapta/security/clients
__init__.py	26	12	54%	27–28, 34–35, 41–42, 48–49, 53–54, 58–59
_azure_client.py	72	49	32%	42, 55–64, 74–77, 80, 83–85, 94–152, 155, 158–196
adapta/security/clients/aws
_aws_client.py	37	16	57%	37–39, 46, 56, 62, 74, 99–107, 113–116
_aws_credentials.py	73	30	59%	60–79, 83, 87, 91, 95, 99, 108–112, 116, 120, 124, 128, 132
adapta/security/clients/hashicorp_vault
hashicorp_vault_client.py	30	4	87%	46, 86, 90, 94
kubernetes_client.py	21	5	76%	45–48, 67–68
oidc_client.py	45	20	56%	33–62, 80–83, 92
token_client.py	17	7	59%	42–45, 52–53, 56, 59
adapta/storage/blob
azure_storage_client.py	119	51	57%	71–78, 88, 95–96, 99–105, 127–128, 131–157, 160, 176–189, 197–201, 220, 241–243, 252, 268–272, 282–284, 287–306, 313
local_storage_client.py	45	5	89%	39, 64, 90, 93, 96
s3_storage_client.py	119	84	29%	56–66, 75–88, 97–102, 123–131, 139–140, 154–171, 187–191, 209–218, 228–249, 260–281
adapta/storage/cache
redis_cache.py	37	37	0%	19–107
adapta/storage/database/v2
azure_sql.py	34	21	38%	57–66, 78, 92–130, 140
odbc.py	73	54	26%	62–78, 81–111, 114–115, 121, 131–135, 145–159, 179–219
snowflake_sql.py	69	51	26%	51–64, 71–85, 100–102, 113–122, 127–149, 177–228
trino_sql.py	39	23	41%	66–98, 101–111, 114–115, 127
adapta/storage/database/v3
azure_sql.py	32	20	38%	55, 70, 84–122, 132
odbc.py	72	11	85%	94–104, 114, 125–126, 140, 149–155, 182
snowflake_sql.py	67	19	72%	63–77, 92–94, 105–114, 141
trino_sql.py	38	38	0%	20–119
adapta/storage/delta_lake/v2
_functions.py	63	40	37%	71–99, 146–161, 203–287
adapta/storage/delta_lake/v3
_functions.py	61	11	82%	148, 153, 155, 157, 213, 223, 233–243, 249, 284
adapta/storage/distributed_object_store/v2/datastax_astra
_models.py	19	9	53%	54–59, 62, 65, 68
astra_client.py	196	140	29%	40–43, 113–137, 143–184, 190–191, 197–198, 201, 211, 223–224, 233, 273–361, 369, 393–520, 530, 541–556, 579–594, 616–632, 656–673
adapta/storage/distributed_object_store/v3/datastax_astra
_models.py	19	9	53%	53–58, 61, 64, 67
astra_client.py	194	118	39%	38–41, 137–178, 184–185, 191–192, 195, 205, 217–218, 227, 271–362, 370, 396–523, 533, 544–559, 582–597, 619–635, 659–676
adapta/storage/models
astra.py	35	10	71%	37, 40, 44, 59–61, 64, 70–73
aws.py	36	11	69%	36–39, 46–49, 57–59, 82, 91
azure.py	60	17	72%	32, 36–40, 47, 67, 70–71, 74–75, 89–93, 100, 113, 120, 123–124, 127
filter_expression.py	125	5	96%	55, 183–184, 238, 329
hive.py	57	25	56%	36, 40, 43, 91–98, 110, 113–114, 123–163, 166
local.py	21	4	81%	31, 35, 38, 50
adapta/storage/query_enabled_store
_models.py	61	10	84%	76, 138–141, 147–150, 156–157, 163
_qes_astra.py	57	13	77%	65–66, 78, 83–96, 106–109
_qes_delta.py	32	4	88%	28, 50, 62, 70
adapta/storage/secrets
azure_secret_client.py	20	20	0%	19–66
adapta/utils
_common.py	95	15	84%	36–37, 70–83, 94, 122, 142, 162, 245
concurrent_task_runner.py	27	1	96%	109
adapta/utils/data_structures
_functions.py	34	1	97%	134
adapta/utils/decorators
_logging.py	41	1	98%	32
_rate_limit.py	25	1	96%	58
adapta/utils/python_typing
_functions.py	11	3	73%	7–9, 22
tests
test_filtering_api.py	32	2	94%	197–198
test_python_typing_functions.py	11	2	82%	38–39
test_utils.py	166	1	99%	371
test_vault_client.py	80	18	78%	33–35, 40–42, 47–51, 56–57, 62–66
TOTAL	4274	1258	71%

Tests	Skipped	Failures	Errors	Time
198	6 :zzz:	0 :x:	0 :fire:	1m 1s :stopwatch:

[matt035343]

create_package

[github-actions[bot]]

Created package with number v2.11.6a440.dev3

[matt035343]

create_package

[matt035343]

create_package

[github-actions[bot]]

Created package with number v2.11.7a440.dev6

[matt035343]

create_package

[github-actions[bot]]

Created package with number v2.11.8a440.dev8

[matt035343]

Need to throw deprecation warnings