Iceberg Connector Ingestion Issue with pydantic V2

[Prajwal214]

Affected module Does it impact the UI, backend or Ingestion Framework? -- Ingestion Framework

Describe the bug A clear and concise description of what the bug is. -- While running Metadata Ingestion using Iceberg connector using Rest Catalog , we are getting below error in the logs.

pyiceberg==0.7.1

in <module>
    from pyiceberg.catalog.glue import GlueCatalog
  File "/home/airflow/.local/lib/python3.10/site-packages/pyiceberg/catalog/glue.py", line 31, in <module>
    from mypy_boto3_glue.client import GlueClient
ModuleNotFoundError: No module named 'mypy_boto3_glue'

pyiceberg==0.5.1

[2024-09-26 12:24:33] INFO     {metadata.OMetaAPI:server_mixin:74} - OpenMetadata client running with Server version [1.5.5] and Client version [1.5.5.0]
[2024-09-26 12:24:33] ERROR    {metadata.Utils:logger:178} - 
Error initializing metadata: 'pydantic_core._pydantic_core.Url' object has no attribute 'endswith'

To Reproduce

While running Metadata Ingestion we can replicate the issue

Expected behavior A clear and concise description of what you expected to happen.

Version:

• OS: [e.g. iOS]
• Python version: 3.10
• OpenMetadata version: [e.g. 0.8] OM v.1.5.5
• OpenMetadata Ingestion package version: [e.g. openmetadata-ingestion[docker]==XYZ] -- openmetadata-ingestion[iceberg]==1.5.5.0

Additional context Add any other context about the problem here.

[Prajwal214]

Closing by #18016