search

aws / sagemaker-core

Apache License 2.0
8 stars 4 forks source link

Pydantic issue #169

Open niklas-palm opened 2 months ago

niklas-palm commented 2 months ago

Pydantic 2.7 seems to be required.

When using SageMaker distribution 1.9, I get the following error installing sagemaker-core

!pip install --upgrade pip -q
!pip install sagemaker-core

->
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
gluonts 0.13.7 requires pydantic~=1.7, but you have pydantic 2.8.2 which is incompatible.
sagemaker-jupyterlab-extension-common 0.1.18 requires pydantic==1.*, but you have pydantic 2.8.2 which is incompatible.

Creating a training job, I get: TypeError: list_schema() got an unexpected keyword argument 'fail_fast'

It seems I must first uninstall pydantic, then re-install pydantic==2.7.0 and then sagemaker-core, to get it working.

niklas-palm commented 2 months ago

A work-around is to uninstall everything and install pydantic==2.7.0 first, then sagemaker-core:

!pip uninstall pydantic sagemaker-core -y
!pip install --upgrade pip -q
!pip install pydantic==2.7.0
!pip install sagemaker-core
A-yush commented 2 months ago

The issue is not resolved yet.

Getting same error when trying to create FeatureDefinition and in pyspark processing.

Code 1

from sagemaker_core.shapes import ProcessingInput,ProcessingResources,AppSpecification,ProcessingS3Input
from sagemaker_core.shapes import ProcessingResources,ProcessingClusterConfig

processing_input = ProcessingInput(input_name="code",s3_input = [ProcessingS3Input(
                                        s3_uri="s3://sagemaker-us-east-1-774297356213/sm-spark-2024-08-30-05-25-18-294/input/code/",s3_data_type="S3Prefix")])

Code 2:

from sagemaker_core.shapes import FeatureDefinition
CustomerFeatureDefinitions = [FeatureDefinition(feature_name='customer_id',feature_type='Integral'),
                            FeatureDefinition(feature_name='city_code',feature_type='Integral'),
                              FeatureDefinition(feature_name='state_code',feature_type='Integral'), 
                              FeatureDefinition(feature_name='country_code',feature_type='Integral'),
                              FeatureDefinition(feature_name='EventTime',feature_type='Fractional')]

Error:

TypeError: list_schema() got an unexpected keyword argument 'fail_fast'

Sagemaker core version: 1.0.2