search

GoogleCloudPlatform / mlops-with-vertex-ai

An end-to-end example of MLOps on Google Cloud using TensorFlow, TFX, and Vertex AI
Apache License 2.0
341 stars 117 forks source link

02-Experimentation: Create classifier fails w/layer requires matching shapes #10

Closed jth1911 closed 2 years ago

jth1911 commented 2 years ago

ValueError: A Concatenate layer requires inputs with matching shapes except for the concat axis. Got inputs shapes: [(None, 2), (None, 4), (7,), (None, 3), (None, 1), (None, 1), (6,), (None, 3), (None, 3), (None, 1), (None, 10)]

Input Features dropoff_grid_xf <dtype: 'int64'>: [0, 0, 0] euclidean_xf <dtype: 'float32'>: [0.669279932975769, -0.8318284749984741, -0.8318284749984741] loc_cross_xf <dtype: 'int64'>: [0, 0, 0] payment_type_xf <dtype: 'int64'>: [2, 0, 0] pickup_grid_xf <dtype: 'int64'>: [0, 0, 0] trip_day_of_week_xf <dtype: 'int64'>: [0, 6, 2] trip_day_xf <dtype: 'int64'>: [8, 30, 1] trip_hour_xf <dtype: 'int64'>: [1, 13, 6] trip_miles_xf <dtype: 'float32'>: [2.3255326747894287, -0.22459185123443604, -0.4029441475868225] trip_month_xf <dtype: 'int64'>: [3, 1, 0] trip_seconds_xf <dtype: 'float32'>: [0.9550504088401794, -0.2630620300769806, -0.24356801807880402] target: [0, 0, 0]

ksalama commented 2 years ago

Could you please double-check that you are using TensorFlow version 2.5 and TFX version 1.2?

immch commented 2 years ago

@ksalama using TFX version 1.2 doesn't produce the above error, but this version causes dependency conflicts when installing packages from requirements.txt

requirements.txt

kfp==1.8.1
google-cloud-bigquery==2.26.0
google-cloud-bigquery-storage==2.7.0
google-cloud-aiplatform==1.4.2
cloudml-hypertune==0.1.0.dev6
pytest

Seems that tfx=1.2.0 conflicts with google-cloud-bigquery==2.26.0 and google-cloud-aiplatform==1.4.2

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
(...)
tfx 1.2.0 requires google-cloud-aiplatform<0.8,>=0.5.0, but you have google-cloud-aiplatform 1.4.2 which is incompatible.
tfx 1.2.0 requires google-cloud-bigquery<2.21,>=1.28.0, but you have google-cloud-bigquery 2.26.0 which is incompatible.
tfx-bsl 1.2.0 requires google-cloud-bigquery<2.21,>=1.28.0, but you have google-cloud-bigquery 2.26.0 which is incompatible.
tensorflow 2.5.2 requires grpcio~=1.34.0, but you have grpcio 1.41.1 which is incompatible.
tensorflow-transform 1.2.0 requires google-cloud-bigquery<2.21,>=1.28.0, but you have google-cloud-bigquery 2.26.0 which is incompatible.
tensorflow-model-analysis 0.33.0 requires google-cloud-bigquery<2.21,>=1.28.0, but you have google-cloud-bigquery 2.26.0 which is incompatible.
tensorflow-io 0.15.0 requires tensorflow<2.4.0,>=2.3.0, but you have tensorflow 2.5.2 which is incompatible.
tensorflow-data-validation 1.2.0 requires google-cloud-bigquery<2.21,>=1.28.0, but you have google-cloud-bigquery 2.26.0 which is incompatible.
fairness-indicators 0.26.0 requires tensorflow-data-validation<0.27,>=0.26, but you have tensorflow-data-validation 1.2.0 which is incompatible.
fairness-indicators 0.26.0 requires tensorflow-model-analysis<0.27,>=0.26, but you have tensorflow-model-analysis 0.33.0 which is incompatible.
Successfully installed google-cloud-aiplatform-1.4.2 google-cloud-bigquery-2.26.0 grpcio-1.41.1
ksalama commented 2 years ago

I think it is safe to ignore these dependencies issues.