search

tensorflow / tensorflow

An Open Source Machine Learning Framework for Everyone
https://tensorflow.org
Apache License 2.0
185.45k stars 74.17k forks source link

Tensorflow incompatible shapes. #73301

Closed TheWalkingSea closed 2 weeks ago

TheWalkingSea commented 1 month ago

Issue type

Support

Have you reproduced the bug with TensorFlow Nightly?

Yes

Source

source

TensorFlow version

2.17.0

Custom code

Yes

OS platform and distribution

Windows

Mobile device

No response

Python version

3.11.4

Bazel version

No response

GCC/compiler version

No response

CUDA/cuDNN version

No response

GPU model and memory

No response

Current behavior?

I am using a dictionary to pass data into model.fit with the names of the input layers corresponding in the dictionary with data. However, for some reason, tensorflow is mixing up the data in category and amzData but if you look at the output from input({layer: tf.convert_to_tensor(X_train[layer].to_list()).numpy().shape for layer in layer_names}) it shows that the data is correctly labelled. So why is tensorflow mixing up these two variables despite it being correctly correlated in the dictionary?

Standalone code to reproduce the issue

import numpy as np
import tensorflow as tf
import keras
import pandas as pd
from sklearn.preprocessing import MinMaxScaler
from sklearn.model_selection import train_test_split
from keras.api.utils import pad_sequences
import logging
import sys
import json

keras.config.disable_traceback_filtering()

root = logging.getLogger()
root.setLevel(logging.INFO)
logFormatter = logging.Formatter(fmt="%(asctime)s-%(name)s-%(levelname)s:%(message)s", 
    datefmt="%X")
fileHandler = logging.FileHandler(filename="./data/logs/scraper.log")
fileHandler.setFormatter(logFormatter)
root.addHandler(fileHandler)
root.addHandler(logging.StreamHandler(sys.stdout))

def create_ts_model(*args, **kwargs) -> tuple[keras.Layer, keras.Layer]:
    input = keras.layers.Input(shape=(None, 2), dtype=tf.float32, *args, **kwargs)
    x = keras.layers.LSTM(16, return_sequences=True)(input)
    x = keras.layers.Dropout(0.2)(x)
    x = keras.layers.LSTM(8, return_sequences=True)(x)
    x = keras.layers.Dropout(0.2)(x)
    x = keras.layers.GlobalAveragePooling1D()(x)

    return input, x

def get_compiled_model(catvocab: list) -> keras.Model:
    logging.info("Checkpoint not found. Compiling new model...")

    # Category
    category_feature = keras.Input(shape=(1,), dtype=tf.string, name="category")

    strlookup = keras.layers.StringLookup(output_mode='int')
    strlookup.adapt(catvocab)
    layer = strlookup(category_feature)

    embedding_dim = 8
    # Add 1 to include [OOV]
    category_output = keras.layers.Embedding(input_dim=len(catvocab) + 1, output_dim=embedding_dim)(layer)

    # buyBoxIsFBA & buyBoxIsAmazon
    bbinfo_feature = keras.Input(shape=(2,), dtype=tf.uint8, name="bbinfo")

    # All numeric features
    numinput_feature = keras.Input(shape=(10,), dtype=tf.float32, name="numeric_inputs")

    # Time series data
    fba_feature, fba_ts_output = create_ts_model(name="fbaData")
    amz_feature, amz_ts_output = create_ts_model(name="amzData")
    fbm_feature, fbm_ts_output = create_ts_model(name="fbmData")
    bb_feature, bb_ts_output = create_ts_model(name="buyBoxData")
    monthly_sold_feature, monthly_sold_ts_output = create_ts_model(name="monthlySoldData")
    sales_rank_feature, sales_rank_ts_output = create_ts_model(name="salesRankData")
    offer_count_feature, offer_count_ts_output = create_ts_model(name="offerCountData")
    review_count_feature, review_count_ts_output = create_ts_model(name="reviewCountData")
    rating_feature, rating_ts_output = create_ts_model(name="ratingData")

    x = keras.layers.Concatenate()([
        category_output, 
        bbinfo_feature, 
        numinput_feature,
        fba_ts_output,
        amz_ts_output,
        fbm_ts_output,
        bb_ts_output,
        monthly_sold_ts_output,
        sales_rank_ts_output,
        offer_count_ts_output,
        review_count_ts_output,
        rating_ts_output,
    ])

    x = keras.layers.Dense(128, activation='relu')(x)
    x = keras.layers.Dense(64, activation='relu')(x)

    score_pred = keras.layers.Dense(1, activation="sigmoid", name="score")(x)

    return keras.Model(
        inputs=[
            category_feature, 
            bbinfo_feature, 
            numinput_feature,
            fba_feature,
            amz_feature,
            fbm_feature,
            bb_feature,
            monthly_sold_feature,
            sales_rank_feature,
            offer_count_feature,
            review_count_feature,
            rating_feature
        ],
        outputs=[score_pred]
    )

logging.info("Reading data")
data = pd.read_json('./data/out/data_processed.jsonl', lines=True, nrows=10)

logging.info("Processing features...")

# For later: Masking layer should use value of zero since the min is -1 and this will scale -1 to 0
scalar = MinMaxScaler(feature_range=(0, 1))

# Scaling continuous features
features_to_scale = [
    'bsr',
    'buy_box',
    'estimated_sales',
    'category_product_count',
    'salesRankDrops30',
    'rating',
    'reviewCount',
    'offerCount'
    ]

for feature in features_to_scale:
    data[feature] = scalar.fit_transform(data[[feature]]).astype(np.float32)

# Scale each graph individually
graphs_to_scale = [
    "fbaData",
    "amzData",
    "fbmData",
    "buyBoxData",
    "monthlySoldData",
    "salesRankData",
    "offerCountData",
    "reviewCountData",
    "ratingData"
]

for graph in graphs_to_scale:
    for row in data.T:
        if (data[graph][row]):
            data.at[row, graph] = scalar.fit_transform(data.at[row, graph]).astype(np.float32)
    data[graph] = list(pad_sequences(data[graph], padding='post'))

# Multi-hot encoding two boolean vectors
data['bbinfo'] = data.apply(lambda x: [int(x['buyBoxIsFBA']), int(x['buyBoxIsAmazon'])], axis=1)

# Creating numeric_inputs column (input to model)
data['numeric_inputs'] = data.apply(lambda row: [
    row['bsr'],
    row['buy_box'],
    row['estimated_sales'],
    row['category_product_count'],
    row['salesRankDrops30'],
    row['rating'],
    row['reviewCount'],
    row['offerCount'],
    row['bsr_rank'],
    row['estimated_sales_by_category_cnt']
    ], axis=1)

logging.info("Creating model")

model: keras.Model = get_compiled_model(data['category'])
keras.utils.plot_model(model, "data/out/model.png", show_shapes=True, show_layer_names=True)

model.compile(
    optimizer='adam',
    loss=keras.losses.MeanSquaredError,
    metrics=[keras.metrics.MeanSquaredError],

)

logging.info("Training model")

y = data.pop('score')
X_train, X_test, Y_train, Y_test = train_test_split(data, y, test_size=0.2)

callbacks = [
    keras.callbacks.TensorBoard(
        log_dir="./data/logs/tb/",
        histogram_freq=0,
        embeddings_freq=0,
        update_freq="epoch",
    ),
    keras.callbacks.ModelCheckpoint(
        filepath="/data/models/model_{epoch}.keras",
        save_best_only=True,
        monitor="val_loss",
        verbose=1,
    )
]

layer_names = [layer.name for layer in model.input]
# print({layer: X_train[layer] for layer in layer_names})
# for layer in layer_names:
#     print(layer)
#     print(len(X_train[layer][0]))
    # print(tf.convert_to_tensor(X_train[layer].to_list()))

input({layer: tf.convert_to_tensor(X_train[layer].to_list()).numpy().shape for layer in layer_names})

model.fit(
    {layer: tf.convert_to_tensor(X_train[layer].to_list()) for layer in layer_names},
    {'score': Y_train},
    epochs=35,
    batch_size=30,
    callbacks=callbacks
)

logging.info("Model successfully trained... Saving model")

model.save('./data/out/model.keras')

logging.info("Evaluating model...")

results = model.evaluate(
    {layer: tf.convert_to_tensor(X_train[layer].to_list()) for layer in layer_names},
    Y_test
)

logging.info("Evaluation results: %s" % results)

Relevant log output

Reading data
Processing features...
Creating model
Checkpoint not found. Compiling new model...
2024-08-07 12:12:14.148288: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
Training model
{'category': (8,), 'bbinfo': (8, 2), 'numeric_inputs': (8, 10), 'fbaData': (8, 215, 2), 'amzData': (8, 481, 2), 'fbmData': (8, 177, 2), 'buyBoxData': (8, 848, 2), 'monthlySoldData': (8, 0), 'salesRankData': (8, 9685, 2), 'offerCountData': (8, 995, 2), 'reviewCountData': (8, 5435, 2), 'ratingData': (8, 250, 2)}
C:\Users\austi\Downloads\code\py\solarisAIO\venv\Lib\site-packages\keras\src\callbacks\tensorboard.py:667: UserWarning: Model failed to serialize as JSON. Ignoring... MeanSquaredError.get_config() missing 1 required positional argument: 'self'
  warnings.warn(f"Model failed to serialize as JSON. Ignoring... {exc}")
Epoch 1/35
Traceback (most recent call last):
  File "c:\Users\austi\Downloads\code\py\solarisAIO\model.py", line 211, in <module>
    model.fit(
  File "C:\Users\austi\Downloads\code\py\solarisAIO\venv\Lib\site-packages\keras\src\utils\traceback_utils.py", line 113, in error_handler
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "C:\Users\austi\Downloads\code\py\solarisAIO\venv\Lib\site-packages\keras\src\backend\tensorflow\trainer.py", line 320, in fit
    logs = self.train_function(iterator)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\austi\Downloads\code\py\solarisAIO\venv\Lib\site-packages\tensorflow\python\util\traceback_utils.py", line 153, in error_handler
    raise e.with_traceback(filtered_tb) from None
  File "C:\Users\austi\Downloads\code\py\solarisAIO\venv\Lib\site-packages\keras\src\backend\tensorflow\trainer.py", line 121, in one_step_on_iterator
    outputs = self.distribute_strategy.run(
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\austi\Downloads\code\py\solarisAIO\venv\Lib\site-packages\keras\src\backend\tensorflow\trainer.py", line 108, in one_step_on_data
    return self.train_step(data)
           ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\austi\Downloads\code\py\solarisAIO\venv\Lib\site-packages\keras\src\backend\tensorflow\trainer.py", line 51, in train_step
    y_pred = self(x, training=True)
             ^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\austi\Downloads\code\py\solarisAIO\venv\Lib\site-packages\keras\src\utils\traceback_utils.py", line 113, in error_handler
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "C:\Users\austi\Downloads\code\py\solarisAIO\venv\Lib\site-packages\keras\src\layers\layer.py", line 901, in __call__
    outputs = super().__call__(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\austi\Downloads\code\py\solarisAIO\venv\Lib\site-packages\keras\src\utils\traceback_utils.py", line 113, in error_handler
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "C:\Users\austi\Downloads\code\py\solarisAIO\venv\Lib\site-packages\keras\src\ops\operation.py", line 54, in __call__
    return self.call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\austi\Downloads\code\py\solarisAIO\venv\Lib\site-packages\keras\src\models\functional.py", line 167, in call
    inputs = self._standardize_inputs(inputs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\austi\Downloads\code\py\solarisAIO\venv\Lib\site-packages\keras\src\models\functional.py", line 259, in _standardize_inputs
    return self._adjust_input_rank(flat_inputs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\austi\Downloads\code\py\solarisAIO\venv\Lib\site-packages\keras\src\models\functional.py", line 244, in _adjust_input_rank
    raise ValueError(
ValueError: Invalid input shape for input Tensor("functional_1/Cast:0", shape=(None, 481, 2), dtype=string). Expected shape (None, 1), but input has incompatible shape (None, 481, 2)

data/out/model.png: model

tilakrayal commented 1 month ago

@TheWalkingSea, I tried to execute the mentioned code and faced a different error. Kindly find the gist of it here and also looks like this issue is more related to keras, so could you please raise a new issue in keras-team/keras repo for quick resolution. Thank you!

github-actions[bot] commented 3 weeks ago

This issue is stale because it has been open for 7 days with no activity. It will be closed if no further activity occurs. Thank you.

github-actions[bot] commented 2 weeks ago

This issue was closed because it has been inactive for 7 days since being marked as stale. Please reopen if you'd like to work on this further.

google-ml-butler[bot] commented 2 weeks ago

Are you satisfied with the resolution of your issue? Yes No