jjbrosnan
commented
2 years ago TensorFlow code (note that I don't enforce a version in the pip install, so that could be from a version mismatch. I will be performing more rigorous testing of that now.
import os
os.system("pip install tensorflow")# Deephaven imports
from deephaven import DynamicTableWriter
from deephaven import dtypes as dht
from deephaven.learn import gather
from deephaven.csv import read
from deephaven import learn
# Machine learning imports
import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense
# Python imports
import numpy as np, random, threading, time
iris_raw = read("https://media.githubusercontent.com/media/deephaven/examples/main/Iris/csv/iris.csv")
classes = {}
num_classes = 0
def get_class_number(c):
global classes, num_classes
if c not in classes:
classes[c] = num_classes
num_classes += 1
return classes[c]
iris = iris_raw.update(formulas = ["Class = (int)(byte)get_class_number(Class)"])
# Our neural network
model = Sequential()
model.add(Dense(16, activation = tf.nn.relu))
model.add(Dense(12, activation = tf.nn.relu))
model.add(Dense(3, activation = tf.nn.softmax))
# A function that trains the model
def train_model(X_train, Y_train):
model.compile(optimizer = tf.keras.optimizers.Adam(learning_rate = 0.01), loss = tf.keras.losses.SparseCategoricalCrossentropy(from_logits = True), metrics = ["accuracy"])
model.fit(x = X_train, y = Y_train, epochs = 100)
# A function that gets the model's predictions on input data
def predict_with_model(features):
if features.ndim == 1:
features = np.expand_dims(features, 0)
predictions = model.predict(features)
return np.array([np.argmax(item) for item in predictions], dtype = np.intc)
# A function to gather data from table columns into a NumPy array of doubles
def table_to_array_double(rows, cols):
return gather.table_to_numpy_2d(rows, cols, np_type = np.double)
# A function to gather data from table columns into a NumPy array of doubles
def table_to_array_int(rows, cols):
return gather.table_to_numpy_2d(rows, cols, np_type = np.intc)
# A function to extract a list element and cast to an integer
def get_predicted_class(data, idx):
return int(data[idx])# Use the learn function to train our neural network
learn.learn(
table = iris,
model_func = train_model,
inputs = [learn.Input(["SepalLengthCM", "SepalWidthCM", "PetalLengthCM", "PetalWidthCM"], table_to_array_double), learn.Input(["Class"], table_to_array_int)],
outputs = None,
batch_size = 150
)
niloc132
Jeremiahcheng1206
Description
deephaven.learn, when used in conjunction with PyTorch/TensorFlow causesSIGSEGVcrashes. These crashes areSEGV_MAPERRaccording to the logs. This occurs with our supported version of PyTorch (1.10.2) that we use in theserver-pytorchimage. This also occurs with the latest TensorFlow (2.8.0), and I am going to test with the supported version of TensorFlow from theserver-tensorflowimage shortly.Steps to reproduce
From a Python session:
That takes a couple of minutes. Then run this:
That code should run without issue. This next code block will result in the error:
Expected results
The Torch model to train. There should be some printouts to the console window as well.
Actual results
The table viewer window in the console complains for a second, then the screen just goes blank. TRANSIENT_FAILURE messages appear in the terminal window, and a file is generated:
hs_err_pid1.log(among other files).Additional details and attachments
This file is from one of the TensorFlow crashes.
hs_err_pid1.log
I will include the TensorFlow code blocks in a comment.
Versions