google / earthengine-community

Tutorials and content created by Earth Engine users, for Earth Engine users
https://earthengine.google.com/
Apache License 2.0
540 stars 707 forks source link

Error in TF_demo1_keras.ipynb tutorial #736

Closed tpfd closed 6 months ago

tpfd commented 9 months ago

Hi there,

I am running this code in a Jupyter Notebook on a VertexAI instance, n1-highmem-16 with a GPU attached, using the pre-built TF instance.

I think there may be an issue with: https://github.com/google/earthengine-community/blob/master/guides/linked/TF_demo1_keras.ipynb

Running the "Write the predictions to a TFRecord file" code snippet generates the error: TypeError: only integer scalar arrays can be converted to a scalar index for line : int64_list=tf.train.Int64List( value=patch[0])),

I have attempted to resolve this by implementing: int_predictions_p0 = np.array(patch[0], dtype=np.int64) ... feature={ 'prediction': tf.train.Feature( int64_list=tf.train.Int64List( value=int_predictions_p0)),

In order to set the dtype to the desired one, noting that the other three classes/patches are requesting float lists and therefore should be fine. Apolgoies, I cannot yet tell you if this fix worked as it is taking time to run! And I am not sure if that is as intended or due to what I have done here...

EDIT: three hours later can confirm that my attempted fix did not work. No data has been written to the output file either during the opertaion or upon terminating the process.

OliverJMG commented 9 months ago

Changing this line

patch[0].append(tf.argmax(prediction, 1))

to

patch[0].append(tf.argmax(prediction[0]))

resolves the issue. The prediction variable is a 2D array with only one row, so tf.argmax(prediction, 1) returns a 1D array with only one element as output. This means patch[0] ends up as a list of arrays rather than a list of integers which causes the problem. Passing the only row of prediction to argmax instead means you get an integer output and the code works.

jdbcode commented 6 months ago

The page where this notebook is linked from (https://developers.google.com/earth-engine/guides/tf_examples) is deprecated. For others reading this, please use the updated example: https://developers.google.com/earth-engine/guides/ml_examples#multi-class-prediction-with-a-dnn-from-scratch. TF_demo1_keras.ipynb file should be labeled as deprecated and point to the current version.

jdbcode commented 6 months ago

767 adds deprecation note and redirect