taomanwai / tensorboardcolab

A library make TensorBoard working in Colab Google
MIT License
149 stars 26 forks source link

FailedPreconditionError when adding callback in Colab #2

Open austinmw opened 6 years ago

austinmw commented 6 years ago

Hi, when I add this callback to a simple Keras script in Colab I get the error:

FailedPreconditionError: Error while reading resource variable conv_dw_8_1/depthwise_kernel from Container: localhost. This could mean that the variable was uninitialized. Not found: Resource localhost/conv_dw_8_1/depthwise_kernel/N10tensorflow3VarE does not exist. [[Node: conv_dw_8_1/depthwise/ReadVariableOp = ReadVariableOpdtype=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:GPU:0"]] [[Node: metrics_1/acc/Mean/_99 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_6754_metrics_1/acc/Mean", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"]()]]

Any idea why?

stetelepta commented 5 years ago

Ran into the same error, this is caused by using tf.keras together with tensorboardcolab.

Either fix the imports and use the full keras library or patch tensorboardcolab to import the tf.keras module (made these changes in this fork. I needed to use the tf.keras version, because I want to use TPU's for training.

You can find a bit more details at the question on Stackoverflow on this issue.