apple / tensorflow_macos

TensorFlow for macOS 11.0+ accelerated using Apple's ML Compute framework.
Other
3.66k stars 308 forks source link

Not sure ML Compute is working ? #195

Open labdong801 opened 3 years ago

labdong801 commented 3 years ago

Here is the logs when i run the example "speech_commands"

python tensorflow/examples/speech_commands/train.py

======== 2021-03-08 20:50:58.297782: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:196] None of the MLIR optimization passes are enabled (registered 0 passes) 2021-03-08 20:50:58.297981: W tensorflow/core/platform/profile_utils/cpu_utils.cc:126] Failed to get CPU frequency: 0 Hz INFO:tensorflow:Training from step: 1 I0308 20:50:58.441840 4368547136 train.py:212] Training from step: 1 2021-03-08 20:50:58.716976: I tensorflow/compiler/tf2mlcompute/kernels/mlc_subgraph_op.cc:332] Compute: Failed in processing TensorFlow graph MLCSubgraphOp_0_1 with frame_id = 0 and iter_id = 0 with error: Internal: AddLayerToMLCGraph: Failed to add layer to MLCGraph for node: n10 = dropout/RealDiv (RealDiv). (error will be reported 5 times unless TF_MLC_LOGGING=1). 2021-03-08 20:50:58.810334: I tensorflow/compiler/tf2mlcompute/kernels/mlc_subgraph_op.cc:332] Compute: Failed in processing TensorFlow graph MLCSubgraphOp_0_2 with frame_id = 0 and iter_id = 0 with error: Internal: AddLayerToMLCGraph: Failed to add layer to MLCGraph for node: n11 = dropout_1/RealDiv (RealDiv). (error will be reported 5 times unless TF_MLC_LOGGING=1). 2021-03-08 20:50:58.823076: I tensorflow/compiler/tf2mlcompute/kernels/mlc_subgraph_op.cc:332] Compute: Failed in processing TensorFlow graph train/gradients/add_2_grad/MLCSubgraphOp_0_5 with frame_id = 0 and iter_id = 0 with error: Internal: PerformGradientPassNodeRoutine: Failed to find forward-pass output for node: add_2 (error will be reported 5 times unless TF_MLC_LOGGING=1). 2021-03-08 20:50:58.823401: I tensorflow/compiler/tf2mlcompute/kernels/mlc_subgraph_op.cc:332] Compute: Failed in processing TensorFlow graph train/gradients/MatMul_grad/MLCSubgraphOp_0_6 with frame_id = 0 and iter_id = 0 with error: Internal: PerformGradientPassNodeRoutine: Failed to find forward-pass output for node: MatMul (error will be reported 5 times unless TF_MLC_LOGGING=1). 2021-03-08 20:50:58.837354: I tensorflow/compiler/tf2mlcompute/kernels/mlc_subgraph_op.cc:332] Compute: Failed in processing TensorFlow graph train/gradients/MLCSubgraphOp_0_7 with frame_id = 0 and iter_id = 0 with error: Internal: PerformGradientPassNodeRoutine: Failed to find forward-pass output for node: Reshape_1 (error will be reported 5 times unless TF_MLC_LOGGING=1). 2021-03-08 20:50:58.858146: I tensorflow/compiler/tf2mlcompute/kernels/mlc_subgraph_op.cc:332] Compute: Failed in processing TensorFlow graph train/gradients/dropout_1/RealDiv_grad/MLCSubgraphOp_0_8 with frame_id = 0 and iter_id = 0 with error: Internal: PerformGradientPassNodeRoutine: Failed to find forward-pass output for node: dropout_1/RealDiv (error will be reported 5 times unless TF_MLC_LOGGING=1). 2021-03-08 20:50:58.867047: I tensorflow/compiler/tf2mlcompute/kernels/mlc_subgraph_op.cc:332] Compute: Failed in processing TensorFlow graph train/gradients/MLCSubgraphOp_0_9 with frame_id = 0 and iter_id = 0 with error: Internal: PerformGradientPassNodeRoutine: Failed to find forward-pass output for node: Relu_1 (error will be reported 5 times unless TF_MLC_LOGGING=1). 2021-03-08 20:50:58.875929: I tensorflow/compiler/tf2mlcompute/kernels/mlc_subgraph_op.cc:332] Compute: Failed in processing TensorFlow graph train/gradients/MLCConv2D_1_grad/MLCSubgraphOp_0_10 with frame_id = 0 and iter_id = 0 with error: Internal: PerformGradientPassNodeRoutine: Failed to find forward-pass output for node: MLCConv2D_1 (error will be reported 5 times unless TF_MLC_LOGGING=1). 2021-03-08 20:50:59.068169: I tensorflow/compiler/tf2mlcompute/kernels/mlc_subgraph_op.cc:332] Compute: Failed in processing TensorFlow graph train/gradients/MLCSubgraphOp_0_11 with frame_id = 0 and iter_id = 0 with error: Internal: PerformGradientPassNodeRoutine: Failed to find forward-pass output for node: MaxPool2d (error will be reported 5 times unless TF_MLC_LOGGING=1). 2021-03-08 20:50:59.152503: I tensorflow/compiler/tf2mlcompute/kernels/mlc_subgraph_op.cc:332] Compute: Failed in processing TensorFlow graph train/gradients/dropout/RealDiv_grad/MLCSubgraphOp_0_12 with frame_id = 0 and iter_id = 0 with error: Internal: PerformGradientPassNodeRoutine: Failed to find forward-pass output for node: dropout/RealDiv (error will be reported 5 times unless TF_MLC_LOGGING=1). 2021-03-08 20:50:59.933185: I tensorflow/compiler/tf2mlcompute/kernels/mlc_subgraph_op.cc:332] Compute: Failed in processing TensorFlow graph MLCSubgraphOp_0_1 with frame_id = 0 and iter_id = 0 with error: Internal: AddLayerToMLCGraph: Failed to add layer to MLCGraph for node: n10 = dropout/RealDiv (RealDiv). (error will be reported 5 times unless TF_MLC_LOGGING=1). 2021-03-08 20:51:00.019623: I tensorflow/compiler/tf2mlcompute/kernels/mlc_subgraph_op.cc:332] Compute: Failed in processing TensorFlow graph MLCSubgraphOp_0_2 with frame_id = 0 and iter_id = 0 with error: Internal: AddLayerToMLCGraph: Failed to add layer to MLCGraph for node: n11 = dropout_1/RealDiv (RealDiv). (error will be reported 5 times unless TF_MLC_LOGGING=1). 2021-03-08 20:51:00.030359: I tensorflow/compiler/tf2mlcompute/kernels/mlc_subgraph_op.cc:332] Compute: Failed in processing TensorFlow graph train/gradients/add_2_grad/MLCSubgraphOp_0_5 with frame_id = 0 and iter_id = 0 with error: Internal: PerformGradientPassNodeRoutine: Failed to find forward-pass output for node: add_2 (error will be reported 5 times unless TF_MLC_LOGGING=1). 2021-03-08 20:51:00.030557: I tensorflow/compiler/tf2mlcompute/kernels/mlc_subgraph_op.cc:332] Compute: Failed in processing TensorFlow graph train/gradients/MatMul_grad/MLCSubgraphOp_0_6 with frame_id = 0 and iter_id = 0 with error: Internal: PerformGradientPassNodeRoutine: Failed to find forward-pass output for node: MatMul (error will be reported 5 times unless TF_MLC_LOGGING=1). 2021-03-08 20:51:00.044688: I tensorflow/compiler/tf2mlcompute/kernels/mlc_subgraph_op.cc:332] Compute: Failed in processing TensorFlow graph train/gradients/MLCSubgraphOp_0_7 with frame_id = 0 and iter_id = 0 with error: Internal: PerformGradientPassNodeRoutine: Failed to find forward-pass output for node: Reshape_1 (error will be reported 5 times unless TF_MLC_LOGGING=1). 2021-03-08 20:51:00.062611: I tensorflow/compiler/tf2mlcompute/kernels/mlc_subgraph_op.cc:332] Compute: Failed in processing TensorFlow graph train/gradients/dropout_1/RealDiv_grad/MLCSubgraphOp_0_8 with frame_id = 0 and iter_id = 0 with error: Internal: PerformGradientPassNodeRoutine: Failed to find forward-pass output for node: dropout_1/RealDiv (error will be reported 5 times unless TF_MLC_LOGGING=1). 2021-03-08 20:51:00.069622: I tensorflow/compiler/tf2mlcompute/kernels/mlc_subgraph_op.cc:332] Compute: Failed in processing TensorFlow graph train/gradients/MLCSubgraphOp_0_9 with frame_id = 0 and iter_id = 0 with error: Internal: PerformGradientPassNodeRoutine: Failed to find forward-pass output for node: Relu_1 (error will be reported 5 times unless TF_MLC_LOGGING=1). 2021-03-08 20:51:00.079133: I tensorflow/compiler/tf2mlcompute/kernels/mlc_subgraph_op.cc:332] Compute: Failed in processing TensorFlow graph train/gradients/MLCConv2D_1_grad/MLCSubgraphOp_0_10 with frame_id = 0 and iter_id = 0 with error: Internal: PerformGradientPassNodeRoutine: Failed to find forward-pass output for node: MLCConv2D_1 (error will be reported 5 times unless TF_MLC_LOGGING=1). 2021-03-08 20:51:00.273360: I tensorflow/compiler/tf2mlcompute/kernels/mlc_subgraph_op.cc:332] Compute: Failed in processing TensorFlow graph train/gradients/MLCSubgraphOp_0_11 with frame_id = 0 and iter_id = 0 with error: Internal: PerformGradientPassNodeRoutine: Failed to find forward-pass output for node: MaxPool2d (error will be reported 5 times unless TF_MLC_LOGGING=1). 2021-03-08 20:51:00.358131: I tensorflow/compiler/tf2mlcompute/kernels/mlc_subgraph_op.cc:332] Compute: Failed in processing TensorFlow graph train/gradients/dropout/RealDiv_grad/MLCSubgraphOp_0_12 with frame_id = 0 and iter_id = 0 with error: Internal: PerformGradientPassNodeRoutine: Failed to find forward-pass output for node: dropout/RealDiv (error will be reported 5 times unless TF_MLC_LOGGING=1). 2021-03-08 20:51:01.136845: I tensorflow/compiler/tf2mlcompute/kernels/mlc_subgraph_op.cc:332] Compute: Failed in processing TensorFlow graph MLCSubgraphOp_0_1 with frame_id = 0 and iter_id = 0 with error: Internal: AddLayerToMLCGraph: Failed to add layer to MLCGraph for node: n10 = dropout/RealDiv (RealDiv). (error will be reported 5 times unless TF_MLC_LOGGING=1). 2021-03-08 20:51:01.224057: I tensorflow/compiler/tf2mlcompute/kernels/mlc_subgraph_op.cc:332] Compute: Failed in processing TensorFlow graph MLCSubgraphOp_0_2 with frame_id = 0 and iter_id = 0 with error: Internal: AddLayerToMLCGraph: Failed to add layer to MLCGraph for node: n11 = dropout_1/RealDiv (RealDiv). (error will be reported 5 times unless TF_MLC_LOGGING=1). 2021-03-08 20:51:01.235280: I tensorflow/compiler/tf2mlcompute/kernels/mlc_subgraph_op.cc:332] Compute: Failed in processing TensorFlow graph train/gradients/add_2_grad/MLCSubgraphOp_0_5 with frame_id = 0 and iter_id = 0 with error: Internal: PerformGradientPassNodeRoutine: Failed to find forward-pass output for node: add_2 (error will be reported 5 times unless TF_MLC_LOGGING=1). 2021-03-08 20:51:01.235464: I tensorflow/compiler/tf2mlcompute/kernels/mlc_subgraph_op.cc:332] Compute: Failed in processing TensorFlow graph train/gradients/MatMul_grad/MLCSubgraphOp_0_6 with frame_id = 0 and iter_id = 0 with error: Internal: PerformGradientPassNodeRoutine: Failed to find forward-pass output for node: MatMul (error will be reported 5 times unless TF_MLC_LOGGING=1). 2021-03-08 20:51:01.250128: I tensorflow/compiler/tf2mlcompute/kernels/mlc_subgraph_op.cc:332] Compute: Failed in processing TensorFlow graph train/gradients/MLCSubgraphOp_0_7 with frame_id = 0 and iter_id = 0 with error: Internal: PerformGradientPassNodeRoutine: Failed to find forward-pass output for node: Reshape_1 (error will be reported 5 times unless TF_MLC_LOGGING=1). 2021-03-08 20:51:01.268031: I tensorflow/compiler/tf2mlcompute/kernels/mlc_subgraph_op.cc:332] Compute: Failed in processing TensorFlow graph train/gradients/dropout_1/RealDiv_grad/MLCSubgraphOp_0_8 with frame_id = 0 and iter_id = 0 with error: Internal: PerformGradientPassNodeRoutine: Failed to find forward-pass output for node: dropout_1/RealDiv (error will be reported 5 times unless TF_MLC_LOGGING=1). 2021-03-08 20:51:01.276243: I tensorflow/compiler/tf2mlcompute/kernels/mlc_subgraph_op.cc:332] Compute: Failed in processing TensorFlow graph train/gradients/MLCSubgraphOp_0_9 with frame_id = 0 and iter_id = 0 with error: Internal: PerformGradientPassNodeRoutine: Failed to find forward-pass output for node: Relu_1 (error will be reported 5 times unless TF_MLC_LOGGING=1). 2021-03-08 20:51:01.284660: I tensorflow/compiler/tf2mlcompute/kernels/mlc_subgraph_op.cc:332] Compute: Failed in processing TensorFlow graph train/gradients/MLCConv2D_1_grad/MLCSubgraphOp_0_10 with frame_id = 0 and iter_id = 0 with error: Internal: PerformGradientPassNodeRoutine: Failed to find forward-pass output for node: MLCConv2D_1 (error will be reported 5 times unless TF_MLC_LOGGING=1). 2021-03-08 20:51:01.479738: I tensorflow/compiler/tf2mlcompute/kernels/mlc_subgraph_op.cc:332] Compute: Failed in processing TensorFlow graph train/gradients/MLCSubgraphOp_0_11 with frame_id = 0 and iter_id = 0 with error: Internal: PerformGradientPassNodeRoutine: Failed to find forward-pass output for node: MaxPool2d (error will be reported 5 times unless TF_MLC_LOGGING=1). 2021-03-08 20:51:01.553270: I tensorflow/compiler/tf2mlcompute/kernels/mlc_subgraph_op.cc:332] Compute: Failed in processing TensorFlow graph train/gradients/dropout/RealDiv_grad/MLCSubgraphOp_0_12 with frame_id = 0 and iter_id = 0 with error: Internal: PerformGradientPassNodeRoutine: Failed to find forward-pass output for node: dropout/RealDiv (error will be reported 5 times unless TF_MLC_LOGGING=1). 2021-03-08 20:51:02.327338: I tensorflow/compiler/tf2mlcompute/kernels/mlc_subgraph_op.cc:332] Compute: Failed in processing TensorFlow graph MLCSubgraphOp_0_1 with frame_id = 0 and iter_id = 0 with error: Internal: AddLayerToMLCGraph: Failed to add layer to MLCGraph for node: n10 = dropout/RealDiv (RealDiv). (error will be reported 5 times unless TF_MLC_LOGGING=1). 2021-03-08 20:51:02.414561: I tensorflow/compiler/tf2mlcompute/kernels/mlc_subgraph_op.cc:332] Compute: Failed in processing TensorFlow graph MLCSubgraphOp_0_2 with frame_id = 0 and iter_id = 0 with error: Internal: AddLayerToMLCGraph: Failed to add layer to MLCGraph for node: n11 = dropout_1/RealDiv (RealDiv). (error will be reported 5 times unless TF_MLC_LOGGING=1). 2021-03-08 20:51:02.425896: I tensorflow/compiler/tf2mlcompute/kernels/mlc_subgraph_op.cc:332] Compute: Failed in processing TensorFlow graph train/gradients/add_2_grad/MLCSubgraphOp_0_5 with frame_id = 0 and iter_id = 0 with error: Internal: PerformGradientPassNodeRoutine: Failed to find forward-pass output for node: add_2 (error will be reported 5 times unless TF_MLC_LOGGING=1). 2021-03-08 20:51:02.426110: I tensorflow/compiler/tf2mlcompute/kernels/mlc_subgraph_op.cc:332] Compute: Failed in processing TensorFlow graph train/gradients/MatMul_grad/MLCSubgraphOp_0_6 with frame_id = 0 and iter_id = 0 with error: Internal: PerformGradientPassNodeRoutine: Failed to find forward-pass output for node: MatMul (error will be reported 5 times unless TF_MLC_LOGGING=1). 2021-03-08 20:51:02.440612: I tensorflow/compiler/tf2mlcompute/kernels/mlc_subgraph_op.cc:332] Compute: Failed in processing TensorFlow graph train/gradients/MLCSubgraphOp_0_7 with frame_id = 0 and iter_id = 0 with error: Internal: PerformGradientPassNodeRoutine: Failed to find forward-pass output for node: Reshape_1 (error will be reported 5 times unless TF_MLC_LOGGING=1). 2021-03-08 20:51:02.458625: I tensorflow/compiler/tf2mlcompute/kernels/mlc_subgraph_op.cc:332] Compute: Failed in processing TensorFlow graph train/gradients/dropout_1/RealDiv_grad/MLCSubgraphOp_0_8 with frame_id = 0 and iter_id = 0 with error: Internal: PerformGradientPassNodeRoutine: Failed to find forward-pass output for node: dropout_1/RealDiv (error will be reported 5 times unless TF_MLC_LOGGING=1). 2021-03-08 20:51:02.465790: I tensorflow/compiler/tf2mlcompute/kernels/mlc_subgraph_op.cc:332] Compute: Failed in processing TensorFlow graph train/gradients/MLCSubgraphOp_0_9 with frame_id = 0 and iter_id = 0 with error: Internal: PerformGradientPassNodeRoutine: Failed to find forward-pass output for node: Relu_1 (error will be reported 5 times unless TF_MLC_LOGGING=1). 2021-03-08 20:51:02.475185: I tensorflow/compiler/tf2mlcompute/kernels/mlc_subgraph_op.cc:332] Compute: Failed in processing TensorFlow graph train/gradients/MLCConv2D_1_grad/MLCSubgraphOp_0_10 with frame_id = 0 and iter_id = 0 with error: Internal: PerformGradientPassNodeRoutine: Failed to find forward-pass output for node: MLCConv2D_1 (error will be reported 5 times unless TF_MLC_LOGGING=1). 2021-03-08 20:51:02.669428: I tensorflow/compiler/tf2mlcompute/kernels/mlc_subgraph_op.cc:332] Compute: Failed in processing TensorFlow graph train/gradients/MLCSubgraphOp_0_11 with frame_id = 0 and iter_id = 0 with error: Internal: PerformGradientPassNodeRoutine: Failed to find forward-pass output for node: MaxPool2d (error will be reported 5 times unless TF_MLC_LOGGING=1). 2021-03-08 20:51:02.743400: I tensorflow/compiler/tf2mlcompute/kernels/mlc_subgraph_op.cc:332] Compute: Failed in processing TensorFlow graph train/gradients/dropout/RealDiv_grad/MLCSubgraphOp_0_12 with frame_id = 0 and iter_id = 0 with error: Internal: PerformGradientPassNodeRoutine: Failed to find forward-pass output for node: dropout/RealDiv (error will be reported 5 times unless TF_MLC_LOGGING=1). 2021-03-08 20:51:03.510376: I tensorflow/compiler/tf2mlcompute/kernels/mlc_subgraph_op.cc:332] Compute: Failed in processing TensorFlow graph MLCSubgraphOp_0_1 with frame_id = 0 and iter_id = 0 with error: Internal: AddLayerToMLCGraph: Failed to add layer to MLCGraph for node: n10 = dropout/RealDiv (RealDiv). (error will be reported 5 times unless TF_MLC_LOGGING=1). 2021-03-08 20:51:03.597387: I tensorflow/compiler/tf2mlcompute/kernels/mlc_subgraph_op.cc:332] Compute: Failed in processing TensorFlow graph MLCSubgraphOp_0_2 with frame_id = 0 and iter_id = 0 with error: Internal: AddLayerToMLCGraph: Failed to add layer to MLCGraph for node: n11 = dropout_1/RealDiv (RealDiv). (error will be reported 5 times unless TF_MLC_LOGGING=1). 2021-03-08 20:51:03.608759: I tensorflow/compiler/tf2mlcompute/kernels/mlc_subgraph_op.cc:332] Compute: Failed in processing TensorFlow graph train/gradients/add_2_grad/MLCSubgraphOp_0_5 with frame_id = 0 and iter_id = 0 with error: Internal: PerformGradientPassNodeRoutine: Failed to find forward-pass output for node: add_2 (error will be reported 5 times unless TF_MLC_LOGGING=1). 2021-03-08 20:51:03.608962: I tensorflow/compiler/tf2mlcompute/kernels/mlc_subgraph_op.cc:332] Compute: Failed in processing TensorFlow graph train/gradients/MatMul_grad/MLCSubgraphOp_0_6 with frame_id = 0 and iter_id = 0 with error: Internal: PerformGradientPassNodeRoutine: Failed to find forward-pass output for node: MatMul (error will be reported 5 times unless TF_MLC_LOGGING=1). 2021-03-08 20:51:03.623108: I tensorflow/compiler/tf2mlcompute/kernels/mlc_subgraph_op.cc:332] Compute: Failed in processing TensorFlow graph train/gradients/MLCSubgraphOp_0_7 with frame_id = 0 and iter_id = 0 with error: Internal: PerformGradientPassNodeRoutine: Failed to find forward-pass output for node: Reshape_1 (error will be reported 5 times unless TF_MLC_LOGGING=1). 2021-03-08 20:51:03.641110: I tensorflow/compiler/tf2mlcompute/kernels/mlc_subgraph_op.cc:332] Compute: Failed in processing TensorFlow graph train/gradients/dropout_1/RealDiv_grad/MLCSubgraphOp_0_8 with frame_id = 0 and iter_id = 0 with error: Internal: PerformGradientPassNodeRoutine: Failed to find forward-pass output for node: dropout_1/RealDiv (error will be reported 5 times unless TF_MLC_LOGGING=1). 2021-03-08 20:51:03.648295: I tensorflow/compiler/tf2mlcompute/kernels/mlc_subgraph_op.cc:332] Compute: Failed in processing TensorFlow graph train/gradients/MLCSubgraphOp_0_9 with frame_id = 0 and iter_id = 0 with error: Internal: PerformGradientPassNodeRoutine: Failed to find forward-pass output for node: Relu_1 (error will be reported 5 times unless TF_MLC_LOGGING=1). 2021-03-08 20:51:03.657414: I tensorflow/compiler/tf2mlcompute/kernels/mlc_subgraph_op.cc:332] Compute: Failed in processing TensorFlow graph train/gradients/MLCConv2D_1_grad/MLCSubgraphOp_0_10 with frame_id = 0 and iter_id = 0 with error: Internal: PerformGradientPassNodeRoutine: Failed to find forward-pass output for node: MLCConv2D_1 (error will be reported 5 times unless TF_MLC_LOGGING=1). 2021-03-08 20:51:03.851571: I tensorflow/compiler/tf2mlcompute/kernels/mlc_subgraph_op.cc:332] Compute: Failed in processing TensorFlow graph train/gradients/MLCSubgraphOp_0_11 with frame_id = 0 and iter_id = 0 with error: Internal: PerformGradientPassNodeRoutine: Failed to find forward-pass output for node: MaxPool2d (error will be reported 5 times unless TF_MLC_LOGGING=1). 2021-03-08 20:51:03.926553: I tensorflow/compiler/tf2mlcompute/kernels/mlc_subgraph_op.cc:332] Compute: Failed in processing TensorFlow graph train/gradients/dropout/RealDiv_grad/MLCSubgraphOp_0_12 with frame_id = 0 and iter_id = 0 with error: Internal: PerformGradientPassNodeRoutine: Failed to find forward-pass output for node: dropout/RealDiv (error will be reported 5 times unless TF_MLC_LOGGING=1). ....

=====

Mac os Big sur 11.1
Apple M1