[benchmark] latest benchmark 'convert.sh' throws 'MemroyError'

[zhaoming0]

When we run convert.sh with 967fcdd It throw MemoryError

ENV INFO

ubuntu 17.10_64bit
CPU: Intel(R) Core(TM) i7-6700K CPU @ 4.00GHz
RAM: 31GB

STEP README

Error Info:

[convert_keras.py] Generating graph descriptor
[convert_keras.py] BackendName: webgpu
/usr/local/lib/python3.6/dist-packages/webdnn-1.1.0-py3.6.egg/webdnn/util/console.py:30: Warning: [WebGPUDescriptorGenerator] 'xcrun' command is not found. validation of generated source code in webgpu backend is skipped.
  warnings.warn(message, category)
[convert_keras.py] BackendName: webgl
[convert_keras.py] Failed generating descriptor for webgl backend
Traceback (most recent call last):
  File "../../bin/convert_keras.py", line 97, in main
    graph_exec_data = generate_descriptor(backend, graph, constant_encoder_name=args.encoding)
  File "/usr/local/lib/python3.6/dist-packages/webdnn-1.1.0-py3.6.egg/webdnn/backend/interface/generator.py", line 107, in generate_descriptor
    return generator(graph, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/webdnn-1.1.0-py3.6.egg/webdnn/backend/webgl/generator.py", line 92, in generate
    return WebGLDescriptorGenerator.generate(graph, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/webdnn-1.1.0-py3.6.egg/webdnn/backend/webgl/generator.py", line 45, in generate
    graph, _ = WebGLOptimizeRule().optimize(copy.deepcopy(original_graph))
  File "/usr/local/lib/python3.6/dist-packages/webdnn-1.1.0-py3.6.egg/webdnn/graph/optimize_rule.py", line 110, in optimize
    graph, flag_changed = sub_rule.optimize(graph)
  File "/usr/local/lib/python3.6/dist-packages/webdnn-1.1.0-py3.6.egg/webdnn/graph/optimize_rule.py", line 110, in optimize
    graph, flag_changed = sub_rule.optimize(graph)
  File "/usr/local/lib/python3.6/dist-packages/webdnn-1.1.0-py3.6.egg/webdnn/backend/webgl/optimize_rules/unroll_concat.py", line 51, in optimize
    hs.append(xs[0])
MemoryError

@Kiikurage PTAL

[Kiikurage]

With my MacBook Pro with 16GB RAM, I succeeded to convert. Since the conversion is deterministic, this error looks very strange. I'll investigate further. (I'm sorry, but I am very busy in these days and it may take several days to reply).

[zhaoming0]

Check with my MacBook Pro 8GB, also convert fail

Env

MacBook Pro (Retina, 13-inch, Early 2015)
OS Version: 10.13.3
Commit ID: 967fcdd

Step:

1. checkout latest code
2. remove webdnn/example/benchmark/output folder
3. run convert.sh

Error Message:

[convert_keras.py] Generating feedforward graph
2018-01-30 10:47:39.611599: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
/usr/local/lib/python3.6/site-packages/webdnn-1.1.0-py3.6.egg/webdnn/util/console.py:30: Warning: [KerasConverter] keras.layers.AveragePooling computes average by dividing number of valid elements in window (without padding element), but WebDNN divides it by the number of elements including padding element, so different result will be generated on the edge.
  warnings.warn(message, category)
[convert_keras.py] Generating graph descriptor
[convert_keras.py] BackendName: webgpu
[convert_keras.py] BackendName: webgl
./convert.sh: line 40: 30200 Killed: 9               OPTIMIZE=0 python3 ../../bin/convert_keras.py output/kerasjs/inception_v3/model.h5 --input_shape '(1,299,299,3)' --out output/webdnn/inception_v3/non_optimized 

BTW, Please ignore the above PR