search

lutzroeder / netron

Visualizer for neural network, deep learning and machine learning models
https://netron.app
MIT License
27.72k stars 2.75k forks source link

MLIR support #1044

Open lutzroeder opened 1 year ago

lutzroeder commented 1 year ago

Documentation MLIR Bytecode Format

Examples: 1043.mlir.zip embedding_bm1684x_f16_tpu.mlir.txt examples.mnist_xla.mlir.zip versioned-op-2.0.mlirbc.zip

tucan9389 commented 1 year ago

@lutzroeder First of all, thanks for your issue. And thanks for the repo as a person who has used this tool for almost 4 years.

And I'm personally interested in this issue.

Would you happen to have any good references to make some progress on this? I'm trying to understand the overall code structure and the detailed implementation (especially for tflite and coreml), but it's pretty hard to understand at once. Would you give some guides or tips? I think that good reference commits might be helpful for understanding the things that I should implement.

lutzroeder commented 1 year ago

Initially this is likely about writing a JavaScript parser for MLIR to tokenize the file and extract the module structure. onnx.Reader.text as a starting point of what such code looks like for a similar text format.

There is already a mlir.ModelFactory which gets invoked when dragging an .mlir file into a view. The rest of the work is about mapping the parsed MLIR module into the object model the viewer consumes. CONTRIBUTING.md has steps to launch a debugger. node ./test/models.js mlir to run the existing test files. dnn.js is a format with a short implementation hat might be easier to follow to learn about the basic graph, node, attribute and parameter types in the object model.

tucan9389 commented 1 year ago

@lutzroeder Gently reminder. Could you review #1088?

And I'm planning quite a big update for testing with the .mlir files on Github and reinforcement of the mlir.Tokenizer and mlir.Parser based on the #1088. If you think we should have a #1088 review first and move forward, please tell me ASAP.

Thanks :)

lutzroeder commented 1 year ago

@tucan9389 continue iterating on the pull request. Let's review once it is in good shape to move towards adding it to a release.

tucan9389 commented 1 year ago

@lutzroeder Oh, I see. The MLIR parser can be a massive complex. Before it gets too big, I thought we could merge or get some reviews. I'm not sure if this contribution adds value to this repository.

Do you happen to have a useful guide for releasing this feature or any test cases to merge into the main branch?

I would prefer to get reviews in between rather than getting big reviews at once, but if it's difficult for you now, could you please let me know the minimum requirements for merging? I wonder if you have thought about the work unit to merge.

tucan9389 commented 1 year ago

@lutzroeder Hi, I made some additional progress. Please check https://github.com/tucan9389/netron/issues/1 out.

I think this can be merged into the main repo. Can you review and pull my implementation? If you think there are remaining works, please let me know.

This is my side project, so the next progress could take some time.

Anyway, thanks for your comment and advice :)

tucan9389 commented 11 months ago

@lutzroeder Hi, I found that my previous implementation (to parse a text MLIR file in js level) was not a good way to achieve sustainability. I'm looking for a more proper way to use WebAssembly in netron to use MLIR parser directly. This can make parse MLIR graphs with ease and with less error. I'll comment if I make some progress.

Here the keywords are that I'm looking at:

lutzroeder commented 11 months ago

@tucan9389 do you have some additional context what you mean by "achive sustainability"? The goal is to avoid dependencies, run in many environments and support a unified debugging experience. Improving the JavaScript parser implementation seems the right path to get there.