search

iree-org / iree

A retargetable MLIR-based machine learning compiler and runtime toolkit.
http://iree.dev/
Apache License 2.0
2.85k stars 614 forks source link

VMFBs fail to load with "required module 'io_parameters' not registered on the context" #17988

Open gpetters-amd opened 3 months ago

gpetters-amd commented 3 months ago

What happened?

Compiling a model (Llama2) on various backends, compilation succeeds but ireert is unable to load the vmfb on any of them. They all fail with Error creating vm context with modules: c/runtime/src/iree/vm/context.c:156: NOT_FOUND; required module 'io_parameters' not registered on the context

Steps to reproduce your issue

  1. python repro.py

What component(s) does this issue relate to?

Compiler, Runtime

Version information

20240621.931

Additional context

This is on SHARK-Turbine and IREE_SAVE_TEMPS isn't outputting reproducers, so it may be hard to get a smaller reproducer.

gpetters-amd commented 3 months ago

I'll be adding reproducers once I get them uploaded

benvanik commented 3 months ago

if you are using parameters then you'll need to pass the parameters by way of loading the parameters module. This is the runtime saying "hey, your module tried to use parameters but the application never told me they exist".

ScottTodd commented 3 months ago

https://iree.dev/guides/parameters/#from-python

Reference code:

https://github.com/iree-org/iree/blob/0934526e288315552650554da12d49b94845c7bf/runtime/bindings/python/tests/io_runtime_test.py#L69-L78

https://github.com/nod-ai/sharktank/blob/7cd7c6a2851d8d774879be6a1418ac6fee858717/shortfin/shortfin/framework/session.py#L174-L179