search

0xPolygonMiden / miden-vm

STARK-based virtual machine
MIT License
621 stars 158 forks source link

Allow querying exported procedures MAST roots from compiled modules #1291

Open hackaugusto opened 6 months ago

hackaugusto commented 6 months ago

Currently there is no API to request the MAST root of a given procedure. This results in a lot of hacks to get the MAST root in the miden-base repo for testing. Which is error prone, time consuming, and usually results in a worse development workflow.

We should be able to query the MAST root of any exported procedure via a public API to make testing easier.

bobbinth commented 6 months ago

Agreed! I think this would be a part of the spec discussed in https://github.com/0xPolygonMiden/compiler/pull/129.

hackaugusto commented 6 months ago

Agreed! I think this would be a part of the spec discussed in https://github.com/0xPolygonMiden/compiler/pull/129.

Why should this be blocked by the module spec? a simple get_proc_by_name API to a compiled Module should be enough here.

bitwalker commented 6 months ago

Why should this be blocked by the module spec? a simple get_proc_by_name API to a compiled Module should be enough here.

The issue here is twofold: 1.) We are moving to a model where MAST is our primary artifact, not MASM, so any metadata not provided as part of/with the MAST cannot be assumed to be available; 2.) Currently, MAST has no such metadata associated with it. As it stands, the only reason why that information can be retrieved at all right now, is because we are assembling MASM to MAST and executing it all at once, so the links between the original MASM and the MAST are still available. That won't be the case for much longer, so we need to account for that.

I've brought this up in #1277 as well, but I think there are two ways we should go about this:

1.) Enhance MAST a bit with some additional metadata about a given MAST root. See my comment here for more detail. This doesn't have to be part of the MAST node itself, but would need to be associated with it in some way. 2.) Provide this metadata as part of the package format, i.e. the package provides a map of MAST roots to procedure metadata.

Then, when loading MAST into memory for execution, we also have this metadata available, which can be introspected as needed.

We need this kind of functionality for more than just the specific use case described by this issue, so it's certainly an important issue to solve.

hackaugusto commented 6 months ago

@bitwalker thanks for the input. I'm not sure if you're agreeing or disagreeing with me though, so just to clarify: I just want to remove a pain point on developing miden-base, which is the need to manually maintain a list of mast roots, that come from test MASM code, compiled on the fly by the tests. I was trying to make the solution to this simple, and I think this can be handled as an orthogonal problem to the packaging done by the compiler, which would unblock this for me much sooner.

bitwalker commented 6 months ago

I'm 100% agreeing with you that this is needed, just that implementing an API for this isn't quite as straightforward (yet).

However #1277 does implement some infrastructure to help with this issue specifically in regards to testing, though your specific needs may vary. See the definition of one such helper here and an example of how it is used here. You could build a similar set of helpers in miden-base on top of the same primitives I believe.

hackaugusto commented 6 months ago

Oh, the linked utility seems great, and it will likely be sufficient for the issue I had. I think we can close this issue once #1277 is merged.