JavaScript: accessing generic resource/`share` files (e.g. cxxrtl headers)

[thoughtpolice]

Would it be possible to export some subset of Yosys' resource files directly from the APIs? In particular, I'd like to get access to the (compatible!) cxxrtl headers for my given yosys build, so that I can spit them out alongside a write_cxxrtl

There are probably some other (reasonable, but perhaps marginal) use cases that could be satisfied by providing generic access to other resource files. But I'm also ok with just the cxxrtl headers since that's all I need, I think. :)

[whitequark]

You're talking about the JS embedding, right? (The Python embedding provides that already.)

I wonder if maybe Yosys itself should be taught to export those files.

[thoughtpolice]

Ah, I didn't realize it was already in the Python bindings. Yeah, in that sense I just want some parity in the JavaScript API.

I wonder if maybe Yosys itself should be taught to export those files.

Yeah, this actually does sound smart, though I'm not sure what form it should take or look like, OTTOMH.

[whitequark]

I wonder if maybe Yosys itself should be taught to export those files.

The reason I'm asking is because there is the plan to eventually use wasi-virt to "bake" the resource files into the final .wasm file, so that the runtime does not need to care about them whatsoever. But wasi-virt is currrently broken when combined with jco, so we can't use it...

If the files are baked in, then they aren't present in a form that can be extracted without the assistance of the called binary.

Yeah, this actually does sound smart, though I'm not sure what form it should take or look like, OTTOMH.

With my Yosys maintainer hat on, I would propose a new pass (write_share_files or something, name TBD) which takes a glob of files under the Yosys share directory and simply copies it to a given path. Do you think you could implement that?

[thoughtpolice]

After doing some spelunking in the source code, motivated by the fact I wanted to leverage and understand the +/ syntax for a hypothetical patch (I've seen it often used in techmapping library cells, for example), I've found that through the power of the internal rewrite_filename() in kernel.cc, this is already supported in the little known, tiny, small, but mighty write_file pass! Try this for yourself in a fresh yosys repl:

yosys> write_file /dev/stdout +/include/backends/cxxrtl/runtime/cxxrtl/cxxrtl.h

Sure enough this already works with YoWASP, assuming that I have a Tree coming back from a synthesis call:

// synthesis script writes out `cxxrtl.h` with `write_file` like above
const cxxrtlUint8 = synthTree["cxxrtl.h"] as Uint8Array;
const cxxrtlString = new TextDecoder().decode(cxxrtlUint8);
console.log("cxxrtl.h file: ", cxxrtlString);

So really this just needs to be documented better; any command like write_file that internally uses extra_args() (which calls rewrite_filename on the argument, assuming it's not a heredoc-like parameter) can do this, too.

---

Honestly this completely handles my use case, and most others I could think of, so my desire to write a patch has wavered a bit. :) While having to repeat the filenames of the cxxrtl/ folder is a chore, it's not so bad. Globbing support would be nice, while I was thinking of it, I realized some care would need to be applied e.g. you may want to change the prefix of the copied directory (something like strip prefix in tar), or strip some files out that you otherwise wouldn't want matched by a normal glob. So it would basically need to still be a whole new command with some specialized matching logic, beyond rewrite_filename().

So, for right now this may be a case where I think the simpler, more basic approach is fine, and the extra baggage may not be worth the weight unless we want to rewrite it a bunch. Happy to close this if you agree.

[whitequark]

While having to repeat the filenames of the cxxrtl/ folder is a chore, it's not so bad.

Note that I reserve the right to introduce new headers in any Yosys commit, they are not a part of backwards compatibility contract. So you will definitely encounter that if you upgrade across Yosys versions.

[thoughtpolice]

Yes that's not a problem, at least for me; typically I just read patches directly anyway when I do updates for tooling like this, so it's not so bad (and in this case my build system driving this is hermetic with only a single version of yowasp, so failures like this will typically creep up very immediately and can be fixed)

[whitequark]

Ah! Yeah, that works then.