Implement NRO loading - Githubissues

I've been working on a proof of concept for loading NROs through an NRO for the last few days. You can see it here. I'm making this issue so that I can dump my knowledge here and for the possibility of help.

The way the homebrew menu loads NROs is by using envSetNextLoad and then returning. This simply tells hbloader what NRO to load next, and requires the current NRO to exit. This would not be ideal for nxsh, where you would want to keep the connection while loading an NRO so that you could still send commands afterwards. It also would return to hbmenu after the NRO is loaded, unless the loaded NRO also uses envSetNextLoad. So, in short, this won't work for us.

hbloader never stops running until you close whatever it's running though (i.e. the album). This seems like a step up from using envSetNextLoad, but it also has its drawbacks. If nxsh were to use my proof of concept as is, it might be able to launch an NRO and keep running, but it would never be able to stop launching NROs. It could launch itself after it's done launching NROs without any assistance, but there would need to be some way of detecting that an NRO just launched so that the user doesn't have to establish a connection and enter their password again. That seems hacky though, and is worse than envSetNextLoad since we would be starting another instance of nxsh without exiting the first one. There is also the fact that I was unable to figure out how to not override the libnx functions appInit, appExit, and __libnx_initheap. It might not end up being a big deal, but I think it would be cleaner if that weren't necessary.

So, if we could get hbloader to exit, that would be great. But it's not that simple. In fact, hbloader is so adamant about not exiting that it calls the error 2347-0008 if you somehow get it to stop launching NROs (which I have not been able to fully do). The reason hbloader never returns is that it calls loadNro, which eventually calls nroEntrypointTrampoline (marked as a function that doesn't return so the compiler knows no code after it can be reached), an assembly function. That function resets the stack, calls the NRO, saves the return value of that NRO, resets the stack again, and calls loadNro, creating an infinite loop. If you return from loadNro after it's been called from nroEntrypointTrampoline though, you get an error, since it's simply branched to, and not branched with a link. But if you branch with a link, you still get an error, and I have tried everything I can think of (storing the link register at the beginning so that we can return there once loadNro returns, saving the stack to a buffer on the heap and loading that buffer back in once nroEntrypointTrampoline returns, among some other small things that didn't work), and the closest I have gotten is getting to one printf after nroEntrypointTrampoline, and then it gives up with a 2168-0002 error. If you call exit from nroEntrypointTrampoline once loadNro returns, then it will return to the normal hbloader, but then we might as well be using envSetNextLoad, as nxsh would have to exit once it's done loading NROs.

There is also the problem of the kip version. If loadng NROs through a kip is even possible, I don't think there would be enough memory. As I mentioned in #9, sysmodules have a pretty limited memory, and many NROs will extend over the allotted memory for nxsh (2.5 MB). And even if you could put the NRO on the heap, it would have basically no heap left for itself. All that pretty much ensures NRO loading would be exclusive for the NRO version.

There is also the smaller problem of what to do with stdio. Ideally, nxsh would be able to redirect stdio to the socket so that the user could see all the printf's and stuff, but looking at the code for nxlink and twili, it seems like redirecting the stdio has to be explicitly done by the loaded NRO. nxsh could do it like twili does, by having a service running that listens for NROs trying to link their stdio, but that seems like a lot of overhead for something like nxsh.

So there are a few options: we could settle for simply using envSetNextLoad at the cost of having to exit, we could do it the way hbloader does it so that we never have to exit, but at the cost of never being able to exit, or we could wait until I or someone else figures out how to make my proof of concept exit normally without throwing an error, and cannibalize that. Obviously the last one is most ideal, but that could simply be impossible, so I think the next best thing would be to use envSetNextLoad, or possibly a combination of envSetNextLoad and my proof of concept, where my proof of concept always loads nxsh (maybe from /nxsh/nxsh.nro ?) and then nxsh can use envSetNextLoad to tell my proof of concept to load another NRO before eventually loading nxsh.

Cesura / nxsh

Implement NRO loading #10