AShiningRay commented 2 years ago

Hi there, i caught attention of this emulator a while ago and started playing around with it, mostly because it has a Retroarch core. The standalone one is fine, but i'd rather keep all my emulators on a single frontend so i started focusing on how to build it for Retroarch on linux and in a few minutes i got it working in there, but after testing it, i found out it lacked some really important features and also had some problems, those being:

Closing a content running in freej2me closes the entire frontend
The save locations aren't working very well, the game config files are saved in a folder alongside the saves, screenshots and others, while the rms folder is saved at the root of where you opened Retroarch (so if i open RA through a symlink in my desktop, it creates a rms folder there)
The core doesn't expose any input remap options at all (at least it didn't here). On a joypad it's easier to get the inputs down, but on a keyboard it's hard to know since RA already uses some keys by itself.
The core also doesn't have any of its game config variables exposed to the frontend, such as screen resolution, rotation, etc. I know it has a button combo to bring the builtin menu up, but that's not very intuitive.

With that in mind, i set out to try to fix those problems myself since i deemed it a good side project to get myself reacquainted with java, launch args and whatnot.

Currently i managed to solve almost everything i noted above, however, there's a few things left to iron out, but i'll note all the changes below:

The core no longer closes the entire frontend when a "close content" call is issued. However, this brought up another problem: The frontend freezes if a "Restart" command is issued. I'm thinking of finding a way to disable the support for that function in the core since it isn't very useful for java games, and some changes in the core require a complete restart of the core itself. Fixed, also applies core configs after restarting.
The save locations are working as intended now, freej2me saves both the config and rms folders into "saves/freej2me/" on the Retroarch directory. I had to make some changes to the paths in the java files, but so far the standalone versions still work as intended at least on my linux machine, creating the folders at the emulator's root.
The input mappings are now exposed to the libretro API, allowing remaps and also giving the user some reference as to which button or key is related to what.
Screen Res, Rotation and Control Types are exposed in the core's settings, but i left out fps and sound because one doesn't seem to make any difference on the games i have, and the other isn't particularly useful when the emulator is running through libretro's frontend. It's also not very clean in the way it was implemented since i pass those as arguments to the java file and process them in the Libretro.java and Config.java files, which mandates a core restart on every change made in RA's menu... but it's what i managed to come up with so far.

Some of those pics are from previous builds, so you'll likely see some things i removed. Anyway, due to those changes, i disabled the button combo that brings up the emulator's menu inside RetroArch. That one is located inside freej2me_ibretro.c so it doesn't seem to affect the standalone version, which still works just as well as it did before on my PC, however, i didn't test the RA core on windows, ~~nor the standalone build on it~~ (standalone on windows 10 works fine too), but i plan to do it soon whenever i have the time.

The commits have more info on what i did, so please refer to them if any extra info is needed (and to check if the code is in a reasonable state to be used in the main branch): https://github.com/AShiningRay/freej2me/commits/master-dev

I also found some other minor things like the incorrect pointer coordinates when the screen is rotated, and the possibility of freezing the frontend when the builtin menu's "exit" option is selected. I listed the individual PRs below.

Sorry for the wall of text.

vadosnaprimer commented 2 years ago

If you've already done it, it's mostly safe to send pull requests with individual features. Even if it's not finished completely, the PR can be marked as a draft while things are being reviewed. That gives more incentive for more people to actually look into it.

AShiningRay commented 2 years ago

Sure, i'll get a new branch up and try to push the PRs for each feature (some aren't very detached from the others though, but should be doable overall).

Edit: Alright, the first PR is up.

imkrut commented 2 years ago

Can this be used with Retroarch on Windows?

recompileorg commented 2 years ago

I did, at one point, build the core under MinGW (using MSYS2). I've never personally tried it. I wrote the AWT frontend to run this on Windows and desktop Linux as I don't particularly care for the Retroarch UI.

AShiningRay commented 2 years ago

Can this be used with Retroarch on Windows?

With some major adjustments, yes. From what i tested, compiling the core with MinGW and MSYS2 results in a failure while trying to load it through Retroarch's windows frontend. Making the core build natively for windows is one of my goals, but i'm postponing it until the linux core is in a better state overall, no sense trying to port it so soon when the ifdefs might make the code much more complicated to read and debug while the features are still in development (it's either that, or making a separate core file for windows, which is also not ideal).

Quick edit: To be more precise about the adjustments : Things like fork() and pipe() for example need to be translated to their closest windows counterparts, and some of those calls are pretty different on windows when compared to linux.

imkrut commented 2 years ago

Been waiting (literally) years for squirreljme to be functional on retroarch, would be such a happy camper if a windows core ( I know many people would be too) was released, also as the Java catalogue is fairly rich and quite underrated (and at times hidden behind a difficult control scheme that Retroarch can take care of), I feel this would open up the library to lots of people.

If there's any way I can help to make this possible, I'd be glad to help.

AShiningRay commented 2 years ago

The retroarch core was one of the reasons i was drawn to this project, as well as it being the only active project that has a native linux build while being reasonably simple to understand and manipulate with my current skill level. So i also stand by the possibility of serving a lots of people with a good windows RA core, even if i won't use it myself (but i do know some people that will, so i have some ways of testing things on windows).

Now, about that help statement, i think there's many ways one can improve this project:

Well, the project is open-source, so if you have the skills to develop freej2me in any front, you can make changes and submit pull requests for review, i doubt anyone would turn them down if they're beneficial, you might just have to make some adjustments to have them fit better in the main repository. If coding isn't your cup of tea, i think it would be very helpful to test as many j2me games as possible and build a good database with what works, what doesn't, and in those cases, why it doesn't. Knowing why some games have problems even if on a surface level is a great way of helping improve compatibility. My repo's wiki has a lot of games that i have tested to varying degrees of compatibility, which is where i'm basing some of my PRs from.

At the end of the day, i can't speak for every other dev on freej2me, but to me, simply having people interested in the project is enough of a motivator to keep going, so if you can't help it directly, that's not really a problem.

imkrut commented 2 years ago

Awesome, will take a look at the wiki and contribute how I can, since unfortunately I am not code-savy.

Thank you for your (and the others) work! Hopefully we can enjoy the core sometime soon on Windows, and eventually some of the fantastic features like Savestates, rewind, or even retro achievements through Retroarch

AShiningRay commented 2 years ago

Since this one was split into multiple Pull Requests to make things easier to track, i'll be including the individual Pull Requests below:

140 (merged) -> Helps the core to shutdown and restart without freezing the whole frontend (Isn't bulletproof though, games that have an "exit" option and FreeJ2ME crashes still freeze the frontend, but the workarounds to those warrant some discussions, because the handling of "game exits" is standard behavior for example).

141 (merged) -> Exposes input data to the frontend, allowing input remaps, and also giving the user some info about which buttons are mapped to what in FreeJ2ME.

142 (merged) -> Allows the libretro core to save game configs and rms files correctly, without interfering on the standalone versions.

150 (merged) -> Fixes the pointer coordinates when the game's screen is rotated.

151 (merged) -> Disables inputs that call the config menu built into freej2me-lr.jar. We'll be using the libretro frontend for that, and closing the core through that menu also results in freezes. Better safe than sorry, given the pain that force-closing the libretro frontend is on some systems.

157 (merged, it's the one i talk about below) -> Fixes mouse drag events on libretro, and also adds support for screen rotation on AWT. I was dumb and forgot that my last messages about the core_option updates were already using this one with the exposed pointer options.

175 (merged) -> This is the one where i intend to expose FreeJ2ME configs like screen res, rotation, fps, etc to the frontend. I'll introduce this PR when the ones above get merged, because not only it makes some fundamental changes to the core, but also relies on things like the correct save paths for testing. I also want to redo it from scratch, so while it might work the way it is now, it's not very clean, and is also limited in functionality by requiring a core restart to apply any kind of change.

AShiningRay commented 1 year ago

Been a while since i last updated this issue, but there's been some progress.

For starters, i've reworked how the core exposed and applied its settings through RetroArch's frontend.

I updated the libretro headers and made sure that the core used the newer functions, so now it is pretty much in line with libretro's newest CORE_OPTIONS, allowing us to organize settings into categories and give descriptions for each:

Besides those enhancements, the core can now apply any option changes from the frontend instantly, not requiring a restart anymore, which means that, at least on linux, the core is basically fully fledged compared to standalone. If anyone's willing to try it, the code resides on my dev-clean branch.

That branch also has my other pull requests on this main repo loaded in, meaning that, as they are now, they shouldn't cause issues. But in any case i'll wait for them to be accepted here (or rejected, who knows) before bringing yet another PR, so if you guys have any requests for those other PRs, please tell me.

After this issue is closed i'll be moving onto porting the core to win32, and i already have some progress there although the core still doesn't boot any game yet despite being built natively and using win32 APIs instead of POSIX ones.

imkrut commented 1 year ago

Looking forward to trying this out on Retroarch (windows)!

imkrut commented 1 year ago

Any shot of a Christmas release? absurdly hyped about this

recompileorg commented 1 year ago

I'll do my best to get though the backlog so that @AShiningRay can submit a pull request for this before Christmas. It's been a very busy year and things haven't slowed for the holidays. I'll try to make special time for it later today or tomorrow.

AShiningRay commented 1 year ago

Oh, nice to see that those commits were merged, it's almost all set to bring the major core overhaul, except that i was too busy those last few months and forgot to add that issue #157 is also a requirement to bring that next PR since the big rewrite i've shown above actually depends on it.

Bear in mind that once #157 gets reviewed, and hopefully merged without issues, the next PR will only bring the core overhaul for linux, as i'm rather stuck with the win32 port because windows' child process creation and pipe functions differ way too much from unix, and i'm not really that knowledgeable on win32 system calls to understand why it isn't working as i thought it would.

Nevertheless, i did make some progress and the core actually attempts to load freej2me-lr.jar with the command-line arguments and pipe setup but it just doesn't work as expected yet, although i think i'll leave the details to a future issue (or an existing one, such as #92 which seems close enough), as i'm pretty sure i'll need some help.

AShiningRay commented 1 year ago

Okay, just noticed that all the pull requests related to this issue were merged... great!

Unfortunately i'm pretty busy this weekend so i can't guarantee the final PR to close this issue will come that soon. Still, i'll try to make some time for it anyway, as it's about time we got the RetroArch core up-to-date, at least on Linux distros.

imkrut commented 1 year ago

That's great news! Is there a tentative date for a Windows based core yet? Keep up the great work friends.

AShiningRay commented 1 year ago

Not yet, looks like i'll have to rework the whole core (again) to figure out a cleaner way of dealing with win32 debugging and api calls as a whole. So far the core builds natively on win32 (with mingw32 to use the current makefile for now) but fails to show any signs of life on retroarch despite freej2me-lr.jar being successfully initialized and running in the background. It also doesn't output any meaningful info yet and this is not a situation where the fix can be found by guesswork.

But i'd rather wait for this issue to be closed first, then i can work on the win32 port on top of everything that's already done and in the master branch.

AShiningRay commented 1 year ago

Good news everyone! Managed to get it working natively on Win32... Had to get my old laptop and convince myself to put up with windows, but it's finally done and we can close this issue soon. Unfortunately, it really depends on the merging of #175 since it's another rather far-reaching overhaul on the core.

Screenshot

Took a long time, i know... Been rather short on time this year. But on the upside, it's already pretty much on par with the linux core after #175 gets merged.

recompileorg commented 1 year ago

This is great! I understand being short on time. Thanks for this. Your contributions are always appreciated.

imkrut commented 11 months ago

Is there a public version available of the core to try out on win? thanks a lot for the efforts everyone! I really appreciate it!

AShiningRay commented 11 months ago

Is there a public version available of the core to try out on win? thanks a lot for the efforts everyone! I really appreciate it!

Not really, no. You'd have to build it by yourself... although lemme see if i still have a build lying around.

AShiningRay commented 11 months ago

Is there a public version available of the core to try out on win? thanks a lot for the efforts everyone! I really appreciate it!

Huh, turns out i do have one... a bit old however, and won't match the latest state of those trees 100%. Still, it should be perfectly functional.

Here you go, has a matching jar and dll to make sure things won't randomly break due to version mismatch: FreeJ2ME_Win.zip

imkrut commented 11 months ago

I love you @AShiningRay

Edit: Retroarch just closes after loading the content (freej2me-lr,jar went into /system folder and freej2me_libretro.dll to /cores folder)

Thanks anyway friend, I'll keep checking hopefully there's a public release for stupid people like me down the road.

AShiningRay commented 11 months ago

I love you @AShiningRay

Edit: Retroarch just closes after loading the content (freej2me-lr,jar went into /system folder and freej2me_libretro.dll to /cores folder)

Thanks anyway friend, I'll keep checking hopefully there's a public release for stupid people like me down the road.

Do you have java installed? That is a requirement to run this core (and the standalone app as well), and the version needed depends on which one was used to build the freej2me jar... it's been a while since i built that jar i sent, so i don't really remember if i built it using Java 8 or Java 11 (hell, even Java 19) for internal testing, sorry.

First i'd check if running "java -jar" on windows' cmd returns something other than "command not found" or similar, because that's what the core uses to fire up the jar.

Still, there is still a long way to go on this core, haven't even checked with Libretro/Retroarch's heads to see if it can be added to the official list of cores yet, and it'd certainly help standardize what the user needs to get it running.

imkrut commented 11 months ago

I love you @AShiningRay Edit: Retroarch just closes after loading the content (freej2me-lr,jar went into /system folder and freej2me_libretro.dll to /cores folder) Thanks anyway friend, I'll keep checking hopefully there's a public release for stupid people like me down the road.

Do you have java installed? That is a requirement to run this core (and the standalone app as well), and the version needed depends on which one was used to build the freej2me jar... it's been a while since i built that jar i sent, so i don't really remember if i built it using Java 8 or Java 11 (hell, even Java 19) for internal testing, sorry.

First i'd check if running "java -jar" on windows' cmd returns something other than "command not found" or similar, because that's what the core uses to fire up the jar.

Still, there is still a long way to go on this core, haven't even checked with Libretro/Retroarch's heads to see if it can be added to the official list of cores yet, and it'd certainly help standardize what the user needs to get it running.

You were right, forgot I didn't have java installed in the pc I was trying it out.

You guys did wonders on the core, already looking so promising and working great! can't wait til core features like save states or fastforward are implemented, but even as it is now it's a freaking blast.

Hopefully this will make some of the hidden gems on the platform more visible to players, thank you again.

Obviously will keep looking forward to updates!

hex007 / freej2me

The retroarch core is kinda barebones. Possible solution here. #138

141 (merged) -> Exposes input data to the frontend, allowing input remaps, and also giving the user some info about which buttons are mapped to what in FreeJ2ME.

142 (merged) -> Allows the libretro core to save game configs and rms files correctly, without interfering on the standalone versions.

150 (merged) -> Fixes the pointer coordinates when the game's screen is rotated.

151 (merged) -> Disables inputs that call the config menu built into freej2me-lr.jar. We'll be using the libretro frontend for that, and closing the core through that menu also results in freezes. Better safe than sorry, given the pain that force-closing the libretro frontend is on some systems.

157 (merged, it's the one i talk about below) -> Fixes mouse drag events on libretro, and also adds support for screen rotation on AWT. I was dumb and forgot that my last messages about the core_option updates were already using this one with the exposed pointer options.