Closed MvBonin closed 1 year ago
coderedart
commented
1 year ago I dont know if wayland even has support for this feature. I am currently working on some dependency crates like glfw while waiting for mlua 0.9 release which will allow me to release jokolay with scripting/plugin support. I am just procrastinating as we already have burrito for linux people anyway.
I will update jokolay to use all the latest dependencies and check if wayland works (atleast on kde).
MvBonin
commented
1 year ago Ah haha Burrito is not working on Wayland either.. Do you mean this https://github.com/khvzak/mlua ? What will they do with 0.9 that you would rely on? What is the current state of Jokolay? Do you think it would be theoretically possible to put a marker addon like jokolay or Taco completely inside a d3dx9.dll or d3dx11.dll?
coderedart
commented
1 year ago Ah haha Burrito is not working on Wayland either..
I don't know if jokolay will work on wayland. I think it should because i have it patched https://github.com/coderedart/glfw-sys-passthrough/blob/master/src/wl_window.c#L1324 , but i will test it later today.
Do you mean this https://github.com/khvzak/mlua ? What will they do with 0.9 that you would rely on?
There's a gui library called egui (like imgui that arcdps uses, but in rust). and I wanted to enable lua addons to have their own gui too like blish plugins. you can see how it would work roughly with rhai_egui (a tiny proof of concept). but mlua currently expects me to implement the trait, which i cant do due to orphan rules (as egui types are not mine). mlua 0.9 will allow me to register types manually instead (like rhai allows), which will make it super easy to have addons/gui using luaegui.
What is the current state of Jokolay?
In terms of just barebones marker functionality. it would just take a day to polish and release it. if we want scripting, then we are kinda blocked on mlua 0.9 (I will ask the dev how long it might take). I am also waiting for rend3 releasing a huge upgrade in Feb, according to their matrix, which will allow me to push 99% of marker rendering work to it instead of using wgpu directly.
Do you think it would be theoretically possible to put a marker addon like jokolay or Taco completely inside a d3dx9.dll or d3dx11.dll?
that would actually be much easier than this overlay method that taco/blish/burrito/jokolay use. but you are entering a grey area with dll files. overlays are completely independent apps, so it is ToS safe. and i also don't want to bring down game due to a crash inside the overlay. and if your overlay is slow, it will also bring down gw2 fps. you also have to play nice with other dll addons like arcdps. and ofcourse, you can only run that while gw2 is on.
I do have some progress made in a local branch somewhere, i will see if i can atleast release a version with markers functionality and then wait for mlua. meanwhile, i can instead polish the overlay i.e. deal with everyone complaining about their favorite window manager not working.
MvBonin
commented
1 year ago I don't know if jokolay will work on wayland. I think it should because i have it patched https://github.com/coderedart/glfw-sys-passthrough/blob/master/src/wl_window.c#L1324 , but i will test it later today. Thanks! Since lots of linux users use wayland nowadays this will be nice in the future :) If you need someone to test it, i would love to help you!
There's a gui library called egui (like imgui that arcdps uses, but in rust). and I wanted to enable lua addons to have their own gui too like blish plugins. you can see how it would work roughly with rhai_egui (a tiny proof of concept). but mlua currently expects me to implement the trait, which i cant do due to orphan rules (as egui types are not mine). mlua 0.9 will allow me to register types manually instead (like rhai allows), which will make it super easy to have addons/gui using luaegui.
Egui looks really nice and would be a cool thing to use I think! Would it be possible to implement Lua Addons in a way that we can just use existing Blish Addons in Jokolay? Or would we write our own ones? Also using the gw2 api with addons is a really nice thing! The Idea of powerful addons is very nice and having good tools is a huge advantage. Many ppl wouldn't play the game without blish, taco or gw2efficiency.. I also like the idea of having a kind of repo inside the Overlay Program, where the user can just install Addons from git and a functionality that upgrades existing ones..
In terms of just barebones marker functionality. it would just take a day to polish and release it. if we want scripting, then we are kinda blocked on mlua 0.9 (I will ask the dev how long it might take). I am also waiting for
rend3releasing a huge upgrade in Feb, according to their matrix, which will allow me to push 99% of marker rendering work to it instead of using wgpu directly.
That sounds really nice! I would love to test it and maybe help you somehow.. I never used rust until now but I think it's the next thing i'm learning.
that would actually be much easier than this overlay method that taco/blish/burrito/jokolay use. but you are entering a grey area with dll files. overlays are completely independent apps, so it is ToS safe. and i also don't want to bring down game due to a crash inside the overlay. and if your overlay is slow, it will also bring down gw2 fps. you also have to play nice with other dll addons like arcdps. and ofcourse, you can only run that while gw2 is on.
Ah I see!
I do have some progress made in a local branch somewhere, i will see if i can atleast release a version with markers functionality and then wait for mlua. meanwhile, i can instead polish the overlay i.e. deal with everyone complaining about their favorite window manager not working.
I'm looking forward to test the marker version for you, if that helps!
coderedart
commented
1 year ago Would it be possible to implement Lua Addons in a way that we can just use existing Blish Addons in Jokolay?
we have to write our own. blish addons are based on C#/dotnet core (mono engine). and blish has a custom UI thing to match in game ui. Ideally i wanted javascript or python, but we don't have a decent library for embedding them like mlua. wasm is not yet ready :(
where the user can just install Addons from git
one good thing about me working on this for so long is that, i have experimented with a bunch of things. including git repositories for marker packs :) I used libgit2 rust crate, which was not ideal, but still good enough. the main problem is that github (and other public hosts) have a rate limit for git operations (60 ish per sec). so, users will have to add a client api key to jokolay, which is way beyond what we can ask from casual players (even if they use arch! xD).
I also thought about using a database like sqlite or postgres. or as static websites (cloudflare provides unlimited bandwidth). and p2p methods like webrtc etc.. for now, static hosting seems like the easiest option. example: https://packlist.jokolay.com/packlist.json . and jokolay will get the latest json file at startup, and check if the installed marker packs or addons are latest. if not, we will prompt for update/download.
I'm looking forward to test the marker version for you, if that helps!
Nice. i will see if i can get this done by friday or saturday.
coderedart
commented
1 year ago still need to add basics like progress bar for marker pack downloads, or a menu to select categories or polishing rendering etc.. but the basics are working atleast. will probably take a little longer than expected, because i basically slept through my whole weekend.
MvBonin
commented
1 year ago That looks very nice to build on! Is this the latest commit? Then I would love to test it!
coderedart
commented
1 year ago This took a while. Anyway, rend3 doesn't seem to be releasing a new version anytime soon. So, i will just do manual rendering with wgpu for now.
Wayland is unsupported (can't create transparent vulkan windows with wayland it seems). If someone can figure that out, we can also build/publish a wayland version.
Although, i can still use jokolay on kde wayland session due to xwayland.
MvBonin
commented
1 year ago Hi. I am testing 0.2 release right now and it works on GNOME Endeavour OS with wayland. Only issue is, that it doesn't find GW2 in fullscreen/full window and displays on my second screen. When using windowed gw2, it works perfectly and i have to say: Its very nice!!
coderedart
commented
1 year ago @MvBonin you can try debugging it by opening mumble manager window (JKL button -> enable mumble manager).
As the below screenshot shows, you can find the gw2 size/position as client size/pos. Then, you can find out why jokolay moves to second screen.
Although, fullscreen/windowed fullscreen may not work anyway, because according to x11 spec, fullscreen windows should always be on top when focused. or gw2 capturing mouse cursor etc..
There is a hack where we can mark jokolay window as "dialog" window of gw2, which would force it to always stay on top even in fullscreen. But that was disabled when i was attempting pure wayland support. For now, the best option is to have window mode gw2 with borders disabled (I use kwin rules for this), and to resize gw2 to almost fullscreen size.
Pure wayland is definitely possible. But, transparency is very hit or miss on vulkan + wayland. So, we need to shift to openGL first (in progress).
Right now, i am kinda experimenting with a custom gui library based on skia (with openGL) which would enable wasm scripting (along with luau). The new gui library will allow all sorts of cool effects like blur and animations eg: https://skia.org/docs/user/modules/skottie/ .
And with wasm plugins, i can keep jokolay small and develop faster. While pushing features like marker packs or other modules as plugins, which can be updated independently. The reason i couldn't do markers as plugins before was that luau would be too slow. But now, wasm + rust with a jit would gets us almost near native performance.
once we get skia + opengl based UI working, i can simply release a jokolay-x11 and a jokolay-wayland version.
MvBonin
commented
1 year ago Ok! I tried it out, when in windowed fullscreen, the mumble manager displays client pos 0, 0. client size 1920, 1080. The gw2 window is on the left screen, jokolay on the right one.
When changing to window mode, that changes a bit but displays nearly the same values. Jokolay then shows on the left screen too.
When changing back to windowed fullscreen, the same thing as before happens, jokolay right, gw2 left.
When I change gw2 to window, move it to the right screen and put it in fullscreen mode, Jokolay is displayed correctly, but the client pos shows 1920, 0. When doing this, i get keyboard passthrough / focus issues.
edit: When i change to borderless window, it seems to work on the right screen, the issues were on fullscreen.
coderedart
commented
1 year ago That is definitely weird. Unfortunately, i don't have my second monitor hooked in atm. But, even if we fixed the monitor bug, we still are going to deal with the focus problem anyway.
As i mentioned in my previous comment, when you set to fullscreen, gw2 tries to capture the pointer (cursor) and won't release it. But Alt + Tab will break focus, and release the pointer capture from gw2. But if you alt-tab to jokolay, then gw2 is not focused, so until you click on gw2 again, keyboard events will go to jokolay.
This was actually the first issue #1 on this repo. The best workaround (for now) is just sticking to window mode, disabling borders and resizing it to almost fullscreen.
On windows, this led to another bug, where if gw2 was fullscreen making jokolay also resize to fullscreen, transparency would be disabled xD. So, i make sure to always be just 1 pixel smaller than gw2. This overlay stuff is really just a bunch of hacks.
MvBonin
commented
1 year ago On accident I changed to fullscreen and had the described issues, on windowed fullscreen it works. When changing to windowed, moving to the right screen and changing back to borderless windwo, it works.
Jokolay then shows the x coordinate of the client pos as 1920. When I have the borderless window on the left screen, the coordinate is 0 but Jokoklay seems to use the 0 coordinate of the wrong (the right screen).
coderedart
commented
1 year ago coordinates are global. So, your top-left monitor will start with (0,0). Then, because it is FHD 1920x1080, the width will be 1920. The monitor to the right of that will start from (1920, 0) as its top left point. So, the coordinates of gw2 shown are correct.
And jokolay will just reuse gw2's coordinates. So, if gw2 is at (0, 0), then jokolay will aim for (0, 0).
can you check jokolay's logs. It should be in your user data directory (usually ~/.local/share/jokolay/jokolay.log unless you changed xdg dirs/env). We also log when gw2's size changes or jokolay resizes to make debugging easier.
I am also in gw2 dev community discord btw. we have a jokolay channel, and it could be faster to debug this there instead of github issues. on discord's left, in the server list, at the bottom, there's a explore discoverable servers button. click -> search for gw2 dev and join gw2 development community server
With lots of distros coming in wayland nowadays we should target wayland too and make it run there.
Also maybe a List of working and todo Stuff could be added to the README.md or in https://github.com/coderedart/jokolay/projects so that ppl who want to contribute can do so more easily. Is this even still developed?