tr1ckydev / webview-bun

Bun bindings for webview, a tiny library for creating web-based desktop GUIs.
https://www.npmjs.com/package/webview-bun
MIT License
247 stars 11 forks source link

Why not Linux on ARM ? #16

Closed WebReflection closed 2 months ago

WebReflection commented 2 months ago

There is a "hard no" https://github.com/tr1ckydev/webview-bun/blob/main/src/ffi.ts#L26 for linux on ARM and I wonder what is the reason.

I also read you are working on something else now so if this is already abandoned please let me know so I can take it over from here, thank you!

edit ... uhm, OK, it looks like it's not even built so I guess that is the issue? https://github.com/WebReflection/webview-bun/blob/main/scripts/build.ts#L11

tr1ckydev commented 2 months ago

There is a "hard no" https://github.com/tr1ckydev/webview-bun/blob/main/src/ffi.ts#L26 for linux on ARM and I wonder what is the reason.

I tried lot of ways to compile it on my x64 host but couldn't get it to work. It would be really helpful if you could find out any way to compile it through webview's own cmake build system.

I also read you are working on something else now so if this is already abandoned please let me know so I can take it over from here, thank you!

That is not true, I don't have any plans to abandon this project. I try to keep this repository up to date with webview time to time.

edit ... uhm, OK, it looks like it's not even built so I guess that is the issue? https://github.com/WebReflection/webview-bun/blob/main/scripts/build.ts#L11

Yes, as I couldn't seem to find a way where I can compile for linux-arm64 from a x64 host. I would be glad if you could contribute to this issue.

WebReflection commented 2 months ago

With bun scripts/build.ts the platform is inevitably those already listed but I need to understand a few things here:

My idea is that I can get my Raspberry Pi and build this from source, then publish the artifact and do an MR.

That's not ideal as I'd need to keep that in sync, but the only alternative is to manage to build providing that arm64 target via gcc-unknown-arm64 target ... this might work too but I am on Linux, not sure about yourself.

I find the initial idea kinda better and easier to do, harder to keep in sync-maintain, but I don't mind automating that process per each new webview version, that project itself doesn't seem to change a lot over time, pretty stable as it is.

tr1ckydev commented 2 months ago
  • would it be OK to have both ../build/libwebview-x86.so and ../build/libwebview-arm64.so ?

Sure. This is how it was for both archs of darwin earlier.

  • would it be OK if in my fork I just build those and provide a pre-built arm64 file?

Right now, I would recommend only modifying the build script as a PR.

My idea is that I can get my Raspberry Pi and build this from source, then publish the artifact and do an MR.

The thing is I don't want to require a arm64 host only to compile. That's the last resort.

Does webview's cmake provide any actual way to cross compile for linux? Afaik i don't think we can. So it would be better to have the support there first, just like how they provide a darwin cross compilation toolchain which is being used currently in this repo.

WebReflection commented 2 months ago

Does webview's cmake provide any actual way to cross compile for linux?

I don't think such thing exists to start with ... the darwin file size is 3 times others too and it was their choice during x64 to arm64 migration but I believe they'll soon (relatively) remove dual target functionality.

Linux requires the target libraries to be compiled with that target in mind and the same is for Windows so if you are looking for a solution that works like it does in darwin you won't find it to the best of my knowledge.

Webview project uses whatever gcc or compiler it finds for that platform and I think one might pass an emulator for the arm64 variant.

As publisher of few ArchLinux ARM modules, I have used my Raspberry Pi 4 (or 5) to build and publish because on emulation it was both slower and sometimes I had other issues due tiny discrepancies among shared libs, so it's usually the safest, fastest, easiest, way to go.

WebReflection commented 2 months ago

you know what? before me trying to do anything you are right, I need to check if I can even get webview to compile on arm64 then maybe we can come up with something ... how are you instrumenting that build to use a different architecture though? just ia bun target flags?

tr1ckydev commented 2 months ago

you know what? before me trying to do anything you are right, I need to check if I can even get webview to compile on arm64 then maybe we can come up with something

Sounds great! Lmk how it goes.

how are you instrumenting that build to use a different architecture though? just ia bun target flags?

Sorry I can't seem to get the question. Currently the script can compile for win64, linux64 and universal for darwin. The host device os and arch is checked and according build commmands are triggered.

All the shared library files are provided in the npm package which helps for cross-compilation when creating a bun binary of your app.

WebReflection commented 2 months ago

Sorry I can't seem to get the question

How do you build those 3 variants out of that scripts/build.ts before you update this module?

tr1ckydev commented 2 months ago

Oh, it's kinda manual process. I build the .so file on linux and in windows after building the dll, i copy the file back to my linux and also got macos on vm from where i also transfer the generated dylib to linux and push them.

WebReflection commented 2 months ago

so ... you are doing what I've proposed about arm64 already ... I hence don't understand your reaction as "last resource" when you are doing exactly the same thing to provide those "blobs" everyone needs to trust you produced without CI automation or other kind of tools anyone could see and be sure what you provide can also be trusted (I am not suggesting you have ill intents in here, I am just sharing thoughts around pre-built binaries or compiled code and possible side-effects in the wild we've already seen in the Web and Server space).

WebReflection commented 2 months ago

there you go https://github.com/tr1ckydev/webview-bun/pull/17

to build I had to install Manjaro ('cause ArchLinux latest is broken) then update all the things then install gtk4 and webkitgtk-6.0 then ninja, base-devel, doxygen, than pass through qemu to have all the things available on my Linux Desktop. I don't know which linux distro you are running so I can't provide more details but the TL;DR is: webview compiles (with warnings that are kinda irrelevant) on arm64 too.

edit my outdated how-to to have qemu running is here https://gist.github.com/WebReflection/f98ab387a0cbde9230920ad6d16565aa but there are various things that are not covered in there ... that also assumes you are on ArchLinux by default, which is my daily distro to play around with.

WebReflection commented 2 months ago

Side-note: I haven't tested it works on my Raspberry Pi yet, but I will ... meanwhile, I believe that MR is the way to go in general when it comes to Linux target, risc-v might be next once bun runs in there too.

tr1ckydev commented 2 months ago

I hence don't understand your reaction as "last resource" when you are doing exactly the same thing to provide those "blobs"

There seems to be a misunderstanding, I wrote the last resort would be to require an actual arm64 linux host to compile because I wanted it to be compiled from a x64 host which is kind of possible as I had only got stuck on linking the arm64 libs which wasn't present in my system.

everyone needs to trust you produced without CI automation or other kind of tools anyone could see and be sure what you provide can also be trusted

I never really thought about that. Will surely look into it.

edit my outdated how-to to have qemu running is here

Thanks, I'll try it soon.

I haven't tested it works on my Raspberry Pi yet, but I will

Lmk how it goes.

WebReflection commented 2 months ago

I did update it but currently ArchLinux tar file is corrupted and produces errors so I ended up using Manjaro, then I've resized its ext4 partition and used that to qemu in there.

Instructions are similar but you need to add manjaro-arm beside archlinuxarm when you do the --init-key part

Use gtk4 and webkitgtk-6.0 + install git and base-devel and gcc or cmake and you should be good to go.

tr1ckydev commented 2 months ago

@WebReflection I was working on a different method of cross compiling. Basically unzipping Arch Linux ARM and using qemu-aarch to run those binaries and essentially try to build.

Use gtk4 and webkitgtk-6.0 + install git and base-devel and gcc or cmake and you should be good to go.

But I encounter this

pacman -S gtk4
resolving dependencies...
warning: cannot resolve "gstreamer=1.22.0", a dependency of "gst-plugins-base-libs"
warning: cannot resolve "gst-plugins-base-libs=1.22.0", a dependency of "gst-plugins-bad-libs"
warning: cannot resolve "gst-plugins-bad-libs", a dependency of "gtk4"

Also seems like the arm aur doesnt have webkit 6.0

> pacman -Ss webkit 6.0 

> pacman -Ss webkit 4.1
extra/webkit2gtk-4.1 2.38.5-1
    Web content engine for GTK
extra/webkit2gtk-4.1-docs 2.38.5-1
    Web content engine for GTK (documentation)

Also im unable to create proper arm64 qemu vms as they are either too laggy for some reason or dont work properly. If I can't reproduce your setup from my side, I wouldn't want to add support for arm64 just yet. 🙁

WebReflection commented 2 months ago

First step on ArchLinux/Manjaro is to init keys, populate keys then update via pacman -Syyu then you have, as mentioned already, gtk4 and webkitgtk-6.0 ... I have filed an MR it's very possible to do all this, it will be cumbersome for your first setup if you've never done that then it will be, from there, a pcman -Syu to keep it updated and that's it.

WebReflection commented 2 months ago

@tr1ckydev this is how I did it:

# on ArchLinux, be sure you have qemu-static
pacman -Sy --needed --noconfirm qemu-user-static

# Create an 8GB img file and install ArchLinux
dd if=/dev/zero of=arm64.img bs=1M count=8192
mkfs.ext4 arm64.img
mkdir -p ./arm64
sudo mount arm64.img ./arm64
curl -LO http://os.archlinuxarm.org/os/ArchLinuxARM-rpi-aarch64-latest.tar.gz
# you can ignore bsdtar errors ... it will work after all
sudo bsdtar -xpf ArchLinuxARM-rpi-aarch64-latest.tar.gz -C ./arm64
rm ArchLinuxARM-rpi-aarch64-latest.tar.gz
sudo cp /usr/bin/qemu-aarch64-static ./arm64/usr/bin
sudo mount -n -o remount,suid ./arm64

# Enter the arm64 world
sudo arch-chroot ./arm64 /bin/bash

# Setup and update your ArchLinux once
echo "
# setup a DNS
echo 'nameserver 1.1.1.1
nameserver 1.0.0.1'>/etc/resolv.conf

# use more space for the /tmp if needed
mount -o remount,size=2G,noatime /tmp
">>~/.bashrc
source ~/.bashrc

# Update ArchLinux to its latest
pacman-key --init
pacman-key --populate archlinuxarm
# this might take a while
pacman -Syyu --noconfirm

# Install other dev-dependencies
# this also might take a while
pacman -S --noconfirm --needed base-devel unzip cmake doxygen ninja graphviz git gtk4 webkitgtk-6.0

# Install Bun
curl -fsSL https://bun.sh/install | bash
source ~/.bashrc

# Grab the repo
git clone --recurse-submodules https://github.com/tr1ckydev/webview-bun
cd webview-bun
bun i

# Build after merging my MR
# https://github.com/tr1ckydev/webview-bun/pull/17
bun run build

# Profit

Next time you want to do the same:

# Get into the arm64 build
sudo mount arm64.img ./arm64
sudo mount -n -o remount,suid ./arm64
sudo arch-chroot ./arm64 /bin/bash

# Update the system
pacman -Syu --noconfirm

# Enter webview-bun
cd webview-bun
# rebase or update latest ... then ...
bun i
bun run build

That's it, you need to merge my MR first though or you'll be out of luck out of your current exclusion of the arm64 platform.

WebReflection commented 2 months ago

P.S. if you are worried about how much space left you can df / there and see a similar outcome:

df /
Filesystem     1K-blocks    Used Available Use% Mounted on
/dev/loop0       8154588 4338364   3380412  57% /

meaning a 6GB file might work the same ... up to you, I think with 8GB ArchLinyux can go extra miles already and for x86 targets, 8GB on disk is nothing, usually.

tr1ckydev commented 2 months ago

Just tried your method using qemu-static, ended up following this article mentioned in the arch wiki basically similar to your approach. Works just as I wanted. Compilation from a x64 host without needing to setup any virtual machine. Thank you so much!

Now the last thing before we finalize this is, how do I check if it actually works and opens a window. I believe I need a arm64 virtual machine(ubuntu 24.04 arm64) for that but im having trouble setting it up through virt manager. Have you tried doing such from your side?

WebReflection commented 2 months ago

It’s tricky due GPU drivers and stuff … I think the best final approach is to have your arm64 SBC ready to test that but also, once I can confirm it runs, you won’t need to do such check if your arm64 build works, let eventually real-world users inform you that’s not the case anymore but I think that won’t be the case in general (if I can confirm current build works but I need you to approve my MR and make next step easier on my side).

WebReflection commented 2 months ago

Thanks for the merge. Between today and tomorrow I should be able to verify this works on real arm64 platforms but if you could publish latest with the built file too it’d be easier to test. In the worst case scenario I’ll build it again from my Pi, test it in there and file another MR with just that .so file. The x86 was exactly the same so I don’t see reasons to delay a new minor / patch … and that would help, hoping I won’t need to install anything else than gtk4 and webkitgtk-6.0

tr1ckydev commented 2 months ago

you could publish latest with the built file too it’d be easier to test.

Do you mean npm? In that case, I would suggest you to simply clone this and run the example. Once you confirm everything works fine, I'll push to npm.

hoping I won’t need to install anything else

In one of the latest commits of webview, you require python3 to build.

WebReflection commented 2 months ago

works like a charm on my RPi

webview-bun

WebReflection commented 2 months ago

I just cloned and run the example without building anything 🥳

tr1ckydev commented 2 months ago

Wonderful! Thank you so much.