saber-notes / saber

The cross-platform open-source app built for handwriting
https://saber.adil.hanney.org
GNU General Public License v3.0
2.34k stars 152 forks source link

Packaging for the AUR #641

Open PolarianDev opened 1 year ago

PolarianDev commented 1 year ago

Hello world,

So I found saber via fdroid because it bumps recently updated, or added apps to the top of the store. I also realise that the app is not contained in any official arch repository, nor the AUR, so I intended to package it.

The reason I am making an issue to inform you of this is because I believe in communication between a package maintainer and upstream to ensure the highest quality builds possible.

I would be submitting saber to the AUR today, however due to dependency issues this can not be done.

Flutter is within the AUR, but the package is disfunctional and refuses to build any flutter application, due to the fact it constantly tries to connect to google servers, and pull stuff down which is not possible. Therefore, I must write up one (or more) patches to flutter before I can even build saber.

I do notice you provide an Appimage for Linux, I can use this to create a -bin package, but these are generally avoided for the sake of security and privacy, as nobody is able to verify the code compiled is not malicious when its in binary form, which is why source builds are preferred.

So for saber-bin I can most likely submit that by the end of the day, the appimage should contain a natively built executable.

If the package is voted, and popular, and a official package maintainer (Formerly TU) is willing to take the package on, it will be moved into the official repository where anyone can pull it directly through the use of pacman.

Have a good day, Polarian

PolarianDev commented 1 year ago

Also I left you a £2 donation via liberapay, probably nothing for a university student but its better than nothing, keep up the good work! :)

adil192 commented 1 year ago

Hi, I'm thankful for any donation, especially since it lets me publish to the App Store. Also if it's relevant, I recall that @fybx mentioned packaging for the AUR a while ago in

PolarianDev commented 1 year ago

hm I can't find it under the AUR though...

PolarianDev commented 1 year ago

image

I searched again, definitely not there... unless it was released under a different name?

adil192 commented 1 year ago

Yeah I don't know how far they got

PolarianDev commented 1 year ago

I will email them now and ask, thanks for letting me know :)

Midou36O commented 1 year ago

Any update regarding this? I'd love to use this app but i'm somewhat struggling with manually compiling it myself :)

PolarianDev commented 1 year ago

@Midou36O Hey,

I have got it building in a clean chroot, but I have not written up the PKGBUILD, I might be able to get it done tomorrow :)

Remember to vote for the package once I push it to the AUR :)

Polarian

PolarianDev commented 1 year ago

Ok some bad news,

I am currently struggling to get saber building with the package now.

There is currently no standard for packaging flutter software for Arch Linux, and due to issues I brought up about this a package maintainer gromit (formerly known as Trusted User) has drafted a WIP guideline for it:

https://wiki.archlinux.org/title/User:Gromit/Flutter_package_guidelines

However, its not fully working and I will need to speak to gromit (which has a working flutter package) to see if he can help get saber working.

Until then though, the appimage works perfectly on Arch Linux, but a distribution package would be far better and would eliminate dependency duplication which appimages have a lot of issues with.

@Midou36O I will let you know if I make any more progress on this issue.

Polarian

Midou36O commented 1 year ago

Alright, good luck! And thanks for your contribution. I will patiently wait then.

PolarianDev commented 1 year ago

I have been working with gromit to get a PKGBUILD draft which works, and I have now got a package which builds on my laptop, bare in mind this has not been tested within a clean chroot, and therefore could still break.

What is working so far is the package builds, however the simlink to /usr/bin is broken, which is a simple fix.

@adil192 Have you got a .desktop file anywhere here? or do I need to write one for the package.

It would help if upstream provides one along with the asset for the icon :)

Polarian

adil192 commented 1 year ago

@adil192 Have you got a .desktop file anywhere here? or do I need to write one for the package.

The .desktop file I use is at flatpak/com.adilhanney.saber.desktop: https://github.com/adil192/saber/blob/97fe2a6809936dd4c0a4babfef9ed093e87c1158/flatpak/com.adilhanney.saber.desktop#L1-L13

It would help if upstream provides one along with the asset for the icon :)

There's various sizes of icons at https://github.com/adil192/saber/tree/main/assets/icon/resized. If it helps, I use this small list of commands to copy them into the right place when I build the Flatpak: https://github.com/adil192/saber/blob/97fe2a6809936dd4c0a4babfef9ed093e87c1158/flatpak/com.adilhanney.saber.json#L51-L58

PolarianDev commented 1 year ago

Thats I can pull this from the source, might need to write a patch though to patch the .desktop to arch's liking, but apart from that this saves me a ton of time :)

adil192 commented 1 year ago

Thats I can pull this from the source, might need to write a patch though to patch the .desktop to arch's liking, but apart from that this saves me a ton of time :)

Could the patch be applied on this repository, or would that interfere with flatpak?

PolarianDev commented 1 year ago

I have already patched it, it is applied during the prepare stage, all it does is move the flatpak .desktop file to the base path of the source, and rename it to saber.desktop and renaming the icon file expected to saber, then the icon is copied during the build stage into the base directory of the source called saber.png, both are then installed to their respective paths.

I have pushed the first version to a git repository for testing:

https://git.polarian.dev/AUR/saber

however unfortunately when testing, a library libsuper_native_extensions.so: is not in the build directory along with the rest of the saber binaries, and once the build files are removed you get a loader error:

saber: error while loading shared libraries: libsuper_native_extensions.so: cannot open shared object file: No such file or directory

So I have to find this library and figure out where to stick xD

I guess time to ldd the saber binary.

PolarianDev commented 1 year ago

They are the plugins, and they are not a part of the bundle, and they are spat out in with the cmake files, now I got to figure out where to stick these libs in the install path, and then I also got to strip out the sources because its unnecessary.

I will speak to gromit tomorrow and see if he has any ideas.

@Midou36O Hopefully tomorrow you can have a test with the package, but tonight I don't think I will be able to get it working tonight, it still has not be successfully built in a clean chroot.

also after the package is pushed to the AUR then I got to document it on the archwiki, along with all the little issues, such as flutter demanding rustup and not cargo which means the toolchain needs to be pulled from upstream and not from the arch repository, which is a massive drain of bandwidth when I have a local copy of the arch repository for fast package testing and building.

So yes, still a lot to do!

Luciogi commented 1 year ago

Hi, I created PKGBUILD for arch linux https://codeberg.org/Luciogi/saber-pkgbuild I am not gona upload to AUR, because I don't want to maintain it. Anyone can take it and maintain it


Edit 1 : lol i did not checked , it is made by PolarianDev Edit 2: After reviewing https://git.polarian.dev/AUR/saber , my version had minor issue

PolarianDev commented 1 year ago

@Luciogi Yes gtk3 is required, this was something I discussed with gromit as I am also working on the flutter packaging guidelines.

if you ldd saber you will see it has been dynamically linked to gtk3 libraries, if these do not exist at runtime you will have a loader error, therefore gtk3 is a depend not a makedepend.

I have realised looking at yours that I have missed out some libs specified within README.md in saber's repository.

@adil192 What is the following libs used for (ubuntu packages, I see @Luciogi has used the arch equivalents):

libsecret-1-dev libjsoncpp-dev

Its always better to have dependency issues and then add the ones you need, instead of listing everything and half of it never being used, therefore pulling in tons of packages, but this is my personal belief, I am sure some people out there would lean on the side of caution.

@Luciogi As for rustup, unfortunately you must use it, flutter looks explicitly for the rustup binary, I have tried getting cargo to work, as cargo will pull in all the rust components directly from the Arch Repository which speeds up arch builds, as I got my own arch mirror on the build server and can pull at about 550MB/s (4.4gbps), when having to pull rust binaries this must be done from WAN and is a lot slower. I believe pkgbuild.com also has its own mirror for the same reason, using disk or LAN resources instead of flooding the WAN (and wasting bandwidth).

You should not configure rustup within the PKGBUILD, its outside the scope of the package, unfortunately until cargo or rust (ultimately the same package) works with building flutter apps, this step will need to be done manually, as rustup toolchain install stable will delete the toolchain and reinstall every time you build, which is not what we want.

@adil192 if it is your build scripts explicitly calling rustup, please let me know, I am aware super_clipboard is the library using rust.

Polarian

Luciogi commented 1 year ago

What is the following libs used for (ubuntu packages, I see @Luciogi has used the arch equivalents): libsecret-1-dev libjsoncpp-dev

libsecret would be used for managing username and password for nextcloud login linjsoncpp is used , as .sbn is actually json file

Its always better to have dependency issues and then add the ones you need, instead of listing everything and half of it never being used, therefore pulling in tons of packages, but this is my personal belief, I am sure some people out there would lean on the side of caution.

above libs are necesarry for runtime

You should not configure rustup within the PKGBUILD, its outside the scope of the package, unfortunately until cargo or rust (ultimately the same package) works with building flutter apps, this step will need to be done manually, as rustup toolchain install stable will delete the toolchain and reinstall every time you build, which is not what we want.

I did it because rustup and rust both packages are conflict , so choosing rustup should not effect usage of rust. As for deletion thing, rustup install toolchain at ~/.rustup/toolchains/ which remains installed , until user explicitly delete this directory

Luciogi commented 1 year ago

I noticing weird behavior, When building package and executing went fine app works after rebooting system , then running app reports missing of libsuper_native_extensions.so, though other libs are at same location /opt/saber/lib , this single .so file causing problem


Edit: Tried this solution https://aur.archlinux.org/packages/flemozi#comment-909654 but it is also not working What worked .. is to make symbolic link of /opt/saber/lib -> /usr/lib/

PolarianDev commented 1 year ago

Hello,

The solution linked there does actually work, I have tested it on my package.

The issue is that flutter doesn't expect the libraries to be in /opt changing the path for libraries works...

I have the solution in progress... bare with me please :)

Polarian

PolarianDev commented 1 year ago

I am still waiting for Adil to verify both dependencies above are needed, I could ldd saber to see if they are dynamically linked, if so they are needed at runtime.

Apart from that I will probably be able to push the package to the AUR within a few hours.

Polarian

PolarianDev commented 1 year ago

@Luciogi I would also like to point out that we do not create the .desktop file unless it is not contained within upstreams repository.

A simple patch makes it usable within AUR, as you can see in my repository.

Polarian

Luciogi commented 1 year ago

yes. I not did got idea to see .desktop file in flatpak directory xd


Tried this solution https://aur.archlinux.org/packages/flemozi#comment-909654 but it is also not working

using $LD_LIBRARY_PATH works for me

PolarianDev commented 1 year ago

yes. I not did got idea to see .desktop file in flatpak directory xd Well Adil did make a post explaining it :/

using $LD_LIBRARY_PATH works for me As it should

I will fix the package shortly, then you and @Midou36O can test it, if you are up to it?

Luciogi commented 1 year ago

yup I m with you

PolarianDev commented 1 year ago

@Luciogi @Midou36O I have got it building, under annoying circumstances, so you must follow the following build steps to work around issues with flutter:

  1. clone the repository
  2. install flutter, and rustup
  3. run: git config --global --add safe.directory /opt/flutter to prevent flutter error, this is a bug within the flutter package.
  4. run: rustup toolchain install stable
  5. run: rustup default stable
  6. finally makepkg and it should build successfully

Let me know if you have any issues, I have installed this to my system and it works fine.

Thanks for the help, Polarian

Midou36O commented 1 year ago

Looks like it's failing here following your guide.

Error: The Flutter directory is not a clone of the GitHub project.
       The flutter tool requires Git in order to operate properly;
       to install Flutter, see the instructions at:
       https://flutter.dev/get-started

And yes i followed the 3rd step properly, using root (idk if that was a good or bad idea 👀 ) and user.

PolarianDev commented 1 year ago

Looks like it's failing here following your guide.

Error: The Flutter directory is not a clone of the GitHub project.
       The flutter tool requires Git in order to operate properly;
       to install Flutter, see the instructions at:
       https://flutter.dev/get-started

And yes i followed the 3rd step properly, using root (idk if that was a good or bad idea eyes ) and user.

This is another issue with the flutter aur package.

Also you should never build with root, makepkg should stop you however, so I do not know why it did not stop you here...

to fix this, simply:

mv "/opt/flutter/.git-refs" "/opt/flutter/.git"

This was taken from the flutter repository:

https://aur.archlinux.org/packages/flutter#comment-916623

Good luck, Polarian

Midou36O commented 1 year ago

Okay now i get this

patching file saber.desktop (already renamed from flatpak/com.adilhanney.saber.desktop)
Analytics reporting disabled.

You may need to restart any open editors for them to read new settings.
Setting "enable-linux-desktop" value to "true".

You may need to restart any open editors for them to read new settings.

And then it fails during the preparation :p

(Note: i didn't run the makepkg command as root, it was the git command.)

PolarianDev commented 1 year ago

@Midou36O just type y when it promts you.

Midou36O commented 1 year ago

Hmm, those don't really ask to input a command. Anyway, i checked the PKGBUILD file and proceeded to manually run the commands, looks like this specific command is causing issues.

flutter --no-version-check pub get 
[ble: exit 66]

Exit code: 66, searching this on google it says that apparently flutter cannot access some sort of directory due to permissions issues (i sure do hope it's not because of the git command i ran as root).

I deleted .pub-cache and tried doing the same thing, it just took longer before throwing the "66" exit code again.

PolarianDev commented 1 year ago

Hmm, those don't really ask to input a command. Anyway, i checked the PKGBUILD file and proceeded to manually run the commands, looks like this specific command is causing issues.

flutter --no-version-check pub get 
[ble: exit 66]

Exit code: 66, searching this on google it says that apparently flutter cannot access some sort of directory due to permissions issues (i sure do hope it's not because of the git command i ran as root).

I deleted .pub-cache and tried doing the same thing, it just took longer before throwing the "66" exit code again.

This is when you have not run the following command:

git config --global --add safe.directory /opt/flutter

Midou36O commented 1 year ago

image I did run this command as you instructed :p

Luciogi commented 1 year ago

run build by appending -v in flutter command to get full logs

Luciogi commented 1 year ago

Build successfully :) with your PKGBUILD


@PolarianDev append --prefixed-errors in flutter build command, it would show errors to console

PolarianDev commented 1 year ago

Build successfully :) with your PKGBUILD

@PolarianDev append --prefixed-errors in flutter build command, it would show errors to console

sure, I will add that now

PolarianDev commented 1 year ago

now I got to rebuild in clean chroot 😞

PolarianDev commented 1 year ago

@Midou36O if you still can't build it please read below!

I have deployed the final version of saber (I am about to push it to aur) to my unofficial arch repository, which you can see on ArchWiki, website contains instructions.

The official repository will allow you to pull saber directly through pacman, my repository is signed so you will need to import my signature, which is on the website link on the archwiki.

I will also document the build steps and troubleshooting issues on archwiki when creating the saber page.

Enjoy!

Polarian

PolarianDev commented 1 year ago

Link to saber on aur: https://aur.archlinux.org/packages/saber

PolarianDev commented 1 year ago

Link to ArchWiki page (WIP): https://wiki.archlinux.org/title/Saber

Midou36O commented 1 year ago

@Midou36O if you still can't build it please read below!

I have deployed the final version of saber (I am about to push it to aur) to my unofficial arch repository, which you can see on ArchWiki, website contains instructions.

The official repository will allow you to pull saber directly through pacman, my repository is signed so you will need to import my signature, which is on the website link on the archwiki.

I will also document the build steps and troubleshooting issues on archwiki when creating the saber page.

Enjoy!

Polarian

Yep, the repo worked, although i had to use TrustAll for your repo :S. But saber runs perfectly fine now, thanks a lot!

PolarianDev commented 1 year ago

@Midou36O if you still can't build it please read below! I have deployed the final version of saber (I am about to push it to aur) to my unofficial arch repository, which you can see on ArchWiki, website contains instructions. The official repository will allow you to pull saber directly through pacman, my repository is signed so you will need to import my signature, which is on the website link on the archwiki. I will also document the build steps and troubleshooting issues on archwiki when creating the saber page. Enjoy! Polarian

Yep, the repo worked, although i had to use TrustAll for your repo :S. But saber runs perfectly fine now, thanks a lot!

You shouldn't need to my documentation states you need to run the following commands:

curl https://git.polarian.dev/AUR/polarrepo/raw/branch/master/pkg/keys/polarian.asc | sudo pacman-key --add -

You can substitute sudo with doas, or remove it entirely if ran as root. This command will fetch my key from my repository (gpg keyservers are not the most reliable, and this allows for a one line command).

Once pacman gpg store has my key, you then need to locally sign it trusting that it is valid:

pacman-key --lsign-key 0770E5312238C760

And yes, this is my valid key if you are wondering xD

After that it should be able to verify the authenticity of all files within the repository along with the package database (which is also signed for security)

Midou36O commented 1 year ago

I opted for the easiest road since both locally signing it and using trustall for the specific repo were not recommended hehe. I proceeded to do it properly as you instructed me to do.

And regarding the app: image B)

PolarianDev commented 1 year ago

I would like a link to where it says local signing is not recommended, I know it is not advised because Arch Linux staff will not take responsibility for any abuse by third party repositories, but TrustAll is not recommended because you can not verify the authenticity of the files you are downloading, this is especially important if you use http instead of https (but I assumed you use https like I have listed on my repository, because there is little reason not to).

And I am glad the package works fine :)

PolarianDev commented 1 year ago

^^^ Feel free to submit any issues with the package to the repository: https://git.polarian.dev/AUR/saber/issues

Unless its a known issue with upstream :)

Also dont forget to vote for the aur package so its bumped up so more people can find the package.

Midou36O commented 1 year ago

I would like a link to where it says local signing is not recommended, I know it is not advised because Arch Linux staff will not take responsibility for any abuse by third party repositories, but TrustAll is not recommended because you can not verify the authenticity of the files you are downloading, this is especially important if you use http instead of https (but I assumed you use https like I have listed on my repository, because there is little reason not to).

And I am glad the package works fine :)

Looks like you're looking for this https://wiki.archlinux.org/title/Pacman/Package_signing#Signature_is_unknown_trust (4th mitigation option.)

PolarianDev commented 1 year ago

I would like a link to where it says local signing is not recommended, I know it is not advised because Arch Linux staff will not take responsibility for any abuse by third party repositories, but TrustAll is not recommended because you can not verify the authenticity of the files you are downloading, this is especially important if you use http instead of https (but I assumed you use https like I have listed on my repository, because there is little reason not to). And I am glad the package works fine :)

Looks like you're looking for this https://wiki.archlinux.org/title/Pacman/Package_signing#Signature_is_unknown_trust (4th mitigation option.)

Unfortunately there is no way around this, see:

https://wiki.archlinux.org/title/Pacman/Package_signing#Adding_unofficial_keys

PolarianDev commented 1 year ago

Ok I have finished the wiki as best as possible for now, I will ask for it to be reviewed to see if I made any minor errors.

AUR package I wlil also ask gromit to review to see if he can find any potential issues, apart from that I don't think there is much more to do here other than maintenance.

Luciogi commented 1 year ago

@PolarianDev close this issue