simonmichael / hledger

Robust, fast, intuitive plain text accounting tool with CLI, TUI and web interfaces.
https://hledger.org
GNU General Public License v3.0
3.05k stars 320 forks source link

Building on aarch64 #1268

Closed nobodyinperson closed 4 years ago

nobodyinperson commented 4 years ago

TL;DR

I made a Dockerfile to build hledger in the debian Docker container which is also available for aarch64:

https://gitlab.com/nobodyinperson/hledger-build-docker

Hi,

I am desperately trying to get an up-to-date version of hledger on my aarch64 machines (Raspberry Pi 4 and Pinebook Pro with Manjaro ARM for which there is no haskell available unfortunately). To be able to work I am currently extracting the hledger version 1.12.1 binary from the ubuntu or debian ARM Docker image, which is working fine. I was also able to use this Fedora Rawhide aarch64 RPM with a little hack (sudo ln -rsf /lib/libffi.so.7 /lib/libffi.so.6). But still, very outdated version of hledger...

The new Raspberry Pi image seems to be compiled for armhf (/lib/ld-linux-armhf.so.3), not aarch64, at least it doesn't work.

I then tried building from source using the debian Docker image and running these commands:

apt update
apt search cabal
apt install cabal-install
git clone https://github.com/simonmichael/hledger
cd hledger/
git checkout hledger-1.18.1
cabal update
cabal install alex happy
cabal install ./hledger-lib ./hledger

It takes pretty much ages, but then it errors out with

Failed to install hledger-lib-1.18.1
Build log ( /root/.cabal/logs/ghc-8.4.4/hledger-lib-1.18.1-ASU9vbNuzCaKNf5HAMq9eV.log ):
cabal: Entering directory './hledger-lib'
Configuring hledger-lib-1.18.1...
Preprocessing library for hledger-lib-1.18.1..
Building library for hledger-lib-1.18.1..
[ 1 of 50] Compiling Hledger.Utils.Color ( Hledger/Utils/Color.hs, dist/build/Hledger/Utils/Color.o )
[ 2 of 50] Compiling Hledger.Utils.UTF8IOCompat ( Hledger/Utils/UTF8IOCompat.hs, dist/build/Hledger/Utils/UTF8IOCompat.o )
[ 3 of 50] Compiling Hledger.Utils.Regex ( Hledger/Utils/Regex.hs, dist/build/Hledger/Utils/Regex.o )
[ 4 of 50] Compiling Hledger.Utils.Tree ( Hledger/Utils/Tree.hs, dist/build/Hledger/Utils/Tree.o )
[ 5 of 50] Compiling Hledger.Data.Types ( Hledger/Data/Types.hs, dist/build/Hledger/Data/Types.o )
[ 6 of 50] Compiling Hledger.Data.Period ( Hledger/Data/Period.hs, dist/build/Hledger/Data/Period.o )
[ 7 of 50] Compiling Hledger.Data.Json ( Hledger/Data/Json.hs, dist/build/Hledger/Data/Json.o )

Hledger/Data/Json.hs:126:10: error:
    Not in scope: type constructor or class `ToJSONKey'
    |
126 | instance ToJSONKey AccountType
    |          ^^^^^^^^^
cabal: Leaving directory './hledger-lib'
cabal: Error: some packages failed to install:
hledger-1.18.1-3Y8zH8nyreX1T62IIRDrRN depends on hledger-1.18.1 which failed
to install.
hledger-lib-1.18.1-ASU9vbNuzCaKNf5HAMq9eV failed during the building phase.
The exception was:
ExitFailure 1

Sorry if I got the terms wrong, I'm not all that familiar with cross-compilation and architectures...

But maybe I am wasting my time and there is in fact an aarch64 build somewhere out here which I just overlooked?

Cheers,

Yann

simonmichael commented 4 years ago

Hi @nobodyinperson. I hope we'll get some input on the binary situation soon, but let me investigate the compilation failure, which is not expected. I suspect an unexpectedly old version of aeson is being used. Could you report:

cabal --version
ghc-pkg list aeson
cabal exec -- ghc-pkg list aeson
simonmichael commented 4 years ago

Re binaries, https://raspberrypi.stackexchange.com/questions/107235/ubuntu-armhf-vs-arm64, https://stackoverflow.com/questions/37790029/what-is-difference-between-arm64-and-armhf, https://raspberrypi.stackexchange.com/a/87403/103374 seem informative.

simonmichael commented 4 years ago

I see we need to add a lower bound for aeson. Perhaps this works ?

cabal install ./hledger-lib ./hledger --constraint 'aeson >=1'
TheTaoOfSu commented 4 years ago

I can't vouch for building for the Pine 64, but I can explain how I built it on a Raspberry Pi 4. It can be done on an older Pi, but the 4 can get 2+ GB of RAM, which drastically cuts build time because it keeps a ridiculous amount of swap usage from happening, which uses the way slower HDD (that I used, SD cards aren't significantly better to my knowledge) instead of the fast RAM. I didn't watch the entire build since this took somewhere in the range of 4-6 hours of compilation total, but I saw it get up to 1,800 MB of RAM used, which is snug even on 2 GB, but it can at least cut out the vast majority of the swap usage.

Okay, so you've got a Pi 4. I used Raspbian, which I believe has recently been renamed Raspberry Pi OS. Due to the length of the build time and the frequently headless nature of a Pi setup, I recommend using screen so that you can detach and disconnect from SSH without killing the build process. My process wasn't perfect, but these are the commands I used to build it:

Debian packages are way out of date, so take note of the version you get from --version. The next two commands update you to a modern version of cabal. cabal --version cabal update cabal new-install Cabal cabal-install

Once it was done building a new version of cabal, my executable went to ~/.cabal/bin, so you need to either add that to your $PATH, or you need to specifically call that file when you try to use it. Since there's an older version of cabal present on the system, it needs if you add it to $PATH, it needs to be checked BEFORE the directory containing your Debian package version of cabal. Check your version to make sure you're actually calling the updated version of cabal, otherwise building hledger will fail. Also, note that using export to change your $PATH is only temporary, lasting only through the specific shell session it's used in. If you want to make a permanent change, you need to put it in .bashrc or your shell's equivalent. export PATH= ~/.cabal/bin:$PATH cabal --version

These last steps are what's currently advised on the download page on hledger.org, and I have copy/pasted it here because, now that you've updated cabal to a current version, those default commands work. Note that in the last line that builds hledger, it specifically states the version you want. I assume you want the latest version, so if anyone sees this several releases down the road, check the download page and potentially revise this. The --ghc-options='-split-sections' flag can be omitted, but it reduces total binary size to about a third. I didn't notice a huge difference in compile time when comparing building with and without the flag, so it seems worthwhile. May impact RAM and/or CPU usage, though. cabal update cabal install alex happy cabal install --ghc-options='-split-sections' hledger-1.18.1 hledger-web-1.18.1 hledger-ui-1.18.1

Now, I had a slight problem in that, despite knowing where to place the updated cabal binary, it didn't know where to put them for hledger for whatever reason. It recommended that I use a command line flag, I think either --installdir=/path/ or --install-dir=/path/, but when I attempted to redo it with that flag, it did not understand it. I had to manually retrieve my binaries from ~/.cabal/store/ghc-[version]/. My [version] was 8.4.4, so that directory is ghc-8.4.4 for me. Each of hledger, hledger-ui, and hledger-web had a folder named with their version number, as used in the install command above, followed by a random string. Rebuilding created new folders for me, so if you have to figure that out, I recommend checking file creation or access times. As with the updated version of cabal, you'll need to move them out or set your $PATH to include them. Given the folder names and structure, I found it much easier to copy them out and put them in a $PATH folder since there isn't anything for them to conflict with. Moving them to ~/.cabal/bin/ is a good idea.

I don't know the CPU architecture for the Pine 64, but if both of them are aarch64, you might be able to simply copy these compiled binaries onto the Pine 64. The binaries this build process produces don't require many if any libraries, so you should theoretically be just fine as long as the architecture is the same.

simonmichael commented 4 years ago

Thanks @TheTaoOfSu !

@nobodyinperson, you probably won't need to install a new cabal, try the one you have first. Or if you have stack, try that, it is free from those install/binary-finding shenanigans.

TheTaoOfSu commented 4 years ago

Thanks @TheTaoOfSu !

@nobodyinperson, you probably won't need to install a new cabal, try the one you have first. Or if you have stack, try that, it is free from those install/binary-finding shenanigans.

It might be worth a shot, but if he builds it directly on the Pi, he likely will have to update them. Raspbian/Raspberry Pi OS is Debian based, and the package is seriously behind. My Pi is fully updated with version 2.2.0.0, but the new version I built was 3.2.0.0. Maybe not, though, since he's trying to build directly from source. That might change something. Mine refused to work because a repository had moved and old cabal didn't know about it. I want to say that was not an hledger repository, so I would assume it was somewhere in the prerequisites for building.

If he's not building directly on the Pi, then something will likely have to change in the commands he's running. Nothing there indicates architecture. I know nothing about cross-compilation, though, so I can't comment on what would be necessary to address that.

Slack might work, but I know I had problems with Slack as well for the exact same reason, seriously out of date packaging. I've already gotten through the cabal process, so I can address the hangups there, but I have nothing I can offer for Slack except that there will likely be complications there, too.

simonmichael commented 4 years ago

We used to build hledger with cabal 1. given the cost of building things on these systems, it’s worth trying first.

nobodyinperson commented 4 years ago

Hey everyone,

Thanks for your replies.

My setup is the following:

Here some information:

# cabal --version
cabal-install version 2.2.0.0
compiled using version 2.2.0.1 of the Cabal library 
# ghc-pkg list aeson
/usr/lib/ghc/package.conf.d
    (no packages)
/root/.ghc/aarch64-linux-8.4.4/package.conf.d
    (no packages)
# cabal exec -- ghc-pkg list aeson
/usr/lib/ghc/package.conf.d
    (no packages)
/root/.ghc/aarch64-linux-8.4.4/package.conf.d
    (no packages)

But I'm not sure if this was before or after trying to compile...

I don't have much time now but will try to report back on the other approaches.

Cheers,

Yann

TheTaoOfSu commented 4 years ago

I compiled directly on Raspbian/Raspberry Pi OS, not using Docker at all, so I can't say for sure about doing it in Docker for the Pinebook, but I suspect it would work the same. Raspbian is just the latest version of Debian built for the Pi, so I would think the packages would be pretty similar. Your cabal --version output is identical to mine, so that's a good start.

Given that you're apparently missing a GHC package, I'm not sure a simple new Cabal build will fix it. I think Cabal can pull that in in the process, but you may need the --constraint 'aeson >=1' flag like @simonmichael said. My best guess at the moment is to give that a shot. I think Cabal should be able to pull in a new version of aeson if it finds its current version doesn't meet constraints. If Cabal attempts to do so but fails, that's what prompted me to build a new version of Cabal. The old version didn't know where to go to get the new versions of packages I needed for my build, so I had to rebuild Cabal so it could get the new details on those packages. There may be other, "better" ways to fix that, but if you don't know what they are, you have to balance trading compile time for your own personal time working on it. Learning is valuable, but sometimes, 2-3 extra hours of compiling time while I do something else is worth it to save an hour of my own time digging through logs and packages and Google to try to figure out what I'm missing.

nobodyinperson commented 4 years ago

Hm, I tried updating cabal as @TheTaoOfSu suggested, but (after a ridiculous amount of time) it always ends up completely freezing my Pi 4 to the point where I need to kill the power.

If I find the time, I'll try the aeson constraint.

TheTaoOfSu commented 4 years ago

How is your Pi set up? Are you trying to use the Docker image on it? Has the OS been initially installed recently, or is this an old setup that is being adapted? I used a Pi 4 with 4 GB like you did, but I ran into no problems. I set mine up... 2 months ago? So it was a relatively fresh install with totally updated packages. Can you see anything about why it's freezing up? I initially tried building on a Pi 3 that ran out of RAM and swap, but that just killed the process, not locked up the whole machine...

I'm inclined to say it's perhaps heat related? You're going to experience heavy CPU usage, so temperature will likely rise. Mine is in a case with a fan on it, so that might be making the difference by keeping it cool enough. If you're using the official case for the Pi 4, or at least what was the official case when the 4 released (can't say for sure it hasn't changed), that original official case retained heat fairly well, so my Pi got hot even at idle. If you can just uncover the top and hit it with some airflow, that might make a big difference. My cases, both the official case and the later fan cooled case I upgraded to, also came with a few little heat sinks to stick to a few key chips on the board, so if your case or anything came with something like that and you haven't installed them, I would recommend putting them on. Just clean the surface first. I'm not 100% sure on the best way to clean it, so I'm hesitant to give you potentially bad advice there.

TheTaoOfSu commented 4 years ago

My updated Cabal build did take a few hours to finish, though. Probably 2-4, can't remember exactly since it's been a couple weeks. Given identical hardware and software, you should be getting at least very similar results.

To cause a complete system lockup, I can only think of a few things. The first is heat, as I mentioned above. This will slow the processor down as it attempts to avoid cooking itself until it eventually can't help but enter the danger zone and take emergency action to prevent damage. In a standard desktop, this means powering down or perhaps a less likely reboot. Since the Pi isn't what I'm used to, I can't say that a total lockup isn't an unreasonable result of heat issues. It would cease processing a number of things, possibly everything, so it should reduce load and therefore heat production.

The second way is to cause something like a kernel panic, essentially a deep flaw in some software that the OS can't recover from. Given that everything on a Linux distro is typically installed by package, and package managers check their packages to make sure they're valid, you shouldn't be dealing with faulty software from anything here. Checking logs may help you figure out what is causing the crashes, but I don't know which logs to tell you to check. You might be able to trace it to something recently installed. However, even if all your software was installed properly, or if the failure is traced to software that hasn't been updated or changed in a good while, this leads me to...

The third way is your SD card failing. SD cards just aren't designed to handle the constant I/O of acting as a operating system's main disk. Eventually, the SD card will fail, and you'll start running into weird, otherwise inexplicable errors as software corrupts and fails to function properly. I'm not sure how to test them properly once they reach this point, to be honest, but it's pretty much inevitable. Of course, using it as a constant server is vastly different from booting it occasionally for a short, specific purpose, so time for a card to fail will vary wildly based on its usage and brand. I circumvent this by only putting my /boot partition on the SD card and putting root on an external hard drive. There are plans to eventually allow skipping the SD card altogether, but that's going to need a pending EEPROM update to finish development and get released.

I'm not sure if heat or failing SD card is more likely

nobodyinperson commented 4 years ago

Thanks for all of your explanations. My Pi 4 has a case and I built a PCB for PID-controlling the fan, so overheating won't be the issue. The SD-card didn't fail me before, it's a pretty new one...

I'm really lacking time right now unfortunately, I don't want to be rude, but I'll just leave it like this and work with the outdated hledger binary that I have until that's not possible anymore. I might try again if I find the time. Maybe someone else manages to build hledger for aarch64.

Thanks again!

nobodyinperson commented 4 years ago

Alright, I tried again allong the way (god it takes so long to compile),

and in the debian docker container I ran the following commands (I'm just dumping history here):

apt update && apt upgrade -y
apt search stack
apt install libtinfo-dev
wget -qO- https://get.haskellstack.org/ | sh
apt install wget
wget -qO- https://get.haskellstack.org/ | sh
apt install haskell-stack
stack update
stack update
wget  -qO- s3.amazonaws.com//hackage.fpcomplete.com/root.json
stack update
stack update
stack --version
stack upgrade --force-download
cabal update
cabal install alex happy
cabal install hledger-1.18.1 hledger-web-1.18.1 hledger-ui-1.18.1
apt install haskell-cabal
apt search cabal
apt install cabal-install
cabal update
cabal install alex happy
cd
ls
which alex
vim .bashrc
apt install vim
ls
vim .bashrc
source .bashrc
which cabal
which alex
which happy
cabal install --ghc-options='-split-sections' hledger-1.18.1 hledger-web-1.18.1 hledger-ui-1.18.1
cabal install --ghc-options='-split-sections' hledger-1.18.1 hledger-web-1.18.1 hledger-ui-1.18.1 --verbose
cabal install cabal install hledger-1.18.1 --verbose
apt install pkg-config
cabal update
cabal install cabal install hledger-1.18.1 --verbose
cabal install hledger-1.18.1 --verbose
cabal install hledger-1.18.1 --constraint 'aeson >= 1' --verbose
cabal new-install Cabal cabal-install
apt search libz
apt search zlib
apt install zlibc
aupt update
aupt update
apt update
apt install zlibc
apt install zlib1g zlib1g-dev
cabal new-install Cabal cabal-install
cabal install hledger-1.18.1 --constraint 'aeson >= 1' --verbose
cabal install hledger-1.18.1 --verbose
which cabal
cabal --version
cabal install hledger-1.18.1 --verbose
find -name cabal
ls -lh .cabal/bin/cabal
stack upgrade --force-download
stack update
stack install --resolver=lts hledger-lib-1.18.1 hledger-1.18.1
wget -qO- raw.githubusercontent.com//fpco/lts-haskell/master//lts-16.3.yaml
wget -qO- https://raw.githubusercontent.com/fpco/lts-haskell/master//lts-16.3.yaml
stack upgrade --force-download

So I couldn't install stack (the online installer complained) and the Debian package (as always... sigh) seems to be so out of date that it first didn't want to do anything, and then there seems to be no possiblilty to installi t for aarch64. So I managed to update cabal (interestingly, the version number didn't change cabal-install version 2.2.0.0 , okay...), but I never got past the following:

CallStack (from HasCallStack):
  die', called at ./Distribution/Client/Install.hs:993:15 in main:Distribution.Client.Install
cabal: Error: some packages failed to install:
aeson-pretty-0.8.8-3mVZHTF8v1LHycR6o9dgyB depends on aeson-pretty-0.8.8 which
failed to install.
base-compat-0.11.1-3vhoRUtlQ9WEcW1wHnaRnf failed during the final install
step. The exception was:
ExitFailure 1
base-compat-batteries-0.11.1-1ZG97AmjFRlJKu0EId3O0U depends on
base-compat-batteries-0.11.1 which failed to install.
clock-0.8-EbibcIhjzfxLSFEYBmutDS failed during the final install step. The
exception was:
ExitFailure 1
extra-1.7.3-Cw8Vgidf3gLFp3ahUdJblY depends on extra-1.7.3 which failed to
install.
hledger-1.18.1-3Ev2RS6WqLqGtdYNw6yMN9 depends on hledger-1.18.1 which failed
to install.
hledger-lib-1.18.1-7dyaQ3eBrGxImeZf53GrHc depends on hledger-lib-1.18.1 which
failed to install.
tasty-1.3.1-7KKJuJ4jNvR4hrWjdqXTH3 depends on tasty-1.3.1 which failed to
install.
tasty-hunit-0.10.0.2-HpBu0fKrGvQ6iJk66Y4T8F depends on tasty-hunit-0.10.0.2
which failed to install.

I don't know how to go on... It has to be possible to build hledger on aarch64, right? Is Haskell that stubborn?

Cheers,

Yann

simonmichael commented 4 years ago

Hi @nobodyinperson, I haven't seen a reply saying you tried my suggestion ?

If that doesn't work, would you like to visit hledger chat some time (https://hledger.org#help) and I'll help you get this built ?

nobodyinperson commented 4 years ago

Hi @nobodyinperson, I haven't seen a reply saying you tried my suggestion ?

I tried it as you can see in my history above:

cabal install hledger-1.18.1 --constraint 'aeson >= 1' --verbose

I guess that's the same as your suggestion, just not using the repository directly? Anyway, it errors out, I believe with the same error message as above.

If that doesn't work, would you like to visit hledger chat some time (https://hledger.org#help) and I'll help you get this built ?

I think I prefer „asynchronous” communication ATM :wink:

Shouldn't the manual build instructions just work? Do they on a normal x86_64 Debian machine (I don't have one at hand currently)? Is this an ARM-Debian issue? Or an issue with the Docker image? Otherwise, in what environment are these cabal build instructions supposed to work?

simonmichael commented 4 years ago

On Jul 2, 2020, at 12:15 PM, Yann Büchau notifications@github.com wrote:

I tried it as you can see in my history above: cabal install hledger-1.18.1 --constraint 'aeson >= 1' --verbose I guess that's the same as your suggestion, just not using the repository directly?

Ah, not necessarily. I've lost the context and can't say for sure. I think I prefer „asynchronous” communication ATM 😉

Ok, no problem. Unfortunately the fact is that building haskell packages can be highly complicated, with everybody's setup being different, so if it goes off the rails it can require familiarity and a bit of inspection to diagnose and fix correctly. And more so in a low memory situation. Shouldn't the manual build instructions just work? Do they on a normal x86_64 Debian machine (I don't have one at hand currently)?

Yes, everything on the download page should work as described. Of course I can never guarantee this 100% but I try hard to make it so, and would like to hear about any exceptions. Is this an ARM-Debian issue? Or an issue with the Docker image?

Do you mean the "ToJSONKey" error you reported ? That was my mistake in having too loose a bound on the aeson lib. This is a very old bug you are the first to discover, courtesy of the whims of cabal. My suggestion of adding --constraint 'aeson >= 1' was a workaround for this specific issue. Shortly after your report I fixed it in hledger master. Today I've made revisions to rule out this issue with the hledger, hledger-lib & hledger-web 1.18 and 1.18.1 releases on Hackage, also. Otherwise, in what environment are these cabal build instructions supposed to work?

The cabal build instructions are supposed to work on "all platforms where GHC/cabal are supported", assuming sufficient memory. They should work on your Pi.

nobodyinperson commented 4 years ago

Alright, I then suggest following this simple algorithm to find out what's the problem here:

while(doesntWork()) reduceComplexity();
while(doesWork()) increaseComplexity();

And obviously, either Docker (which I must use as Manjaro ARM doesn't provide Haskell) or ARM is the complexity here.

So I'd first like to assert that hledger can - in fact - be built in the standard x86_64 debian docker container, which is a proper assumption as Debian (and sorts) is such a widespread distribution.

I'd like to use the official haskell image (as you do in your Dockerfile), but it keeps failing for me saying this:

sudo docker pull haskell                                                                                                                                                                                
# Using default tag: latest
# latest: Pulling from library/haskell
# no matching manifest for linux/arm64/v8 in the manifest list entries

... which I don't understand as there are linux/arm64 haskell images on the Docker Hub. But then, the haskell image is itself based on Debian, so Debian should definitely work in general.

As I don't have an x86_64 machine at hand (funny, huh :smile:), could one of you Haskell experts please test what exact commands have to be executed in a debian container to get hledger compiled? So we can at least rule Docker out as a problem here?

If that works, I can execute the exact same commands on debian arm64 and see if that works. If it doesn't, then it's software weirdness again, I guess :sweat:

nobodyinperson commented 4 years ago

Okay, so I tried building hledger in the debian container on a x86_64 machine which I found.

No matter, whether I cabal new-install Cabal cabal-install or not, neither of the following commands work:

cabal install ./hledger-lib ./hledger  # from git repo
cabal install hledger-1.18.1 # that downloads from hackage??

It always fails with

Hledger/Cli/DocFiles.hs:44:6: error:
    * Exception when trying to run compile-time code:
        /root/hledger/hledger/embeddedfiles/hledger.info: hGetContents: invalid argument (invalid byte sequence)
      Code: embedFileRelative "embeddedfiles/hledger.info"
    * In the untyped splice:
        $(embedFileRelative "embeddedfiles/hledger.info")
   |
44 |     ,$(embedFileRelative "embeddedfiles/hledger.info")
   |      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
cabal: Leaving directory './hledger'
cabal: Error: some packages failed to install:
hledger-1.18.99-H0C5zirvQBz19W0gIdInRp failed during the building phase. The
exception was:
ExitFailure 1

This is new at least... Any clue?

simonmichael commented 4 years ago

thanks for persevering. I’m no docker expert; @adept do you know anything about that docker problem ?

The last error is a sign of progress. It usually means you need to set LANG, does the discussion at https://hledger.org/hledger.html#troubleshooting help ?

nobodyinperson commented 4 years ago

It usually means you need to set LANG, does the discussion at https://hledger.org/hledger.html#troubleshooting help ?

Yes, that fixes the build on x86_64. The following commands may thus be used to build hledger in the debian container:

export LANG=C.UTF-8
apt update
apt install cabal-install
cabal update
cabal install alex happy
cabal install hledger-1.18.1 
~/.cabal/bin/hledger --version
# hledger 1.18.1 

I am currently repeating this on my Pinebook Pro, let's see how far that gets me...

nobodyinperson commented 4 years ago

:tada: YES :tada: it worked! :tada:

The exact same commands worked on my Pinebook Pro!

So it was really just the LANG? And you @simonmichael fixed some aeson dependency constraint? And maybe there was a RAM issue before, on my PI it seems I didn't have any swap configured, that could explain the lockup...

For anyone not wanting to go through the hassle, here is my (gzipped because GitHub) hledger binary built for aarch64:

:paperclip: hledger.gz (this is just what came out of my Docker container, I am using this binary myself, but no guarantee for anything!)

It works on Manjaro ARM when you either:

Obviously, Debian (as always) uses outdated software, in this case an outdated libffi version 6, whereas Arch LInux is already at version 7.

Thanks everyone for the help!

Cheers,

Yann

simonmichael commented 4 years ago

@nobodyinperson congrats! Yes it sounds like a combination of those 3 issues (plus manjaro, making 4).

I guess not all systems ship with a UTF-8 locale by default even now. So based on your input I have added the annoying LANG issue at https://hledger.org/download.html#building-from-source.

Thanks for sharing your binary - it's hledger-1.18.1, I think ? I will add it to https://github.com/simonmichael/hledger/releases/tag/1.18.1.

Now for hledger-ui and hledger-web, right ? For hledger-ui you may need to install the tinfo-dev lib, per download page.

adept commented 4 years ago

Sorry, late to the party, but i can confirm, that LANG is a known issue - you can find it in the either Alex or happy bug tracker, don't remember which

nobodyinperson commented 4 years ago

Hi, thanks for adding the binary to the releases page.

However, something seems to be mixed up here:

1.18.1

hledger-armhf32-debian.gz - 32-bit Raspberry Pi hledger binary by @nobodyinperson. Built in a Debian docker image, can run on Manjaro.

I built an aarch64 binary of hledger 1.18.1 in side a Debian docker container on a Pinebook Pro running Manjaro ARM, not an armhf32 binary.

1.18

hledger-aarch64.debian.gz - 64-bit Raspberry Pi hledger binaries by @TheTaoOfSu. Built on a Pi 4 running Raspberry Pi OS 10 (buster), should work on Pi 2 or greater.

@TheTaoOfSu built an armhf32 binary on a Raspberry Pi, not an aarch64 binary.

simonmichael commented 4 years ago

Woah, thanks for that. I got super confused. Please let me know if any details are still wrong on these releases.

nobodyinperson commented 4 years ago

Could you please add my aarch64 build to the Downloads Page? Would fit nicely next to the Raspberry Pi section.

simonmichael commented 4 years ago

Done, thanks!

rainbyte commented 4 years ago

I'm testing a setup to build armv7 and armv8 binaries on amd64 machines using docker-hub arm images, maybe that kind of setup could be useful in the future? (I mean, to reproduce arm builds on amd64 machines).

simonmichael commented 4 years ago

I don't know much about it, but reproducible/automated process sounds good!

I wonder how many folks need hledger binaries on raspberry pi.

rainbyte commented 4 years ago

I think it is a general problem, because phones also use arm, and soon apple machines will use it too.

simonmichael commented 4 years ago

https://input-output-hk.github.io/haskell.nix/tutorials/cross-compilation/ looks useful.

rainbyte commented 4 years ago

https://input-output-hk.github.io/haskell.nix/tutorials/cross-compilation/ looks useful.

I'm trying to avoid nix because it is not in Arch repos and it seems to add more complexity. The ideal would be to have something like rustup targets but on stack.

nobodyinperson commented 3 years ago

Here is hledger version 1.20 for aarch64: hledger-1.20-aarch64.gz

simonmichael commented 3 years ago

Thank you! Added to https://github.com/simonmichael/hledger/releases/tag/1.20.

Perhaps we can find ways to provide more safety assurance for contributed binaries, later.

simonmichael commented 3 years ago

PS

I guess this should be added at https://hledger.org/download.html#raspberry-pi too ?

Should it replace the manjaro link there ? This one is Manjaro too ? I forget what "Manjaro" means for raspberry pi binaries.

Is this binary raspberry pi specific, or is it likely to run on other ARM machines ?

simonmichael commented 3 years ago

To make life simpler, I'd like to adopt apple's naming convention, "arm" instead of "aarch". Any objections ?

nobodyinperson commented 3 years ago

To make life simpler, I'd like to adopt apple's naming convention, "arm" instead of "aarch". Any objections ?

As arm64 and aarch64 mean the same thing, that should be no problem. The 64 is important, though.

Should it replace the manjaro link there ? This one is Manjaro too ? I forget what "Manjaro" means for raspberry pi binaries.

Manjaro does not need to appear in the name. Manjaro is just a (very nice) Linux distribution. The ARM 64-bit binaries I built here come from a Debian Docker container run from within a Manjaro host. They should run on any ARM 64-bit platform as long as the shared libraries are available. You may replace the current link hledger-aarch64-manjaro.gz on the Downloads Page so it points to the 1.20 Tag release page.

Perhaps we can find ways to provide more safety assurance for contributed binaries, later.

Good point. I don't really know much about compiled files. Is it possible to ”strip them down” and somehow assert that they do roughly the same? As in: ”Reduce” the standard x86_64-build of hledger and then also the arm64-build and then compare that? So people can verify that I didn't mess around with the source code to steal people's journal data :sunglasses:

I'm very glad that there even are arm64 Debian Docker containers, otherwise I would have to buy another Raspberry Pi 4 just to put a 64-bit Armbian on it (if that is even possible). It would be very nice if there were public arm64 build servers. GitLab CI's containers are all x86_64. I guess it will take a while for ARM container services to become available. I have a specific GitLab runner running on my RPi 4 at home, but then again, even if I managed to install the docker-executor there (which I haven't yet), it would still be a private machine...

It also takes an insane amount of time to build hledger. Yesterday the whole process took more than 2 hours on my Pinebook Pro. GitLab CI for example has a 1 hour timeout, so that might be a problem even if GitLab provided ARM build servers.