search

lavavu / LavaVu

Lightweight, Automatable Visualisation & Analysis Viewing Utility
Other
53 stars 12 forks source link

Build fails on piwheels #81

Closed waveform80 closed 4 years ago

waveform80 commented 4 years ago

NOTE: This may not count as bug, depending on whether you consider Raspberry Pis an important part of your general user-base (if not, apologies for the noise, please feel free to close this without comment):

We're currently observing repeated failure of this package on the piwheels builders. Having had a quick look at the setup.py this is most likely because it's attempting parallel builds based purely on CPU count (a common enough tactic). This is obviously fine on the vast majority of PCs (where CPU and RAM are typically fairly well matched), but on a Pi with 4 cores but only 1Gb of RAM (the piwheels builders are all, at present, Pi 3/3+ models) it kills the builder as swap overwhelms it.

The current versions of this package have been marked "don't build" and future versions are likely to be marked as such as well, unless the build script can be adjusted to better accommodate mismatched compilation platforms like ours. A typical heuristic I suggest (particularly when dealing with C++ builds) is to take min(num_cores, ram_size_in_gb), which on a PC with 4 cores and 8Gb of RAM will yield 4, and on a Pi with 4 cores and 1Gb of RAM will yield 1. Unfortunately, discovering RAM size in a cross-platform manner is non-trivial, and so another heuristic occasionally adopted is simply to determine whether the script is running on a Pi. A simple means of determining this is testing whether the content of /proc/device-tree/model starts with "Raspberry Pi". A bit hacky, but it gets the job done!

Anyway, as mentioned above this may not even constitute a bug depending on your audience, but we just wanted to let you know in case it's worth looking into.

Thanks for your attention,

The piwheels team.

OKaluza commented 4 years ago

Hi and thanks for the report... I'd be surprised we had any rpi based lavavu users, but it is meant to be a lightweight library and I don't like the idea of sending a machine into swap hell just with a package build, so will try and fix this. I'd prefer to do it properly rather than a pi only hack, but as you say it's not easy to reliably get ram size.

I don't know much about compile memory usage as I've never run into issues before. I'll do some profiling of the build when I get back home next week, I suspect the culprit will be the large single c file comprising the sqlite3 library amalgamation.