Archive size quadrupled from 1.15 to 1.16 release

[SteffenDE]

Hello there,

I noticed that the size of the nerves artifact went from 237 MB to 985MB from release 1.15.1 to 1.16. Is there any reason for this? Compared to other nerves systems this seems quite high.

Thanks!

[fhunleth]

Whoa, I hadn't noticed that. That is a surprise since the other systems didn't do that - or at least the ones I use didn't do that.

Thanks for letting us know. We'll take a look when we can to see what's up with this.

[fhunleth]

I took a look and I think it's related to Mesa3D. Here's /usr/lib/dri before:

$ ls -las nerves_system_rpi4-portable-1.15.1/staging/usr/lib/dri
total 287440
    4 drwxr-xr-x 2 fhunleth fhunleth     4096 Apr 22 11:25 .
   12 drwxr-xr-x 8 fhunleth fhunleth    12288 Apr 22 11:25 ..
11976 -rwxr-xr-x 1 fhunleth fhunleth 12263208 Apr 22 11:25 armada-drm_dri.so
11976 -rwxr-xr-x 1 fhunleth fhunleth 12263208 Apr 22 11:25 exynos_dri.so
11976 -rwxr-xr-x 1 fhunleth fhunleth 12263208 Apr 22 11:25 hx8357d_dri.so
11976 -rwxr-xr-x 1 fhunleth fhunleth 12263208 Apr 22 11:25 ili9225_dri.so
11976 -rwxr-xr-x 1 fhunleth fhunleth 12263208 Apr 22 11:25 ili9341_dri.so
11976 -rwxr-xr-x 1 fhunleth fhunleth 12263208 Apr 22 11:25 imx-dcss_dri.so
11976 -rwxr-xr-x 1 fhunleth fhunleth 12263208 Apr 22 11:25 imx-drm_dri.so
11976 -rwxr-xr-x 1 fhunleth fhunleth 12263208 Apr 22 11:25 ingenic-drm_dri.so
11976 -rwxr-xr-x 1 fhunleth fhunleth 12263208 Apr 22 11:25 kms_swrast_dri.so
11976 -rwxr-xr-x 1 fhunleth fhunleth 12263208 Apr 22 11:25 mcde_dri.so
11976 -rwxr-xr-x 1 fhunleth fhunleth 12263208 Apr 22 11:25 mediatek_dri.so
11976 -rwxr-xr-x 1 fhunleth fhunleth 12263208 Apr 22 11:25 meson_dri.so
11976 -rwxr-xr-x 1 fhunleth fhunleth 12263208 Apr 22 11:25 mi0283qt_dri.so
11976 -rwxr-xr-x 1 fhunleth fhunleth 12263208 Apr 22 11:25 mxsfb-drm_dri.so
11976 -rwxr-xr-x 1 fhunleth fhunleth 12263208 Apr 22 11:25 pl111_dri.so
11976 -rwxr-xr-x 1 fhunleth fhunleth 12263208 Apr 22 11:25 repaper_dri.so
11976 -rwxr-xr-x 1 fhunleth fhunleth 12263208 Apr 22 11:25 rockchip_dri.so
11976 -rwxr-xr-x 1 fhunleth fhunleth 12263208 Apr 22 11:25 st7586_dri.so
11976 -rwxr-xr-x 1 fhunleth fhunleth 12263208 Apr 22 11:25 st7735r_dri.so
11976 -rwxr-xr-x 1 fhunleth fhunleth 12263208 Apr 22 11:25 stm_dri.so
11976 -rwxr-xr-x 1 fhunleth fhunleth 12263208 Apr 22 11:25 sun4i-drm_dri.so
11976 -rwxr-xr-x 1 fhunleth fhunleth 12263208 Apr 22 11:25 swrast_dri.so
11976 -rwxr-xr-x 1 fhunleth fhunleth 12263208 Apr 22 11:25 v3d_dri.so
11976 -rwxr-xr-x 1 fhunleth fhunleth 12263208 Apr 22 11:25 vc4_dri.so

Here's after:

$ ls -las nerves_system_rpi4-portable-1.16.1/staging/usr/lib/dri
total 2322260
    4 drwxr-xr-x 2 fhunleth fhunleth     4096 Jul  6 22:54 .
   12 drwxr-xr-x 9 fhunleth fhunleth    12288 Jul  6 22:54 ..
96760 -rwxr-xr-x 1 fhunleth fhunleth 99079952 Jul  6 22:54 armada-drm_dri.so
96760 -rwxr-xr-x 1 fhunleth fhunleth 99079952 Jul  6 22:54 exynos_dri.so
96760 -rwxr-xr-x 1 fhunleth fhunleth 99079952 Jul  6 22:54 hx8357d_dri.so
96760 -rwxr-xr-x 1 fhunleth fhunleth 99079952 Jul  6 22:54 ili9225_dri.so
96760 -rwxr-xr-x 1 fhunleth fhunleth 99079952 Jul  6 22:54 ili9341_dri.so
96760 -rwxr-xr-x 1 fhunleth fhunleth 99079952 Jul  6 22:54 imx-dcss_dri.so
96760 -rwxr-xr-x 1 fhunleth fhunleth 99079952 Jul  6 22:54 imx-drm_dri.so
96760 -rwxr-xr-x 1 fhunleth fhunleth 99079952 Jul  6 22:54 ingenic-drm_dri.so
96760 -rwxr-xr-x 1 fhunleth fhunleth 99079952 Jul  6 22:54 kms_swrast_dri.so
96760 -rwxr-xr-x 1 fhunleth fhunleth 99079952 Jul  6 22:54 mcde_dri.so
96760 -rwxr-xr-x 1 fhunleth fhunleth 99079952 Jul  6 22:54 mediatek_dri.so
96760 -rwxr-xr-x 1 fhunleth fhunleth 99079952 Jul  6 22:54 meson_dri.so
96760 -rwxr-xr-x 1 fhunleth fhunleth 99079952 Jul  6 22:54 mi0283qt_dri.so
96760 -rwxr-xr-x 1 fhunleth fhunleth 99079952 Jul  6 22:54 mxsfb-drm_dri.so
96760 -rwxr-xr-x 1 fhunleth fhunleth 99079952 Jul  6 22:54 pl111_dri.so
96760 -rwxr-xr-x 1 fhunleth fhunleth 99079952 Jul  6 22:54 repaper_dri.so
96760 -rwxr-xr-x 1 fhunleth fhunleth 99079952 Jul  6 22:54 rockchip_dri.so
96760 -rwxr-xr-x 1 fhunleth fhunleth 99079952 Jul  6 22:54 st7586_dri.so
96760 -rwxr-xr-x 1 fhunleth fhunleth 99079952 Jul  6 22:54 st7735r_dri.so
96760 -rwxr-xr-x 1 fhunleth fhunleth 99079952 Jul  6 22:54 stm_dri.so
96760 -rwxr-xr-x 1 fhunleth fhunleth 99079952 Jul  6 22:54 sun4i-drm_dri.so
96760 -rwxr-xr-x 1 fhunleth fhunleth 99079952 Jul  6 22:54 swrast_dri.so
96760 -rwxr-xr-x 1 fhunleth fhunleth 99079952 Jul  6 22:54 v3d_dri.so
96764 -rwxr-xr-x 1 fhunleth fhunleth 99079952 Jul  6 22:54 vc4_dri.so

Quite a few of these aren't even needed (rockhip, stm, sun4i, st*, pl*, imx1, etc.). We currently have the ones we need selected in Buildroot (vc4_dri, v3d_dri, swrast). Unfortunately, I don't know enough aboutmesa3d` to feel comfortable erasing the unnecessary ones. I'd want to submit upstream for second review.

The other part is why did they grow from 12MB each to 99MB and even why they're so large in the first place. Them all being the same size is also weird. I'm guessing that the growth is partly due to debug symbols being enabled in the 1.16.x builds to enable stack traces.

[x4lldux]

Any update on this? All those files are exactly the same, in fact, when building the system from sources, all those files are hard-linked together. Size overhead should be possible to be fixed at the build-system - does CircleCI's filesystem supports hard links?

[fhunleth]

I haven't looked at it since the previous post. I also don't know whether hard links are supported on CircleCI's filesystem. Perhaps an easy win would be to just rm the unneeded files in the post-build.sh. I assume that only one is needed, but I don't know which one (maybe v3d_dri.so or vc4_dri.so?).

[x4lldux]

Both Mesa docs^1 and an official blog post about opensource drivers^2 mention V3D is used on RPi4 while VC4 on RPi 0-3. Maybe to play it safe, instead of deleting unneeded files, post-build.sh would replace them with symlinks to v3d_dri.so .

[x4lldux]

Something like this https://github.com/x4lldux/nerves_system_rpi4/commit/4d34d0b93b6693710d0c50f2d612e9c167ce33b4. It's not tested on RPi yet, though it compiles and archive size is back to 218MB.

[fhunleth]

@x4lldux That seems good to me, and I don't understand why it wouldn't work. Please send a PR when you've tested or let us know if you need someone using Mesa3D to test.

[fhunleth]

Fixed by #124