search

hashbang / aosp-build

A build system for AOSP based roms optimized for determinisim, customization, and automation.
MIT License
42 stars 13 forks source link

[Rebuild only] e2fsdroid: Could not allocate block in ext2 filesystem while populating file system #9

Open ypid opened 4 years ago

ypid commented 4 years ago

I now hit this issue the third time. This only happens when I make changes in the source code and the then do make build again. Any idea how to solve this instead of deleting out? A clean build does not have this issue.

2019-11-22 18:45:03 - build_image.py - ERROR   : Failed to build out/target/product/sargo/product.img from out/target/product/sargo/product
Out of space? Out of inodes? The tree size of out/target/product/sargo/product is 214323200 bytes (204 MB), with reserved space of 0 bytes (0 MB).
The max image size for filesystem files is 232579072 bytes (221 MB), out of a total partition size of 232579072 bytes (221 MB).
Traceback (most recent call last):
  File "./build/tools/releasetools/build_image.py", line 789, in <module>
    main(sys.argv[1:])
  File "./build/tools/releasetools/build_image.py", line 781, in main
    BuildImage(in_dir, image_properties, out_file, target_out)
  File "./build/tools/releasetools/build_image.py", line 423, in BuildImage
    BuildImageMkfs(in_dir, prop_dict, out_file, target_out, fs_config)
  File "./build/tools/releasetools/build_image.py", line 334, in BuildImageMkfs
    mkfs_output = common.RunAndCheckOutput(build_command)
  File "/home/build/build/base/build/make/tools/releasetools/common.py", line 249, in RunAndCheckOutput
    args, proc.returncode, output))
common.ExternalError: Failed to run command '['mkuserimg_mke2fs', '-s', 'out/target/product/sargo/product', 'out/target/product/sargo/product.img', 'ext4', 'product', '232579072', '-j', '0', '-D', 'out/target/product/sargo/system', '-L', 'product', '-i', '510', '-M', '0', '-c', '--inode_size', '256', 'out/target/product/sargo/obj/ETC/file_contexts.bin_intermediates/file_contexts.bin']' (exit code 4):
18:44:43 mkuserimg_mke2fs.py INFO: Env: {'MKE2FS_CONFIG': './system/extras/ext4_utils/mke2fs.conf'}
18:44:43 mkuserimg_mke2fs.py INFO: Running: mke2fs -O ^has_journal -L product -N 510 -I 256 -M /product -m 0 -E android_sparse -t ext4 -b 4096 out/target/product/sargo/product.img 56782
18:44:43 mkuserimg_mke2fs.py INFO: Env: {}
18:44:43 mkuserimg_mke2fs.py INFO: Running: e2fsdroid -p out/target/product/sargo/system -s -S out/target/product/sargo/obj/ETC/file_contexts.bin_intermediates/file_contexts.bin -f out/target/product/sargo/product -a /product out/target/product/sargo/product.img
18:45:03 mkuserimg_mke2fs.py ERROR: Failed to run e2fsdroid_cmd: __populate_fs: Could not allocate block in ext2 filesystem while writing file "Launcher3QuickStep.apk"
e2fsdroid: Could not allocate block in ext2 filesystem while populating file system

mke2fs 1.44.4 (18-Aug-2018)
Creating filesystem with 56782 4k blocks and 512 inodes
Filesystem UUID: 5c5096c4-7682-48b8-aa09-be6035cdfd33
Superblock backups stored on blocks: 
    32768

Allocating group tables: 0/2   done                            
Writing inode tables: 0/2   done                            
Writing superblocks and filesystem accounting information: 0/2   done

__populate_fs: Could not allocate block in ext2 filesystem while writing file "Launcher3QuickStep.apk"
e2fsdroid: Could not allocate block in ext2 filesystem while populating file system

depmod: WARNING: could not open modules.order at /home/build/build/base/out/target/product/sargo/obj/PACKAGING/depmod_vendor_intermediates/lib/modules/0.0: No such file or directory
depmod: WARNING: could not open modules.builtin at /home/build/build/base/out/target/product/sargo/obj/PACKAGING/depmod_vendor_intermediates/lib/modules/0.0: No such file or directory
19:02:23 ninja failed with: exit status 1
Makefile:37: recipe for target 'build' failed
make: *** [build] Error 1
ypid commented 4 years ago

I hit this issue again (very similar) with the 2020-01 security patches and this time a clean build did not help. If anybody either runs into this same issue or if you are not affected, please comment so that I can either check what I am doing wrong or so that we can together search for a solution.

Build logs: https://gist.github.com/ypid/580160332bf465ee9aa74336e748ea70

$ grep 'failed' ...
FAILED: out/target/product/sargo/obj/PACKAGING/target_files_intermediates/aosp_sargo-target_files-01142217.zip
2020-01-19 00:59:59 - common.py - WARNING : Failed to read PRODUCT_SERVICES/build.prop
2020-01-19 00:59:59 - common.py - WARNING : Failed to read PRODUCT_SERVICES/etc/build.prop
2020-01-19 00:59:59 - common.py - WARNING : Failed to read ODM/build.prop
2020-01-19 00:59:59 - common.py - WARNING : Failed to read ODM/etc/build.prop
01:00:17 mkuserimg_mke2fs.py ERROR: Failed to run e2fsdroid_cmd: __populate_fs: Could not allocate block in ext2 filesystem while writing file "android.hardware.automotive.vehicle@2.0.so"
ExternalError: Failed to run command '['mkuserimg_mke2fs', '-s', '/home/build/build/base/out/soong/.temp/tmporKDRT', '/home/build/build/base/out/target/product/sargo/obj/PACKAGING/target_files_intermediates/aosp_sargo-target_files-01142217/IMAGES/system.img', 'ext4', '/', '564908032', '-j', '0', '-T', '1230768000', '-C', '/home/build/build/base/out/soong/.temp/merged_fs_config1DoORN.txt', '-B', '/home/build/build/base/out/target/product/sargo/obj/PACKAGING/target_files_intermediates/aosp_sargo-target_files-01142217/IMAGES/system.map', '-L', '/', '-i', '3377', '-M', '0', '-U', '4729639d-b5f2-5cc1-a120-9ac5f788683c', '-S', 'fd77d4fd-bb91-5ef5-9533-d5d806da85a3', '-c', '--inode_size', '256', '/home/build/build/base/out/target/product/sargo/obj/PACKAGING/target_files_intermediates/aosp_sargo-target_files-01142217/META/file_contexts.bin']' (exit code 4):
01:00:17 mkuserimg_mke2fs.py ERROR: Failed to run e2fsdroid_cmd: __populate_fs: Could not allocate block in ext2 filesystem while writing file "android.hardware.automotive.vehicle@2.0.so"


01:00:17 ninja failed with: exit status 1
Makefile:47: recipe for target 'build' failed

cc: @ubergeek77

ubergeek77 commented 4 years ago

Does this only happen when you try to do a "re" build against a new security patch? Or is it happening for any rebuild, even if you haven't done a new repo sync?

I've been doing local rebuilds to test some cherry-picks for my device, and I haven't run into anything like this. A full build takes about 3-4 hours on my hardware, but a rebuild with cherry picks takes ~15 minutes. Since my cherry-picked rebuilds are done in that much shorter time frame of 15 minutes, and the resulting images flash just fine, I know for certain that my rebuilds are working properly. However, I haven't gone from one upstream build reference to another, so I'll be able to comment on that when the February one rolls around.

I'm building for crosshatch, so do you think this is device-specific? It looks like you're building for sargo.

I know this is a longshot, but you do have enough disk space, right? That particular error looks like it could happen on lack of space.

ubergeek77 commented 4 years ago

That error output from your first comment caught my eye:

Out of space? Out of inodes?

I'm not aware of this, but is there a limit on the number of inodes a Linux filesystem can have, regardless of location? It's a stretch, but perhaps you're hitting this limit. If that was what was going on, deleting out would obviously reduce the number of inodes on your system, so this error wouldn't be hit.

Or, if they aren't talking about your system, but instead the filesystem that the build process is trying to create, maybe there's a bug in the python script that's over-active to the number of files/inodes in the target-files directory, even if ignoring that "limit" would produce a build with no issue.

If you're sure your local filesystem has no issues, then my recommendation would be to see if "e2fsdroid" or "e2fsprogs" has any notable changes between device trees (such as on crosshatch), and if there are, modify your build manifest to pull down crosshatch's branch versions of those instead. If there really is a bug there, considering this isn't happening to me, maybe that will fix it.

ypid commented 4 years ago

I checked the obvious things already I hope.

/dev/vda2        ext4       20G  7.4G   12G  40% /
tmpfs            tmpfs     7.2G     0  7.2G   0% /tmp
/dev/vdb1        btrfs     481G  353G  119G  75% /media/data

aren't talking about your system, but instead the filesystem that the build process is trying to create

That is also my understanding.

I guess I will need to investigate further. Thank you already for your help :+1:

ubergeek77 commented 4 years ago

Devil's advocate, if you wanted to be extra sure about that inode thing

ypid commented 4 years ago

Thanks, btrfs does not use inodes :)

/dev/vda2        ext4       1.3M  314K  967K   25% /
tmpfs            tmpfs      1.8M    15  1.8M    1% /tmp
/dev/vdb1        btrfs         0     0     0     - /media/data
ubergeek77 commented 4 years ago

Oh cool, that's a TIL for me, I didn't know that.

ypid commented 4 years ago

I fixed it in the third clean build. I changed two things:

I will leave the issue open for now because the original one has probably a different cause.

Manouchehri commented 4 years ago

I'm running into this issue on coral as well when building AOSP (not the kernel). I'll try m clean and report back when it finally finishes building. =) 

2020-06-15 10:25:54 - build_image.py - ERROR   : Failed to build out/target/product/coral/obj/PACKAGING/systemimage_intermediates/system.img from out/target/product/coral/system
Out of space? Out of inodes? The tree size of /mnt/builds/android/out/soong/.temp/tmptr9suL is 496683008 bytes (473 MB), with reserved space of 0 bytes (0 MB).
The max image size for filesystem files is 513454080 bytes (489 MB), out of a total partition size of 513454080 bytes (489 MB).
Traceback (most recent call last):
  File "build/make/tools/releasetools/build_image.py", line 789, in <module>
    main(sys.argv[1:])
  File "build/make/tools/releasetools/build_image.py", line 781, in main
    BuildImage(in_dir, image_properties, out_file, target_out)
  File "build/make/tools/releasetools/build_image.py", line 423, in BuildImage
    BuildImageMkfs(in_dir, prop_dict, out_file, target_out, fs_config)
  File "build/make/tools/releasetools/build_image.py", line 334, in BuildImageMkfs
    mkfs_output = common.RunAndCheckOutput(build_command)
  File "/mnt/builds/android/build/make/tools/releasetools/common.py", line 252, in RunAndCheckOutput
    args, proc.returncode, output))
common.ExternalError: Failed to run command '['mkuserimg_mke2fs', '-s', '/mnt/builds/android/out/soong/.temp/tmptr9suL', 'out/target/product/coral/obj/PACKAGING/systemimage_intermediates/system.img', 'ext4', '/', '513454080', '-j', '0', '-D', 'out/target/product/coral/system', '-L', '/', '-i', '3048', '-M', '0', '-c', '--inode_size', '256', 'out/target/product/coral/obj/ETC/file_contexts.bin_intermediates/file_contexts.bin']' (exit code 4):
10:25:51 mkuserimg_mke2fs.py INFO: Env: {'MKE2FS_CONFIG': './system/extras/ext4_utils/mke2fs.conf'}
10:25:51 mkuserimg_mke2fs.py INFO: Running: mke2fs -O ^has_journal -L / -N 3048 -I 256 -M / -m 0 -E android_sparse -t ext4 -b 4096 out/target/product/coral/obj/PACKAGING/systemimage_intermediates/system.img 125355
10:25:51 mkuserimg_mke2fs.py INFO: Env: {}
10:25:51 mkuserimg_mke2fs.py INFO: Running: e2fsdroid -p out/target/product/coral/system -s -S out/target/product/coral/obj/ETC/file_contexts.bin_intermediates/file_contexts.bin -f /mnt/builds/android/out/soong/.temp/tmptr9suL -a / out/target/product/coral/obj/PACKAGING/systemimage_intermediates/system.img
10:25:54 mkuserimg_mke2fs.py ERROR: Failed to run e2fsdroid_cmd: __populate_fs: Could not allocate block in ext2 filesystem while writing file "android.hardware.wifi@1.0.so"
e2fsdroid: Could not allocate block in ext2 filesystem while populating file system

mke2fs 1.44.4 (18-Aug-2018)
Creating filesystem with 125355 4k blocks and 3072 inodes
Filesystem UUID: 1d03f927-293c-4e89-b19f-1405b0cdb64d
Superblock backups stored on blocks: 
        32768, 98304

Allocating group tables: done                            
Writing inode tables: done                            
Writing superblocks and filesystem accounting information: done

__populate_fs: Could not allocate block in ext2 filesystem while writing file "android.hardware.wifi@1.0.so"
e2fsdroid: Could not allocate block in ext2 filesystem while populating file system

10:25:55 ninja failed with: exit status 1

#### failed to build some targets (04:15 (mm:ss)) ####
kheaactua commented 4 years ago

I have the same problem as above, except it fails on vendor.img instead of system.img, I've made clean and re-tried several times now to no avail. There's a thread here that suggests increasing the _PARITION_SIZE settings in BoardConfig.mk, but for x86_64 there is no explicitly VENDOR_PARTITION_SIZE. I gave it a try but nothing seemed to help.

My error:

2020-07-13 00:19:22 - build_image.py - ERROR   : Failed to build out/target/product/generic_x86_64/vendor.img from out/target/product/generic_x86_64/vendor
Out of space? Out of inodes? The tree size of out/target/product/generic_x86_64/vendor is 68457472 bytes (65 MB), with reserved space of 0 bytes (0 MB).
The max image size for filesystem files is 85237760 bytes (81 MB), out of a total partition size of 85237760 bytes (81 MB).
Traceback (most recent call last):
  File "build/make/tools/releasetools/build_image.py", line 789, in <module>
    main(sys.argv[1:])
  File "build/make/tools/releasetools/build_image.py", line 781, in main
    BuildImage(in_dir, image_properties, out_file, target_out)
  File "build/make/tools/releasetools/build_image.py", line 423, in BuildImage
    BuildImageMkfs(in_dir, prop_dict, out_file, target_out, fs_config)
  File "build/make/tools/releasetools/build_image.py", line 334, in BuildImageMkfs
    mkfs_output = common.RunAndCheckOutput(build_command)
  File "/src/aosp/build/make/tools/releasetools/common.py", line 252, in RunAndCheckOutput
    args, proc.returncode, output))
common.ExternalError: Failed to run command '['mkuserimg_mke2fs', 'out/target/product/generic_x86_64/vendor', 'out/target/product/generic_x86_64/vendor.img', 'ext4', 'vendor', '85237760', '-j', '0', '-D', 'out/target/product/generic_x86_64/system', '-L', 'vendor', '-i', '541', '-M', '0', '-c', '--inode_size', '256', 'out/target/product/generic_x86_64/obj/ETC/file_contexts.bin_intermediates/file_contexts.bin']' (exit code 4):
00:19:20 mkuserimg_mke2fs.py INFO: Env: {'MKE2FS_CONFIG': './system/extras/ext4_utils/mke2fs.conf'}
00:19:20 mkuserimg_mke2fs.py INFO: Running: mke2fs -O ^has_journal -L vendor -N 541 -I 256 -M /vendor -m 0 -t ext4 -b 4096 out/target/product/generic_x86_64/vendor.img 20810
00:19:20 mkuserimg_mke2fs.py INFO: Env: {}
00:19:20 mkuserimg_mke2fs.py INFO: Running: e2fsdroid -e -p out/target/product/generic_x86_64/system -s -S out/target/product/generic_x86_64/obj/ETC/file_contexts.bin_intermediates/file_contexts.bin -f out/target/product/generic_x86_64/vendor -a /vendor out/target/product/generic_x86_64/vendor.img
00:19:22 mkuserimg_mke2fs.py ERROR: Failed to run e2fsdroid_cmd: __populate_fs: Could not allocate block in ext2 filesystem while writing file "libGLESv1_CM_swiftshader.so"
e2fsdroid: Could not allocate block in ext2 filesystem while populating file system

mke2fs 1.44.4 (18-Aug-2018)
Creating filesystem with 20810 4k blocks and 544 inodes

Allocating group tables: done
Writing inode tables: done
Writing superblocks and filesystem accounting information: done

__populate_fs: Could not allocate block in ext2 filesystem while writing file "libGLESv1_CM_swiftshader.so"
e2fsdroid: Could not allocate block in ext2 filesystem while populating file system
ypid commented 4 years ago

Jup, that also happened to me this month.

ExternalError: Failed to run command '['mkuserimg_mke2fs', '-s', '/home/build/build/base/out/soong/.temp/tmpx7dyzJ', '/home/build/build/base
/out/target/product/sargo/obj/PACKAGING/target_files_intermediates/aosp_sargo-target_files-07072122/IMAGES/system.img', 'ext4', '/', '552378
368', '-j', '0', '-T', '1230768000', '-C', '/home/build/build/base/out/soong/.temp/merged_fs_configj4qel7.txt', '-B', '/home/build/build/bas
e/out/target/product/sargo/obj/PACKAGING/target_files_intermediates/aosp_sargo-target_files-07072122/IMAGES/system.map', '-L', '/', '-i', '3
384', '-M', '0', '-U', '4729639d-b5f2-5cc1-a120-9ac5f788683c', '-S', 'fd77d4fd-bb91-5ef5-9533-d5d806da85a3', '-c', '--inode_size', '256', '/
home/build/build/base/out/target/product/sargo/obj/PACKAGING/target_files_intermediates/aosp_sargo-target_files-07072122/META/file_contexts.
bin']' (exit code 4):
13:16:37 mkuserimg_mke2fs.py INFO: Env: {'E2FSPROGS_FAKE_TIME': '1230768000', 'MKE2FS_CONFIG': './system/extras/ext4_utils/mke2fs.conf'}
13:16:37 mkuserimg_mke2fs.py INFO: Running: mke2fs -O ^has_journal -L / -N 3384 -I 256 -M / -m 0 -U 4729639d-b5f2-5cc1-a120-9ac5f788683c -E
android_sparse,hash_seed=fd77d4fd-bb91-5ef5-9533-d5d806da85a3 -t ext4 -b 4096 /home/build/build/base/out/target/product/sargo/obj/PACKAGING/
target_files_intermediates/aosp_sargo-target_files-07072122/IMAGES/system.img 134858
13:16:38 mkuserimg_mke2fs.py INFO: Env: {'E2FSPROGS_FAKE_TIME': '1230768000'}
13:16:38 mkuserimg_mke2fs.py INFO: Running: e2fsdroid -T 1230768000 -C /home/build/build/base/out/soong/.temp/merged_fs_configj4qel7.txt -B /home/build/build/base/out/target/product/sargo/obj/PACKAGING/target_files_intermediates/aosp_sargo-target_files-07072122/IMAGES/system.map
-s -S /home/build/build/base/out/target/product/sargo/obj/PACKAGING/target_files_intermediates/aosp_sargo-target_files-07072122/META/file_co
ntexts.bin -f /home/build/build/base/out/soong/.temp/tmpx7dyzJ -a / /home/build/build/base/out/target/product/sargo/obj/PACKAGING/target_fil
es_intermediates/aosp_sargo-target_files-07072122/IMAGES/system.img
13:16:42 mkuserimg_mke2fs.py ERROR: Failed to run e2fsdroid_cmd: __populate_fs: Could not allocate block in ext2 filesystem while writing f$
le "libvintf.so"                                                                       
e2fsdroid: Could not allocate block in ext2 filesystem while populating file system

The second build attempt was successful. But I can not see anything being wrong with my first build attempt either. Both were clean builds.

My hope is that those errors go away with hashbang/aosp-build#4 (GrapheneOS also lists Debian buster as supported). You might try that.

kheaactua commented 4 years ago

For me, this was a result of attempting to write to a zfs drive. As soon as I pointed to an ext4 drive the build succeeded.

Manouchehri commented 4 years ago

I should also mention that I always have this failure on a NFS drive too (backed by ZFS). Occasionally I've had the issue on a a non-clean build (as mentioned in https://github.com/hashbang/aosp-build/issues/9#issuecomment-644185810) on ext4.

ypid commented 4 years ago

Interesting. ZFS is only partly involved in my setup. My build machine looks like this:

2 NVMe SSDs                 -> Partitions -> LUKS -> zpool mirror -> zfs fs -> qcow2 -> KVM Android build VM -> root (`/`) ext4 fs for the OS and Docker images.
2 NVMe SSDs (the same once) -> Partitions -> LUKS -> LVM                    -> 2 LV  -> KVM VM               -> btrfs RAID1 for all the source code and build output.

The VM was still on 4.9.0-12-amd64 until this day. My next build will be on linux-image-4.19.0-0.bpo.9-amd64.

Manouchehri commented 4 years ago

I wonder if it's btrfs that's breaking the build for you. The error messages are pretty confusing and I had a hard time narrowing down where the issue is with e2fsdroid.

sanbrother commented 4 years ago

@kheaactua

I have the same problem as above, except it fails on vendor.img instead of system.img, I've made clean and re-tried several times now to no avail. There's a thread here that suggests increasing the _PARITION_SIZE settings in BoardConfig.mk, but for x86_64 there is no explicitly VENDOR_PARTITION_SIZE. I gave it a try but nothing seemed to help.

BOARD_VENDORIMAGE_PARTITION_SIZE might be the right one?

RahifM commented 4 years ago

guys iv encountered this same error when doing a dirty build of aospa

https://del.dog/stilyfugov.txt

any fix for it?

Sundio-Suen commented 4 years ago

@kheaactua

I have the same problem as above, except it fails on vendor.img instead of system.img, I've made clean and re-tried several times now to no avail. There's a thread here that suggests increasing the _PARITION_SIZE settings in BoardConfig.mk, but for x86_64 there is no explicitly VENDOR_PARTITION_SIZE. I gave it a try but nothing seemed to help.

BOARD_VENDORIMAGE_PARTITION_SIZE might be the right one?

Yes, this is the right solution, make sure that BOARD_VENDORIMAGE_PARTITION_SIZE is a bit larger than the total size of out/target/product/xxx/vendor

ghost commented 3 years ago 
Traceback (most recent call last):
  File "./build/tools/releasetools/build_image.py", line 789, in <module>
    main(sys.argv[1:])
  File "./build/tools/releasetools/build_image.py", line 781, in main
    BuildImage(in_dir, image_properties, out_file, target_out)
  File "./build/tools/releasetools/build_image.py", line 404, in BuildImage
    size = GetDiskUsage(in_dir)
  File "./build/tools/releasetools/build_image.py", line 61, in GetDiskUsage
    output = common.RunAndCheckOutput(cmd, verbose=False)
  File "/data/le/build/make/tools/releasetools/common.py", line 258, in RunAndCheckOutput
    args, proc.returncode, output))
common.ExternalError: Failed to run command '['du', '-b', '-k', '-s', 'out/target/product/phoenix/product']' (exit code 1):
du: Unknown option b (see "du --help")

[ 98% 1222/1236] ----- Making recovery image ------
Generating Changelog for LegionOS Supported Device
08:54:38 ninja failed with: exit status 1

This the error I got looks similar how to fix it ??

Edit(ypid): Use Markdown code blocks.

ypid commented 3 years ago

Well:

du: Unknown option b (see "du --help")

Are you sure you are using https://github.com/hashbang/aosp-build ;-) ?

ashuk1109 commented 3 years ago

i am facing the same du: unknown option b error... and errors in system.img Any fixes??

ypid commented 3 years ago

First step would be to post what exact build environment you are using. I just found evidence that @zoldycc is not using Hashbang OS, ref:

  File "/data/le/build/make/tools/releasetools/common.py", line 258, in RunAndCheckOutput

When you have confirmed that you are actually using Hashbang OS, open a new issue. The du issue seems unrelated to this one. So, please don’t mix different issues into this thread.

antonysigma commented 3 years ago

Possible root cause in the ZFS on linux implementation: https://github.com/openzfs/zfs/issues/10248#issue-606245033

huanfeng commented 2 years ago

Refer to this: https://android-review.googlesource.com/c/platform/build/+/1269603

releasetools: Use du -b

toybox commit: https://github.com/landley/toybox/pull/177

Test: m, builds and boots. Devices: crosshatch blueline bonito sargo (anything with dynamic partitions)

lallolu commented 2 years ago

For me, it was caused by the ROM I was building, including gapps. I solved it by removing some of the google apps that do not neccessarily need to be inbuilt in the ROM from vendor/gms/gms_full.mk. I removed drive, chrome, calendar, calculator, youtube and youtube music.