generated image do not work

siteswapjuggler commented 5 months ago

Hello,

I'm trying to get those script to work in a native arm Virtual Machine (Parallel Desktop, Mac Book Pro M1 Max, Ubuntu 22.04). So far I had no problem installing the environnement, then I had to replace executable file :

sd_update: do not execute, no source found, I tried to swap with armf version found here https://github.com/friendlyarm/sd_update-bin

make_ext4fs: do not execute, source found but deprecated, a solution may be to adapt the script to mke2fs, I managed to get a folder into an img, did not test further, what I can tell is that the img file I produce differ from the one from friendlyarm because mine is mounting correctly while I systematically got "bad superblock error" when I try to mount one of them.

fill_img_to_eflasher: seems to execute correctly

Whell I tried to get an image then burn it into an sd card an try to run it over my nanoPi:

the img produce ./mk-sd-image.sh friendlycore run but seems to be a Debian Jessie...
the img produce ./mk-emmc-image.sh friendlycore run, but the system doesn't mount /dev/mmcblk0p1... when I try to mount it manually I get "wrong fs type, bad option, bad superblock on /dev/mmcblk0p1, missing codepage or helper program, or other error."

Well so far it's quiet a nightmare....

friendlyarm commented 5 months ago

thank you for your feedback, good news, you can package images on arm64 hosts now, use git pull to pull updates.

siteswapjuggler commented 5 months ago

Hi and thanks a lot for your quick answer I just tested the updates after pulling :) So far the script execute without any problem but I'm still confuse :

the friendlycore images output for Jammy doesn't match the official image found on friendlyarm google drive so should I assume that files containend in 03_Partition image files do not match the one in 01_Official image ?

the eflasher image output for Jammy give me the following message :

You are in emergency mode. After logging in, type "journalctl -xb" to view system logs, "systemctl reboot" to reboot, "systemctl default" or "exit" to boot into default mode. Give root password for maintenance (or press Control-D to continue):

I can log in with "fa" but eflasher doesn't start, mmcblk0p1 is not mounted in /mnt/sdcard and when I try to mount it manually I got this error message:

mount: /mnt/sdcard: wrong fs type, bad option, bad superblock on /dev/mmcblk0p1, missing codepage or helper program, or other error.

My main objective here is to find a way to prepare my system for production without having to back up the whole emmc memory. So basically to found a way to prepare an SD-to-emmc image with a reduce raw file like the Official image. I was hoping the script in this repo could help doing exactly that but perhaps I misunderstood.

friendlyarm commented 5 months ago

Can you provide us with your operation so we can reproduce your problem, for example, your command line history

siteswapjuggler commented 5 months ago

Sure :)

So first I initialize the dev environment :

sudo apt-get -y update sudo apt-get -y upgrade sudo apt-get install -y wget wget https://raw.githubusercontent.com/friendlyarm/build-env-on-ubuntu-bionic/master/install.sh sudo chmod +x ./install.sh sudo ./install.sh git clone https://github.com/friendlyarm/sd-fuse_h3

Everything goes well apart from "gcc-multilib" which is not found.

Then I download emmc-flasher-images.tgz, friendlycore-jammy-images.tgz, friendlycore-focal-images.tgz friendlycore-xenial-kernel3.4-images.tgz, friendlycore-images.tgz from your Google Drive and unzip them in the sd-fuse_h3 directory.

At this point please note than none of the archive match the "friendlycore-xenial" but I assume we can just change de directory names.

Finally I try to use the scripts :

sudo ./mk-emmc-image.sh friendlycore-jammy

Results are two .img files in the out subdirectory :

h3_eflasher-20240407.img
h3-sd-friendlycore-jammy-4.14-armhf-20240407.img

Eflasher image does mount rootfs and boot partition but do not have the third partition.. SD image does not mount at all.

friendlyarm commented 5 months ago

thanks, I created a temporary patch, please try again:

cd sd-fuse_h3
wget http://112.124.9.243:8888/0001-misc-fix-h3.patch
git am 0001-misc-fix-h3.patch
./mk-emmc-image.sh friendlycore-jammy

it works in a ubuntu22.04 virtual machine created by OrbStack (HW: apple m2, Mac mini)

siteswapjuggler commented 5 months ago

Hello,

I just tired the patch so now I only got h3_eflasher-20240408.img file in the out subdirectory but this .img still doesn't work.

When I try to mount this img file lsblk gives me the following results :

loop12 7:12 0 7,3G 1 loop ├─loop12p1 259:3 0 6,4G 1 part ├─loop12p2 259:4 0 40M 1 part /media/parallels/boot └─loop12p3 259:5 0 850M 1 part /media/parallels/rootfs

As you can see p2 and p3 are ok but p1 (which should contain friendlycore-jammy files) doesn't mount at all. With the official images this last partition mount on /media/parallels/FriendlyArm.

And when I try to mount it manually with sudo mount /dev/loop12p1 /mnt/imgfile/ I got the following result : mount: /mnt/imgfile: wrong fs type, bad option, bad superblock on /dev/loop12p1, missing codepage or helper program, or other error.

I'm trying to burn it to an SD card right now but I'm quiet sure that I'll get the same error and that emmc system will go in emergency mode.

friendlyarm commented 5 months ago

Here's my collection of logs for you to compare:

ubuntu@ubuntu:/opt/work-ubuntu$ git clone https://github.com/friendlyarm/sd-fuse_h3
Cloning into 'sd-fuse_h3'...
remote: Enumerating objects: 419, done.
remote: Counting objects: 100% (179/179), done.
remote: Compressing objects: 100% (120/120), done.
remote: Total 419 (delta 116), reused 115 (delta 59), pack-reused 240
Receiving objects: 100% (419/419), 26.00 MiB | 7.69 MiB/s, done.
Resolving deltas: 100% (231/231), done.
ubuntu@ubuntu:/opt/work-ubuntu$ cd sd-fuse_h3/
ubuntu@ubuntu:/opt/work-ubuntu/sd-fuse_h3$ tar xvzf /Volumes/ssd/Download/emmc-flasher-images.tgz
eflasher/
eflasher/rootfs.img
eflasher/u-boot-sunxi-with-spl.bin
eflasher/info.conf
eflasher/partmap.txt
eflasher/env_eflasher.bin
eflasher/boot.img
ubuntu@ubuntu:/opt/work-ubuntu/sd-fuse_h3$ tar xvzf /Volumes/ssd/Download/friendlycore-jammy-images.tgz
friendlycore-jammy/
friendlycore-jammy/rootfs.img
friendlycore-jammy/u-boot-sunxi-with-spl.bin
friendlycore-jammy/info.conf
friendlycore-jammy/partmap.txt
friendlycore-jammy/userdata.img
friendlycore-jammy/boot.img
ubuntu@ubuntu:/opt/work-ubuntu/sd-fuse_h3$ ./mk-emmc-image.sh friendlycore-jammy
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
  android-libbase android-liblog android-libsparse libparted2 swig4.0
Suggested packages:
  libparted-dev libparted-i18n parted-doc openssh-server swig-doc swig-examples swig4.0-examples swig4.0-doc
The following NEW packages will be installed:
  android-libbase android-liblog android-libsparse android-sdk-libsparse-utils exfat-fuse exfat-utils libparted2 parted rsync swig swig4.0 wget
0 upgraded, 12 newly installed, 0 to remove and 36 not upgraded.
Need to get 1980 kB of archives.
After this operation, 8245 kB of additional disk space will be used.
Do you want to continue? [Y/n] Y
Get:1 http://mirrors.cloud.tencent.com/ubuntu-ports focal-updates/main arm64 rsync arm64 3.1.3-8ubuntu0.7 [307 kB]
Get:2 http://mirrors.cloud.tencent.com/ubuntu-ports focal-updates/main arm64 libparted2 arm64 3.3-4ubuntu0.20.04.1 [149 kB]
Get:3 http://mirrors.cloud.tencent.com/ubuntu-ports focal-updates/main arm64 parted arm64 3.3-4ubuntu0.20.04.1 [40.9 kB]
Get:4 http://mirrors.cloud.tencent.com/ubuntu-ports focal-updates/main arm64 wget arm64 1.20.3-1ubuntu2 [331 kB]
Get:5 http://mirrors.cloud.tencent.com/ubuntu-ports focal/universe arm64 android-liblog arm64 1:8.1.0+r23-5ubuntu2 [39.8 kB]
Get:6 http://mirrors.cloud.tencent.com/ubuntu-ports focal/universe arm64 android-libbase arm64 1:8.1.0+r23-5ubuntu2 [20.7 kB]
Get:7 http://mirrors.cloud.tencent.com/ubuntu-ports focal/universe arm64 android-libsparse arm64 1:8.1.0+r23-5ubuntu2 [15.6 kB]
Get:8 http://mirrors.cloud.tencent.com/ubuntu-ports focal/universe arm64 android-sdk-libsparse-utils arm64 1:8.1.0+r23-5ubuntu2 [12.6 kB]
Get:9 http://mirrors.cloud.tencent.com/ubuntu-ports focal/universe arm64 exfat-fuse arm64 1.3.0-1 [23.8 kB]
Get:10 http://mirrors.cloud.tencent.com/ubuntu-ports focal/universe arm64 exfat-utils arm64 1.3.0-1 [40.3 kB]
Get:11 http://mirrors.cloud.tencent.com/ubuntu-ports focal/universe arm64 swig4.0 arm64 4.0.1-5build1 [994 kB]
Get:12 http://mirrors.cloud.tencent.com/ubuntu-ports focal/universe arm64 swig all 4.0.1-5build1 [5528 B]
Fetched 1980 kB in 7s (293 kB/s)
Selecting previously unselected package rsync.
(Reading database ... 74936 files and directories currently installed.)
Preparing to unpack .../00-rsync_3.1.3-8ubuntu0.7_arm64.deb ...
Unpacking rsync (3.1.3-8ubuntu0.7) ...
Selecting previously unselected package libparted2:arm64.
Preparing to unpack .../01-libparted2_3.3-4ubuntu0.20.04.1_arm64.deb ...
Unpacking libparted2:arm64 (3.3-4ubuntu0.20.04.1) ...
Selecting previously unselected package parted.
Preparing to unpack .../02-parted_3.3-4ubuntu0.20.04.1_arm64.deb ...
Unpacking parted (3.3-4ubuntu0.20.04.1) ...
Selecting previously unselected package wget.
Preparing to unpack .../03-wget_1.20.3-1ubuntu2_arm64.deb ...
Unpacking wget (1.20.3-1ubuntu2) ...
Selecting previously unselected package android-liblog.
Preparing to unpack .../04-android-liblog_1%3a8.1.0+r23-5ubuntu2_arm64.deb ...
Unpacking android-liblog (1:8.1.0+r23-5ubuntu2) ...
Selecting previously unselected package android-libbase.
Preparing to unpack .../05-android-libbase_1%3a8.1.0+r23-5ubuntu2_arm64.deb ...
Unpacking android-libbase (1:8.1.0+r23-5ubuntu2) ...
Selecting previously unselected package android-libsparse.
Preparing to unpack .../06-android-libsparse_1%3a8.1.0+r23-5ubuntu2_arm64.deb ...
Unpacking android-libsparse (1:8.1.0+r23-5ubuntu2) ...
Selecting previously unselected package android-sdk-libsparse-utils.
Preparing to unpack .../07-android-sdk-libsparse-utils_1%3a8.1.0+r23-5ubuntu2_arm64.deb ...
Unpacking android-sdk-libsparse-utils (1:8.1.0+r23-5ubuntu2) ...
Selecting previously unselected package exfat-fuse.
Preparing to unpack .../08-exfat-fuse_1.3.0-1_arm64.deb ...
Unpacking exfat-fuse (1.3.0-1) ...
Selecting previously unselected package exfat-utils.
Preparing to unpack .../09-exfat-utils_1.3.0-1_arm64.deb ...
Unpacking exfat-utils (1.3.0-1) ...
Selecting previously unselected package swig4.0.
Preparing to unpack .../10-swig4.0_4.0.1-5build1_arm64.deb ...
Unpacking swig4.0 (4.0.1-5build1) ...
Selecting previously unselected package swig.
Preparing to unpack .../11-swig_4.0.1-5build1_all.deb ...
Unpacking swig (4.0.1-5build1) ...
Setting up wget (1.20.3-1ubuntu2) ...
Setting up android-liblog (1:8.1.0+r23-5ubuntu2) ...
Setting up exfat-utils (1.3.0-1) ...
Setting up libparted2:arm64 (3.3-4ubuntu0.20.04.1) ...
Setting up exfat-fuse (1.3.0-1) ...
Setting up swig4.0 (4.0.1-5build1) ...
Setting up android-libbase (1:8.1.0+r23-5ubuntu2) ...
Setting up rsync (3.1.3-8ubuntu0.7) ...
Created symlink /etc/systemd/system/multi-user.target.wants/rsync.service → /lib/systemd/system/rsync.service.
Setting up swig (4.0.1-5build1) ...
Setting up parted (3.3-4ubuntu0.20.04.1) ...
Setting up android-libsparse (1:8.1.0+r23-5ubuntu2) ...
Setting up android-sdk-libsparse-utils (1:8.1.0+r23-5ubuntu2) ...
Processing triggers for systemd (245.4-4ubuntu3.22) ...
Processing triggers for man-db (2.9.1-1) ...
Processing triggers for libc-bin (2.31-0ubuntu9.14) ...
Re-running script under sudo...
Creating RAW image: out/h3-eflasher-20240409.img (7800 MB)
---------------------------------
0+0 records in
0+0 records out
0 bytes copied, 1.1958e-05 s, 0.0 kB/s
Using device: /dev/loop0
---------------------------------
Eflasher filesystem fusing
Image root: ./eflasher

----------------------------------------------------------------
[/dev/loop0] capacity = 7438MB, 7799999488 bytes
current /dev/loop0 partition:
MBR.0 start : 0x0000100000 size 0x01d0da8c00  kB
----------------------------------------------------------------
parsing ./eflasher/partmap.txt:
part.0 flash=mmc,0:boot0:raw:0x2000,0x1fe000:u-boot-sunxi-with-spl.bin:[RAW] ./eflasher/u-boot-sunxi-with-spl.bin
part.1 flash=mmc,0:env:raw:0x200000,0x20000:env_eflasher.bin:[RAW] ./eflasher/env_eflasher.bin
part.2 flash=mmc,0:data:fat:0x39200000,0x0::[MBR]
part.3 flash=mmc,0:boot:fat:0x1800000,0x2800000:boot.img:[MBR] ./eflasher/boot.img
part.4 flash=mmc,0:rootfs:ext4:0x4000000,0x35200000:rootfs.img:[MBR] ./eflasher/rootfs.img
----------------------------------------------------------------
create new MBR 3:
[MBR.0] start : 0x0039200000 size 0x0000000000
[MBR.1] start : 0x0001800000 size 0x0002800000
[MBR.2] start : 0x0004000000 size 0x0035200000
----------------------------------------------------------------
copy from: ./eflasher to /dev/loop0
 [RAW. 0]:     2040 KB | ./eflasher/u-boot-sunxi-with-spl.bin > 100% : done.
 [RAW. 1]:      128 KB | ./eflasher/env_eflasher.bin          > 100% : done.
 [MBR. 1]:    40960 KB | ./eflasher/boot.img                  > 100% : done.
 [MBR. 2]:   654460 KB | ./eflasher/rootfs.img                > 100% : done.
----------------------------------------------------------------
/dev/loop0: msdos partitions 2 3 1
---------------------------------
All done.
mkexfatfs 1.3.0
Creating... done.
Flushing... done.
File system created successfully.
---------------------------------
RAW image successfully created (13:37:54).
-rw-r--r-- 1 root root 7799999488 Apr  9 13:37 out/h3-eflasher-20240409.img
Tip: You can compress it to save disk space.

FUSE exfat 1.3.0
copying /opt/work-ubuntu/sd-fuse_h3/friendlycore-jammy/* to /tmp/media_exfat/friendlycore-jammy ...
-rw-r--r-- 1 root root 7799999488 Apr  9 13:37 out/h3-eflasher-friendlycore-jammy-4.14-armhf-20240409.img
all done.

siteswapjuggler commented 5 months ago

Ok here is my log :

`./mk-emmc-image.sh friendlycore-jammy Re-running script under sudo... Creating RAW image: out/h3-eflasher-20240409.img (7800 MB)

0+0 enregistrements lus 0+0 enregistrements écrits 0 octet copié, 9,3082e-05 s, 0,0 kB/s Using device: /dev/loop13

Eflasher filesystem fusing Image root: ./eflasher

[/dev/loop13] capacity = 7438MB, 7799999488 bytes current /dev/loop13 partition: MBR.0 start : 0x0000100000 size 0x01d0da8c00 kB

parsing ./eflasher/partmap.txt: part.0 flash=mmc,0:boot0:raw:0x2000,0x1fe000:u-boot-sunxi-with-spl.bin:[RAW] ./eflasher/u-boot-sunxi-with-spl.bin part.1 flash=mmc,0:env:raw:0x200000,0x20000:env_eflasher.bin:[RAW] ./eflasher/env_eflasher.bin part.2 flash=mmc,0:data:fat:0x39200000,0x0::[MBR] part.3 flash=mmc,0:boot:fat:0x1800000,0x2800000:boot.img:[MBR] ./eflasher/boot.img part.4 flash=mmc,0:rootfs:ext4:0x4000000,0x35200000:rootfs.img:[MBR] ./eflasher/rootfs.img

create new MBR 3: [MBR.0] start : 0x0039200000 size 0x0000000000 [MBR.1] start : 0x0001800000 size 0x0002800000 [MBR.2] start : 0x0004000000 size 0x0035200000

copy from: ./eflasher to /dev/loop13 [RAW. 0]: 2040 KB | ./eflasher/u-boot-sunxi-with-spl.bin > 100% : done. [RAW. 1]: 128 KB | ./eflasher/env_eflasher.bin > 100% : done. [MBR. 1]: 40960 KB | ./eflasher/boot.img > 100% : done. [MBR. 2]: 654460 KB | ./eflasher/rootfs.img > 100% : done.

`

Also I notice that your VM is on Ubuntu 20.04 (focal), while I'm on Ubuntu 22.04 (Jammy). Regarding the installed package in your logs :

The following additional packages will be installed: android-libbase android-liblog android-libsparse libparted2 swig4.0 Suggested packages: libparted-dev libparted-i18n parted-doc openssh-server swig-doc swig-examples swig4.0-examples swig4.0-doc The following NEW packages will be installed: android-libbase android-liblog android-libsparse android-sdk-libsparse-utils exfat-fuse exfat-utils libparted2 parted rsync swig swig4.0 wget 0 upgraded, 12 newly installed, 0 to remove and 36 not upgraded. Need to get 1980 kB of archives. After this operation, 8245 kB of additional disk space will be used. Do you want to continue? [Y/n] Y

I do not get the message at all, so I tried to install them manually and the exfat-utils packet seems to be unavailable on this version of Ubuntu. The log says "Le paquet « exfat-utils » n'a pas de version susceptible d'être installée".

Thanks again for your time and answers, we seems close to find a solution.

friendlyarm commented 5 months ago

You can actually start with a new ubuntu vm without installing packages, the script will install the required packages, see https://github.com/friendlyarm/sd-fuse_h3/blob/master/tools/util.sh for more details

siteswapjuggler commented 5 months ago

Hi ok then it did actually work from a fresh install, I just had to do a sudo apt update and a sudo apt install git. I'm doing an upgrade right know to see if something is breaking it or not.

Ok it is still working after the upgrade, I don't really understand what went wrong before because I was already started from an almost fresh install but well issue closed to me :)

Thanks a lot for your help and for the updates.

friendlyarm / sd-fuse_h3