Closed elcheco closed 2 years ago
bretton
commented
2 years ago Hi @elcheco, sorry you're having issues. Do you get anything with:
zpool list
zpool importalso what does the following produce:
geom disk listcan also try clearing the partitions with
zpool labelclear -f /dev/ada0
zpool labelclear -f /dev/ada1any idea why the zfsinstall was killed?
You should be able to reboot into recovery console and re-run again?
grembo
commented
2 years ago @elcheco This is how you can clean up the drives:
zpool export -f zroot
zpool destroy -f zroot
zpool labelclear -f /dev/ada0p3
zpool labelclear -f /dev/ada1p3
gpart destroy -F ada0
gpart destroy -F ada1Some commands will fail, which is ok. After the procedure:
zpool import should show no poolsgpart show ada0 and gpart show ada1 should show No such geomWhat kind of server were you using? Maybe you ran out of RAM during install.
@bretton What about adding a script for that (it's super destructive though)?
Something like _mfsbsdclean.sh:
#!/bin/sh
# THIS SCRIPT WILL EAT YOUR DATA
if [ $# -lt 2 ]; then
echo "Usage: $0 pool device..." 1>&2
echo "Example: $0 zroot ada0 ada1" 1>&2
exit 1
fi
pool="$1"
shift
echo "YOU ARE ABOUT TO DESTROY ZPOOL '$pool' AND PARTITIONS $*"
echo "THIS OPERATION MEANS DATA LOSS AND CANNOT BE UNDONE"
echo "Please type 'LETSLOSEDATA' and press enter to continue."
read -r ANSWER
if [ "$ANSWER" != "LETSLOSEDATA" ]; then
echo "Canceled. Phew!"
exit 1
fi
zpool export -f "$pool"
zpool destroy -f "$pool"
for dev in "$@"; do
# lazy
for p in $(jot 9); do
zpool labelclear -f "/dev/${dev}p${p}"
done
gpart destroy -F "$dev"
done
echo "DONE"@bretton What about adding a script for that (it's super destructive though)? Something like _mfsbsdclean.sh:
sure can add something and have a line in the script to download it.
better testing of full install required too, will sort out a temp box.
bretton
commented
2 years ago Can confirm same issue with ax41 host, 2 x 2TB hard drives
mfsbsd@mfsbsd:~ $ sudo su -
root@mfsbsd:~ # zfsinstall -d ada0 -d ada1 -r mirror -s 4G -A -4 -c -p zroot
Fetching base files from: ftp://ftp.freebsd.org/pub/FreeBSD/releases/amd64/13.1-RELEASE
/tmp/base.txz 186 MB 25 MBps 07s
/tmp/kernel.txz 49 MB 21 MBps 02s
Creating GUID partitions on ada0 ... done
Configuring ZFS bootcode on ada0 ... done
=> 40 3907029088 ada0 GPT (1.8T)
40 472 1 freebsd-boot (236K)
512 3584 - free - (1.8M)
4096 8388608 2 freebsd-swap (4.0G)
8392704 3898634240 3 freebsd-zfs (1.8T)
3907026944 2184 - free - (1.1M)
Creating GUID partitions on ada1 ... done
Configuring ZFS bootcode on ada1 ... done
=> 40 3907029088 ada1 GPT (1.8T)
40 472 1 freebsd-boot (236K)
512 3584 - free - (1.8M)
4096 8388608 2 freebsd-swap (4.0G)
8392704 3898634240 3 freebsd-zfs (1.8T)
3907026944 2184 - free - (1.1M)
Creating ZFS pool zroot on ada0p3 ada1p3 ... done
Setting default checksum to fletcher4 for zroot ... done
Enabling default compression on zroot ... done
Creating zroot root partition: ... done
Creating zroot partitions: var tmp ... done
Setting bootfs for zroot to zroot/root ... done
NAME USED AVAIL REFER MOUNTPOINT
zroot 248K 1.76T 24K none
zroot/root 72K 1.76T 24K /mnt
zroot/root/tmp 24K 1.76T 24K /mnt/tmp
zroot/root/var 24K 1.76T 24K /mnt/var
Extracting FreeBSD distribution ...Killed
errordmesg output shows:
ada0 at ata0 bus 0 scbus0 target 0 lun 0
ada0: <QEMU HARDDISK 2.5+> ATA-7 device
ada0: Serial Number QM00001
ada0: 16.700MB/s transfers (WDMA2, PIO 8192bytes)
ada0: 1907729MB (3907029168 512 byte sectors)
ada1 at ata0 bus 0 scbus0 target 1 lun 0
ada1: <QEMU HARDDISK 2.5+> ATA-7 device
ada1: Serial Number QM00002
ada1: 16.700MB/s transfers (WDMA2, PIO 8192bytes)
ada1: 1907729MB (3907029168 512 byte sectors)
random: unblocking device.
lo0: link state changed to UP
em0: link state changed to UP
ZFS filesystem version: 5
ZFS storage pool version: features support (5000)
pid 1044 (bsdtar), jid 0, uid 0, was killed: failed to reclaim memoryno memory or disk space, it should be pulling the install files from local, not remote and extracting?
Mem: 307M Active, 220K Inact, 195M Laundry, 203M Wired, 67M Buf, 48M Free
root@mfsbsd:~ # df -h
Filesystem Size Used Avail Capacity Mounted on
/dev/md0 82M 71M 5.3M 93% /@elcheco This is how you can clean up the drives:
zpool export -f zroot zpool destroy -f zroot zpool labelclear -f /dev/ada0p3 zpool labelclear -f /dev/ada1p3 gpart destroy -F ada0 gpart destroy -F ada1Some commands will fail, which is ok. After the procedure:
zpool importshould show no poolsgpart show ada0andgpart show ada1should showNo such geom
confirming these steps with console dump
root@mfsbsd:~ # zpool export -f zroot
root@mfsbsd:~ # zpool destroy -f zroot
cannot open 'zroot': no such pool
root@mfsbsd:~ # zpool labelclear -f /dev/ada0p3
root@mfsbsd:~ # zpool labelclear -f /dev/ada1p3
root@mfsbsd:~ # gpart destroy -F ada0
ada0 destroyed
root@mfsbsd:~ # gpart destroy -F ada1
ada1 destroyed
root@mfsbsd:~ # zpool import
no pools available to import
root@mfsbsd:~ # gpart show ada0
gpart: No such geom: ada0.
root@mfsbsd:~ # gpart show ada1
gpart: No such geom: ada1.I think it's the qemu settings, maybe add more memory, and zfsinstall should use files on image?
elcheco
commented
2 years ago Dedicated Root Server
geom disk list:
root@mfsbsd:~ # geom disk list
Geom name: cd0
Providers:
1. Name: cd0
Mediasize: 94005248 (90M)
Sectorsize: 2048
Mode: r0w0e0
descr: QEMU QEMU DVD-ROM
ident: (null)
rotationrate: unknown
fwsectors: 0
fwheads: 0
Geom name: cd1
Providers:
1. Name: cd1
Mediasize: 372736 (364K)
Sectorsize: 2048
Mode: r0w0e0
descr: QEMU QEMU DVD-ROM
ident: (null)
rotationrate: unknown
fwsectors: 0
fwheads: 0
Geom name: ada0
Providers:
1. Name: ada0
Mediasize: 512110190592 (477G)
Sectorsize: 512
Mode: r1w1e2
descr: QEMU HARDDISK
ident: QM00001
rotationrate: unknown
fwsectors: 63
fwheads: 16
Geom name: ada1
Providers:
1. Name: ada1
Mediasize: 512110190592 (477G)
Sectorsize: 512
Mode: r1w1e2
descr: QEMU HARDDISK
ident: QM00002
rotationrate: unknown
fwsectors: 63
fwheads: 16increasing memory to 8gb in qemu options in run.sh allows script to proceed in preliminary testing, will update asap
root@mfsbsd:~ # zfsinstall -d ada0 -d ada1 -r mirror -s 4G -A -4 -c -p zroot
Fetching base files from: ftp://ftp.freebsd.org/pub/FreeBSD/releases/amd64/13.1-RELEASE
/tmp/base.txz 186 MB 26 MBps 07s
/tmp/kernel.txz 49 MB 24 MBps 02s
Creating GUID partitions on ada0 ... done
Configuring ZFS bootcode on ada0 ... done
=> 40 3907029088 ada0 GPT (1.8T)
40 472 1 freebsd-boot (236K)
512 3584 - free - (1.8M)
4096 8388608 2 freebsd-swap (4.0G)
8392704 3898634240 3 freebsd-zfs (1.8T)
3907026944 2184 - free - (1.1M)
Creating GUID partitions on ada1 ... done
Configuring ZFS bootcode on ada1 ... done
=> 40 3907029088 ada1 GPT (1.8T)
40 472 1 freebsd-boot (236K)
512 3584 - free - (1.8M)
4096 8388608 2 freebsd-swap (4.0G)
8392704 3898634240 3 freebsd-zfs (1.8T)
3907026944 2184 - free - (1.1M)
Creating ZFS pool zroot on ada0p3 ada1p3 ... done
Setting default checksum to fletcher4 for zroot ... done
Enabling default compression on zroot ... done
Creating zroot root partition: ... done
Creating zroot partitions: var tmp ... done
Setting bootfs for zroot to zroot/root ... done
NAME USED AVAIL REFER MOUNTPOINT
zroot 248K 1.76T 24K none
zroot/root 72K 1.76T 24K /mnt
zroot/root/tmp 24K 1.76T 24K /mnt/tmp
zroot/root/var 24K 1.76T 24K /mnt/var
Extracting FreeBSD distribution ... done
Writing /boot/loader.conf... done
Writing /etc/fstab...Writing /etc/rc.conf... done
Copying /boot/zfs/zpool.cache ... done
Installation complete.
The system will boot from ZFS with clean install on next reboot
You may make adjustments to the installed system using chroot:
chroot /mnt
Some adjustments may require a mounted devfs:
mount -t devfs devfs /mnt/dev
WARNING - Don't export ZFS pool "zroot"!Dedicated Root Server - Intel Xeon E3-1275V6 - 2x SSD M.2 NVMe 512 GB - 64GB RAM (4x RAM 16384 MB DDR4 ECC)
thanks @elcheco it should be fixed now, qemu will load with 8gb memory and you can proceed normally.
bretton
commented
2 years ago Reboot didn't bring up a network so additional documentation/testing might be required, add config to /etc/rc.conf before reboot maybe? thought mfsbsd did all this?
grembo
commented
2 years ago Reboot didn't bring up a network so additional documentation/testing might be required, add config to /etc/rc.conf before reboot maybe? thought mfsbsd did all this?
It probably created an ifconfig entry for vnet0 or em0, but the physical interface outside qemu is a different one.
grembo
commented
2 years ago @bretton In addition, ideally a setup at Hetzner looks like below, which is not somethings mfsBSD does by default:
Example:
Server IP: 1.2.3.4
Gateway IP: 6.7.8.9
IPv6 Prefix: 1234::123:123:1234::/64
ifconfig_igb0_name="untrusted"
ifconfig_untrusted="up
ifconfig_untrusted_ipv6="up"
ifconfig_untrusted_aliases="inet 1.2.3.4/32 inet6 1234::123:123:1234::2/64"
static_routes="gateway default"
route_gateway="-host 6.7.8.9 -interface untrusted"
route_default="default 6.7.8.9"
ipv6_defaultrouter="fe80::1%untrusted"Renaming the interface is my Spiel, so the role of the interface is clear and changing the interface only requires a one line change in rc.conf instead multiple ones + changes in rc.conf. It all also works with spelling out igb0 instead.
This configuration is required so that servers on the same network can talk to each other (Hetzner is locking down the switchports so that servers can't break the local network by accident or on purpose). To be fair, this is something mfsBSD never did in the past and might be beyond the scope of this script anyway, as it moves away from a simple DHCP setup (users with only one server or servers on different networks won't have to bother anyway).
bretton
commented
2 years ago bretton
commented
2 years ago @elcheco did you manage to wipe disks and install again?
bretton
commented
2 years ago closing so long, can be re-opened
I tried to install FreeBSD based on the: https://depenguin.me/
In the zfsinstall step it died and I cannot run it again:
Installation
If I try it again
DF
Whatever I try it fails:
Any ideas how to continue or restart completely?