search

NeccoNeko / UNVR-diy-os

20 stars 2 forks source link

New hardware version #2

Open blark opened 2 years ago

blark commented 2 years ago

I bought a UNVR with similar intentions. I found that they have a new hardware version with an integrated emmc/flash. There is no longer a USB port.

Would you be interested in pull requests with more information here?

Also I'd like to chat with you about your progress. What's the best way to get in touch?

blark commented 2 years ago

Dropping some info here from the u-boot log: 

stage2_loader v2.22.3
SPD I2C Address: 57
Executing next!

-----------------------------------------------------
Stage 3 version: 2.22.0
Commit ID: 6088bc3
CVOS commit ID: bac1d52
HAL commit ID: 61afa9c
Build date: Sep  8 2020 11:40:22
-----------------------------------------------------

agent_wakeup v2.10
EEPROM Revision ID = 39
Device ID = a324
Device Info: v2sil-39-rc1
subsystem id: 0xea1a
hardware revision id: 0x000c7828
instance_num =  3
Loading DT to 01100000 (26236 bytes)...
obj_hdr_dt_offset: 0x98000
Board config ID: alpine_v2_ubnt one nas bt v1.0
Loading application to 01100000 (689728 bytes)...
Executing application...

U-Boot 2015.07-alpine_db-2.21-HAL (Dec 16 2020 - 05:54:51 +0800), Build: jenkins-amaz-alpinev2-boot-master-161
blark commented 2 years ago

Hey, update:

I requested the latest kernel sources from Ubiquiti, they came through with amazing speed. And the Kernel compiled without any issues on the first try. I will post them on my GitHub shortly.

I figured out how to power on the SATA drives in u-boot, load a custom kernel, and boot to a new Debian 11 OS.

For now I'm using Ubiquiti's kernel with all the necessary modules built in as it would be way too much work to port that over to 5.x. I am considering this a win, as now I have complete control of the device and all the hardware works. 

root@debian:/mnt/sdb2# dmesg |head
[    0.000000] Booting Linux on physical CPU 0x0000000000 [0x411fd073]
[    0.000000] Linux version 4.19.152 (root@ip-172-30-3-63) (gcc version 8.3.0 (Debian 8.3.0-6)) #1 SMP PREEMPT Wed Dec 29 20:06:40 UTC 2021
[    0.000000] Machine model: Annapurna Labs Alpine V2 UBNT
[    0.000000] efi: Getting EFI parameters from FDT:
[    0.000000] efi: UEFI not found.
[    0.000000] cma: Reserved 32 MiB at 0x00000000be000000
[    0.000000] NUMA: No NUMA configuration found
[    0.000000] NUMA: Faking a node at [mem 0x0000000000000000-0x000000023fffffff]
[    0.000000] NUMA: NODE_DATA [mem 0x23ffc8c40-0x23ffca3ff]
[    0.000000] Zone ranges:
root@debian:/mnt/sdb2# lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description:    Debian GNU/Linux 11 (bullseye)
Release:        11
Codename:       bullseye
root@debian:/mnt/sdb2# uname -a
Linux debian 4.19.152 #1 SMP PREEMPT Wed Dec 29 20:06:40 UTC 2021 aarch64 GNU/Linux
javy- commented 2 years ago

@blark very cool!

Do you know if the RAID array that gets created in the NVR admin is a HW or SW array?

Also, what else might need to be done to completely de-ubiquiti the device? flashing the EEPROM?

Are you able to test if the SFP+ port works in Debian? Any other drivers/scripts that might be needed (ex: power status indicator light)?

blark commented 2 years ago

Do you know if the RAID array that gets created in the NVR admin is a HW or SW array?

Haven't dug into that too much (I will probably use ZFS so I'm not too concerned). However, lspci shows: 

root@debian:~# lspci
0000:00:01.0 Ethernet controller: Annapurna Labs Ltd. Gigabit Ethernet Adapter (rev 02)
0000:00:02.0 Ethernet controller: Annapurna Labs Ltd. SFP+ 10G Ethernet Adapter (rev 03)
0000:00:04.0 Network and computing encryption device: Annapurna Labs Ltd. Device 0022 (rev 02)
0000:00:05.0 RAID bus controller: Annapurna Labs Ltd. Device 0022 (rev 02)
0000:00:08.0 SATA controller: Annapurna Labs Ltd. Device 0031 (rev 02)
0000:00:09.0 SATA controller: Annapurna Labs Ltd. Device 0031 (rev 02)
0001:00:00.0 PCI bridge: Annapurna Labs Ltd. Device 0031 (rev 01)
0001:01:00.0 USB controller: ASMedia Technology Inc. ASM1042A USB 3.0 Host Controller

Note: as you can probably see here the device actually has 8 sata ports ;) I'm assuming they used this for prototyping the UNVR pro and it was just cheaper to have the same HW. I have confirmed that the unpopulated SATA headers work by soldering on a connector and attaching a drive.

Also, what else might need to be done to completely de-ubiquiti the device? flashing the EEPROM?

I think I'm there. There is a USB disk (now soldered on) which I won't touch, because it's slow anyway (and it's nice to leave intact just incase). Then there is SPI flash, which I have told u-boot to ignore (has kernel/recovery disk), and there is NAND (which I haven't figured out how to access yet.. uboot doesn't see it). I haven't played with eeprom, which I think is where uboot lives... I don't have an adapter to reflash the chip if I brick the device, so I will wait on that for now. Either way it's skipping all of that stuff when u-boot boots from SATA.

Are you able to test if the SFP+ port works in Debian? Any other drivers/scripts that might be needed (ex: power status indicator light)?

I will test 10gig over the next few days. I have a UniFi switch aggregation, and some mikrotik gear with 10gb sfp. There are definitely some software LED controls, but I know at least disk activity LEDs seem to work with whatever is built into the kernel. We'll never be able to be completely free of Ubiquiti because all the LEDs, HDD power, and probably other stuff needs kernel modules to work. Specifically hdd_pwrctl because they have a GPIO expander controlling a power circuit that flips power on/off for drives. Without that the drives won't be powered on. It took me a long time to figure out how to get them on in u-boot heh.

blark commented 2 years ago

@javy- can report some 1gbit testing over the last few days - haven't been able to test with 10gbit yet as my desktop doesn't want to boot with that NIC installed.

Anyway, I sent over 650gb to my zfs mirror on the UNVR via rsync over ssh, with average speed of 81,178,622.74 bytes/sec. I am sending over 50gb now via SMB and Windows is reporting ~100-110MB/s average. I'm using the 10gbit NIC on the UNVR. I'm very happy so far, it's crushing the speed of my old QNAP that was way more costly. I'm guessing the rsync slowdown was due to SSH encryption. Oh and the zfs mirror has compression turned on.

javy- commented 2 years ago

@blark alot of progress, nice work.

just out of curiosity, if you get a chance can you run iperf from your router to the NVR? sounds like the 10gbit NIC does fine out of the box but Im curious if whatever driver it is using in Debian will provide full theoretical bandwidth.

navierb commented 2 years ago

I am also interested in the whole process (the UNVR is a sexy device) but I have not that much knowledge* and would need some tutorial (a detailed step-step) about this.

(*) I removed the boot-disk on my QNAP and successfully running debian 11 with ZFS from the openzfs guide for the last half year.

m-byte commented 2 years ago

Motivated by what I read here, I recently picked up my own UNVR. A very brief description of what I already did (without the roadblocks I hit):

All this could also be done without voiding the warranty by overwriting the envdata in /dev/mtd# (don't remember, which one it was), however, I ran into issues with initramfs and had to get debug access anyway... Even cleaner would be a proper initramfs, but I'd need help from someone who has built those before. In theory, having an initramfs with busybox and dropbear would allow us to make this a one-step process where only uImage would have to be replaced and everything else (possibly even the backup) could be done automatically or via ssh.

Next on my to do list is setting this up as router with a dockerized nas and 10Gb/s connection to my switch.

Btw: with a proper initramfs, it would also be possible to have the rootfs on a HDD without touching u-boot. I didn't want to go that way, though, as I want to keep everything required for booting off those drives.

m-byte commented 2 years ago

Seems like I just forgot to clean my source tree in between some initramfs tests... I'm able to build with initramfs now. Also, I started my own initramfs based on buildroot. SSH already works, still need to put debootstrap in there for a clean/easy migration between UnifiOS and this image. No need to void any warranties, then, since this allows for putting an image back onto the system. It will only be necessary to put the this uImage and a config file into /dev/boot1. From there, the rest of the setup could then be done automatically.

m-byte commented 2 years ago

I managed to build my own initramfs to go with the kernel sources from ubiquiti. Since I don't want the ubiquiti stuff, it just nukes the (in my case builtin) USB drive except for the kernel partition and creates one that uses the maximum amount of space available. Then it uses debootstrap to load a basic debian rootfs into the new partition. Last, it modifies the config file (placed in the kernel partition with my uImage) to clear the setup flag and does a switch_root to load the debian system. If it fails to detect the USB drive or a special flag is set in the config file, it starts a dropbear instance that can be used for figuring out what is going on without opening the box (and e.g. uploading a different uImage). If setup is done and the USB drive cna be found, it justs immediately does a switch_root to debian.

For anyone interested, I might be able to send you either a full uImage or the initramfs to compile with your own kernel on short notice. This should work with both old and new UNVRs. Once I find some time, I will try to properly pack this and put everything into a repo (with sources).

NeccoNeko commented 2 years ago

I bought a UNVR with similar intentions. I found that they have a new hardware version with an integrated emmc/flash. There is no longer a USB port.

Would you be interested in pull requests with more information here?

Also I'd like to chat with you about your progress. What's the best way to get in touch?

Sorry for missing this.

You've provided a lot of fantastic information! Pull requests with the information you collected would be awesome!

m-byte commented 2 years ago

For anyone interested: I cleaned up my code a bit and published everything at @urnvr. The code should work on new and old versions, but has only been tested on my new UNVR (I only have that one).

I also uploaded the GPL code I got there and just asked them to also send me code for the version that just came out... If they do, that'll also go in there.

Pull requests and comments are welcome.

blark commented 2 years ago

@m-byte awesome stuff. I got mine running and have been ignoring it for a while. Work and life have been busy! I'm going check out your repo as soon as I have some time.

jvic1234 commented 1 year ago

Update?