search

nongiach / arm_now

arm_now is a qemu powered tool that allows instant setup of virtual machines on arm cpu, mips, powerpc, nios2, x86 and more, for reverse, exploit, fuzzing and programming purpose.
https://github.com/nongiach/arm_now/wiki
MIT License
867 stars 87 forks source link

mips32 opkg missing? #15

Open bannsec opened 6 years ago

bannsec commented 6 years ago

Looks like opkg is missing from the mips32 image:

arm_now start mips32
# opkg
-sh: opkg: not found
nongiach commented 6 years ago

Yeah opkg is not available on all arch yet, but you can use mips32el We still need to find mirrors for all arch, help needed :) https://github.com/nongiach/arm_now/wiki/4.1-Q&A-and-incoming-features#need-more-package-manager

cpatulea commented 2 years ago

Q&A entry suggests:

Search a package manager for all arch like => https://wiki.openwrt.org/about/mirrors or http://pkg.entware.net/binaries/

It is a good idea but there is a complication, I think it needs to match libc between arm_now and donor project. For example, arm_now selects glibc by default (https://github.com/nongiach/arm_now/blob/master/arm_now/download.py#L59) while currently OpenWrt defaults to musl (and used to be uClibc).

Note bootlin does have mips musl test-systems: https://toolchains.bootlin.com/downloads/releases/toolchains/mips32/test-system/

PS. Great project arm_now!

cpatulea commented 2 years ago

Tried to manually build arm_now directory using mips32--musl--stable-2021.11-1-rootfs.ext2 but having some issue..

$ mkdir arm_now
$ cd arm_now
arm_now$ echo -n mips32 >arch
arm_now$ curl -o rootfs.ext2 https://toolchains.bootlin.com/downloads/releases/toolchains/mips32/test-system/mips32--musl--stable-2021.11-1-rootfs.ext2
arm_now$ curl -o kernel https://toolchains.bootlin.com/downloads/releases/toolchains/mips32/test-system/mips32--musl--stable-2021.11-1-vmlinux
$ cd ..
$ arm_now start mips32
WARNING: arm_now/ already exists, use --clean to restart with a fresh filesystem
File not found by ext2_lookup
WARNING: e2rm file already suppressed
File not found by ext2_lookup
WARNING: e2rm file already suppressed
Tempdir /tmp/tmpsubkhjyz
File not found by ext2_lookup
WARNING: e2rm file already suppressed
Starting qemu-system-mips
stty intr ^]
       export QEMU_AUDIO_DRV="none"
       qemu-system-mips -kernel arm_now/kernel -hda arm_now/rootfs.ext2 -append 'root=/dev/hda console=ttyS0 rw physmap.enabled=0 noapic'                -m 256M                -nographic                -serial stdio -monitor null  -nic user                                 -no-reboot
       stty intr ^c

WARNING: Image format was not specified for 'arm_now/rootfs.ext2' and probing guessed raw.
         Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted.
         Specify the 'raw' format explicitly to remove the restrictions.
Linux version 5.15.0 (br-user@runner-nthfetyx-project-3290221-concurrent-0) (mips-linux-gcc.br_real (Buildroot toolchains.bootlin.com-2021.11-1) 10.3.0, GNU ld (GNU Binutils) 2.36.1) #1 SMP Mon Dec 27 10:07:26 UTC 2021
earlycon: uart8250 at I/O port 0x3f8 (options '38400n8')
printk: bootconsole [uart8250] enabled
CPU0 revision is: 00019300 (MIPS 24Kc)
FPU revision is: 00739300
OF: fdt: No chosen node found, continuing without
MIPS: machine is mti,malta
Software DMA cache coherency enabled
Primary instruction cache 2kB, VIPT, 2-way, linesize 16 bytes.
Primary data cache 2kB, 2-way, VIPT, no aliases, linesize 16 bytes
Zone ranges:
  DMA      [mem 0x0000000000000000-0x0000000000ffffff]
  Normal   [mem 0x0000000001000000-0x000000000fffefff]
Movable zone start for each node
Early memory node ranges
  node   0: [mem 0x0000000000000000-0x000000000fffefff]
Initmem setup node 0 [mem 0x0000000000000000-0x000000000fffefff]
percpu: Embedded 10 pages/cpu s11728 r8192 d21040 u40960
Built 1 zonelists, mobility grouping on.  Total pages: 65023
Kernel command line: root=/dev/hda console=ttyS0 rw physmap.enabled=0 noapic
Unknown command line parameters: noapic
Dentry cache hash table entries: 32768 (order: 5, 131072 bytes, linear)
Inode-cache hash table entries: 16384 (order: 4, 65536 bytes, linear)
Writing ErrCtl register=00000000
Readback ErrCtl register=00000000
mem auto-init: stack:off, heap alloc:off, heap free:off
Memory: 251048K/262140K available (5708K kernel code, 593K rwdata, 1028K rodata, 212K init, 214K bss, 11092K reserved, 0K cma-reserved)
SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
rcu: Hierarchical RCU implementation.
rcu:    RCU restricting CPUs from NR_CPUS=2 to nr_cpu_ids=1.
rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1
NR_IRQS: 256
random: get_random_bytes called from start_kernel+0x49c/0x650 with crng_init=0
CPU frequency 333.33 MHz
clocksource: MIPS: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 11467770521 ns
sched_clock: 32 bits at 166MHz, resolution 6ns, wraps every 12885135356ns
Console: colour dummy device 80x25
Calibrating delay loop... 897.02 BogoMIPS (lpj=1794048)
pid_max: default: 32768 minimum: 301
Mount-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
rcu: Hierarchical SRCU implementation.
smp: Bringing up secondary CPUs ...
smp: Brought up 1 node, 1 CPU
devtmpfs: initialized
clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
futex hash table entries: 256 (order: 2, 16384 bytes, linear)
NET: Registered PF_NETLINK/PF_ROUTE protocol family
vgaarb: loaded
SCSI subsystem initialized
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
pps_core: LinuxPPS API ver. 1 registered
pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
PTP clock support registered
PCI host bridge to bus 0000:00
pci_bus 0000:00: root bus resource [mem 0x10000000-0x17ffffff]
pci_bus 0000:00: root bus resource [io  0x1000-0x1fffff]
pci_bus 0000:00: No busn resource found for root bus, will use [bus 00-ff]
pci 0000:00:00.0: [11ab:4620] type 00 class 0x060000
pci 0000:00:00.0: [Firmware Bug]: reg 0x14: invalid BAR (can't size)
pci 0000:00:00.0: [Firmware Bug]: reg 0x18: invalid BAR (can't size)
pci 0000:00:00.0: [Firmware Bug]: reg 0x1c: invalid BAR (can't size)
pci 0000:00:00.0: [Firmware Bug]: reg 0x20: invalid BAR (can't size)
pci 0000:00:00.0: [Firmware Bug]: reg 0x24: invalid BAR (can't size)
pci 0000:00:0a.0: [8086:7110] type 00 class 0x060100
pci 0000:00:0a.1: [8086:7111] type 00 class 0x010180
pci 0000:00:0a.1: reg 0x20: [io  0x0000-0x000f]
pci 0000:00:0a.1: legacy IDE quirk: reg 0x10: [io  0x01f0-0x01f7]
pci 0000:00:0a.1: legacy IDE quirk: reg 0x14: [io  0x03f6]
pci 0000:00:0a.1: legacy IDE quirk: reg 0x18: [io  0x0170-0x0177]
pci 0000:00:0a.1: legacy IDE quirk: reg 0x1c: [io  0x0376]
pci 0000:00:0a.2: [8086:7112] type 00 class 0x0c0300
pci 0000:00:0a.2: reg 0x20: [io  0x0000-0x001f]
pci 0000:00:0a.3: [8086:7113] type 00 class 0x068000
pci 0000:00:0a.3: quirk: [io  0x1000-0x103f] claimed by PIIX4 ACPI
pci 0000:00:0a.3: quirk: [io  0x1100-0x110f] claimed by PIIX4 SMB
pci 0000:00:0b.0: [1022:2000] type 00 class 0x020000
pci 0000:00:0b.0: reg 0x10: [io  0x0000-0x001f]
pci 0000:00:0b.0: reg 0x14: [mem 0x00000000-0x0000001f]
pci 0000:00:0b.0: reg 0x30: [mem 0x00000000-0x0003ffff pref]
pci 0000:00:12.0: [1013:00b8] type 00 class 0x030000
pci 0000:00:12.0: reg 0x10: [mem 0x00000000-0x01ffffff pref]
pci 0000:00:12.0: reg 0x14: [mem 0x00000000-0x00000fff]
pci 0000:00:12.0: reg 0x30: [mem 0x00000000-0x0000ffff pref]
pci 0000:00:12.0: vgaarb: VGA device added: decodes=io+mem,owns=none,locks=none
pci_bus 0000:00: busn_res: [bus 00-ff] end is updated to 00
pci 0000:00:12.0: BAR 0: assigned [mem 0x10000000-0x11ffffff pref]
pci 0000:00:0b.0: BAR 6: assigned [mem 0x12000000-0x1203ffff pref]
pci 0000:00:12.0: BAR 6: assigned [mem 0x12040000-0x1204ffff pref]
pci 0000:00:12.0: BAR 1: assigned [mem 0x12050000-0x12050fff]
pci 0000:00:0a.2: BAR 4: assigned [io  0x1040-0x105f]
pci 0000:00:0b.0: BAR 0: assigned [io  0x1060-0x107f]
pci 0000:00:0b.0: BAR 1: assigned [mem 0x12051000-0x1205101f]
pci 0000:00:0a.1: BAR 4: assigned [io  0x1080-0x108f]
clocksource: Switched to clocksource MIPS
NET: Registered PF_INET protocol family
IP idents hash table entries: 4096 (order: 3, 32768 bytes, linear)
tcp_listen_portaddr_hash hash table entries: 512 (order: 0, 6144 bytes, linear)
TCP established hash table entries: 2048 (order: 1, 8192 bytes, linear)
TCP bind hash table entries: 2048 (order: 2, 16384 bytes, linear)
TCP: Hash tables configured (established 2048 bind 2048)
UDP hash table entries: 256 (order: 1, 8192 bytes, linear)
UDP-Lite hash table entries: 256 (order: 1, 8192 bytes, linear)
NET: Registered PF_UNIX/PF_LOCAL protocol family
pci 0000:00:0a.2: enabling device (0000 -> 0001)
PCI: CLS 0 bytes, default 16
workingset: timestamp_bits=30 max_order=16 bucket_order=0
Block layer SCSI generic (bsg) driver version 0.4 loaded (major 252)
io scheduler mq-deadline registered
io scheduler kyber registered
cirrusfb 0000:00:12.0: enabling device (0000 -> 0002)
cirrusfb 0000:00:12.0: Cirrus Logic chipset on PCI bus, RAM (4096 kB) at 0x10000000
Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
printk: console [ttyS0] disabled
serial8250.0: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A
serial8250.0: ttyS0 extra baud rates supported: 230400, 460800
serial8250.0: ttyS0 extra baud rates supported: 230400, 460800
printk: console [ttyS0] enabled
printk: console [ttyS0] enabled
printk: bootconsole [uart8250] disabled
printk: bootconsole [uart8250] disabled
serial8250.0: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A
serial8250.0: ttyS1 extra baud rates supported: 230400, 460800
serial8250.0: ttyS2 at MMIO 0x1f000900 (irq = 20, base_baud = 230400) is a 16550A
ata_piix 0000:00:0a.1: enabling device (0000 -> 0001)
scsi host0: ata_piix
scsi host1: ata_piix
ata1: PATA max UDMA/33 cmd 0x1f0 ctl 0x3f6 bmdma 0x1080 irq 14
ata2: PATA max UDMA/33 cmd 0x170 ctl 0x376 bmdma 0x1088 irq 15
pcnet32 0000:00:0b.0: enabling device (0000 -> 0003)
pcnet32: PCnet/PCI II 79C970A at 0x1060, 52:54:00:12:34:56 assigned IRQ 10
pcnet32: eth0: registered as PCnet/PCI II 79C970A
pcnet32: 1 cards_found
uhci_hcd: USB Universal Host Controller Interface driver
uhci_hcd 0000:00:0a.2: UHCI Host Controller
uhci_hcd 0000:00:0a.2: new USB bus registered, assigned bus number 1
uhci_hcd 0000:00:0a.2: irq 11, io base 0x00001040
hub 1-0:1.0: USB hub found
hub 1-0:1.0: 2 ports detected
usbcore: registered new interface driver usbhid
usbhid: USB HID core driver
NET: Registered PF_INET6 protocol family
Segment Routing with IPv6
In-situ OAM (IOAM) with IPv6
sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
NET: Registered PF_PACKET protocol family
ata1.00: ATA-7: QEMU HARDDISK, 2.5+, max UDMA/100
ata1.00: 122880 sectors, multi 16: LBA48
ata2.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100
scsi 0:0:0:0: Direct-Access     ATA      QEMU HARDDISK    2.5+ PQ: 0 ANSI: 5
scsi 1:0:0:0: CD-ROM            QEMU     QEMU DVD-ROM     2.5+ PQ: 0 ANSI: 5
sd 0:0:0:0: [sda] 122880 512-byte logical blocks: (62.9 MB/60.0 MiB)
sd 0:0:0:0: [sda] Write Protect is off
sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
sd 0:0:0:0: [sda] Attached SCSI disk
VFS: Cannot open root device "hda" or unknown-block(0,0): error -6
Please append a correct "root=" boot option; here are the available partitions:
0800           61440 sda
 driver: sd
Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)
---[ end Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0) ]---
cpatulea commented 2 years ago

However with directly launching qemu on the rootfs.ext2, it seems to work:

arm_now$ curl -o rootfs.ext2 https://toolchains.bootlin.com/downloads/releases/toolchains/mips32/test-system/mips32--musl--stable-2021.11-1-rootfs.ext2
$ qemu-system-mips -M malta -kernel kernel -drive file=rootfs.ext2,format=raw -append "rootwait root=/dev/hda " -net nic,model=pcnet -net user  -nographic
Linux version 4.11.3 (root@runner-fa6cab46-project-3290221-concurrent-0) (gcc version 6.4.0 (Buildroot 2018.02-rc2-00006-g39101b7)
...
EXT4-fs (hda): couldn't mount as ext3 due to feature incompatibilities
EXT4-fs (hda): mounting ext2 file system using the ext4 subsystem
EXT4-fs (hda): mounted filesystem without journal. Opts: (null)
VFS: Mounted root (ext2 filesystem) readonly on device 3:0.
devtmpfs: mounted
Freeing unused kernel memory: 268K
This architecture does not have kernel memory protection.
EXT4-fs (hda): re-mounted. Opts: block_validity,barrier,user_xattr
Starting syslogd: OK
Starting klogd: OK
Running sysctl: OK
Saving random seed: OK
Starting network: pcnet32 0000:00:0b.0 eth0: link up
udhcpc: started, v1.34.1
udhcpc: broadcasting discover
udhcpc: broadcasting select for 10.0.2.15, server 10.0.2.2
udhcpc: lease of 10.0.2.15 obtained from 10.0.2.2, lease time 86400
deleting routers
adding dns 10.0.2.3
OK
Starting tests
TODO: add some more tests
Ending tests

Welcome to Buildroot
buildroot login: root
#
cpatulea commented 2 years ago

Ugh, and another complication: hardfloat vs softfloat. The bootlin image looks like hardfloat:

# cd /lib
lrwxrwxrwx    1 root     root            14 Dec 27 10:04 ld-musl-mips.so.1 -> ../lib/libc.so

while OpenWrt packages (example strace) look like softfloat (note -sf):

$ file usr/bin/strace
usr/bin/strace: ELF 32-bit MSB executable, MIPS, MIPS32 rel2 version 1 (SYSV), dynamically linked, interpreter /lib/ld-musl-mips-sf.so.1, no section header

so when trying to run the binary, it does not work:

# ./strace
-sh: ./strace: not found
cpatulea commented 2 years ago

However, as an alternative, OpenWrt initramfs works quite well under QEMU, and has opkg:

https://openwrt.org/docs/guide-user/virtualization/qemu#openwrt_in_qemu_mips

$ wget https://downloads.openwrt.org/releases/18.06.1/targets/malta/be/openwrt-18.06.1-malta-be-vmlinux-initramfs.elf
$ qemu-system-mips -kernel openwrt-18.06.1-malta-be-vmlinux-initramfs.elf -nographic -m 256
[    0.000000] Linux version 4.14.63 (buildbot@builds-03.infra.lede-project.org) (gcc version 7.3.0 (OpenWrt GCC 7.3.0 r7101-a63e38b)) #0 SMP Wed Aug 15 20:42:39 2018
...
Please press Enter to activate this console.
root@OpenWrt:/# opkg
opkg must have one sub-command argument
usage: opkg [options...] sub-command [arguments...]
where sub-command is one of:

Package Manipulation:
    update          Update list of available packages
    upgrade <pkgs>      Upgrade packages
    install <pkgs>      Install package(s)
    configure <pkgs>    Configure unpacked package(s)
    remove <pkgs|regexp>    Remove package(s)
    flag <flag> <pkgs>  Flag package(s)
     <flag>=hold|noprune|user|ok|installed|unpacked (one per invocation)