search

offensive-security / kali-nethunter

The Kali NetHunter Project
https://github.com/offensive-security/kali-nethunter/wiki
3.52k stars 890 forks source link

Nethunter-fs another build error #1257

Open yesimxev opened 5 years ago

yesimxev commented 5 years ago

Device:

armhf

Built from repo (date and build command) or downloaded from website (links):

github latest

Issue:

Nethunter-fs does not build anymore. Recent changes did work with my build few weeks ago, but kali changed after?

I: Configuring systemd...
I: Base system installed successfully.
[+] Installing qemu-arm-static interpreter to rootfs
[+] Starting debootstrap in chroot (second stage)...
chroot: failed to run command '/debootstrap/debootstrap': No such file or directory
[+] Completed stage 1!
[+] Starting stage 2 (repo/config)
Re4son commented 5 years ago

A few changes but they shouldn't cause this error. Can you please check if "qemu-arm-static" has actually been copied to "$rootfs/usr/bin/"?

yesimxev commented 5 years ago

Hmm, I was wondering the same. Checked, and not there. When I'm running from scratch, it copies there, but when doing a second kalifs, it removes chroot, but not copying qemu again. However, it is fine on a new clean VM, so I guess my kali is old for this, even with latest upgrades.

[+] Starting stage 1 (debootstrap)
update-binfmts: warning: qemu-arm already enabled in kernel.
[+] Starting debootstrap (download)...

Still not working properly though. Had to edit build.sh gcc-6 to gcc7, g++6 to g++-7 stage3: a2enmod php7.2 to a2enmod php7.3

But after these, looks like fuse cannot be configured:

Unpacking libntfs-3g883 (1:2017.3.23AR.3-2) ...
Setting up libfuse2:armhf (2.9.9-1) ...
Setting up fuse (2.9.9-1) ...
libkmod: ERROR ../libkmod/libkmod-module.c:1931 kmod_module_get_holders: could not open '/sys/module/vmw_vsock_vmci_transport/holders': No such file or directory
libkmod: ERROR ../libkmod/libkmod-module.c:1931 kmod_module_get_holders: could not open '/sys/module/vsock/holders': No such file or directory
..SNIP...
libkmod: ERROR ../libkmod/libkmod-module.c:1931 kmod_module_get_holders: could not open '/sys/module/scsi_mod/holders': No such file or directory
Unknown device "/dev/fuse": No such device
dpkg: error processing package fuse (--configure):
 installed fuse package post-installation script subprocess returned error exit status 1
Errors were encountered while processing:
 fuse

And the the rest of the packages not installing, but that's not worth to include here.

Re4son commented 5 years ago

I think we can ignore those fuse errors. It seems to work just fine once installed. I'm working on a few more issues and will push the fixes shortly

yesimxev commented 5 years ago

Would ignore but with that, the kalifs becomes 200mb and not installing many important packages when fuse gives error

yesimxev commented 5 years ago

It seems like a non-nethunter issue, maybe kali/fuse bug? Hopefully will be able to build after a few days, will try on Monday

kimocoder commented 5 years ago

"failed to run command '/debootstrap': No such file or directory" .. missing a simple "." in "./debootstrap" over there? I'll do a check tomorrow and we'll see what's up

Re4son commented 5 years ago

Thanks for hanging in there. I'm about to push a commit that will fix all these issues. I'm just doing a final test. It's working well so far and also includes fixes for proxmark & Empire. I'm also installing the latest experimental versions of PowerShell for Empire so that should be a cracker.

Re4son commented 5 years ago

Fixed with the latest commits

yesimxev commented 5 years ago

Fuse error still there, thus not building full fs successfully. Does it build for you? If not, please re-open @Re4son

Re4son commented 5 years ago

I've been building happily for days without any problems. Which arch are you building for?

yesimxev commented 5 years ago

armhf

Re4son commented 5 years ago

Working nicely on my end. I've just build one and it ran through without any problems. Where does yours get stuck?

jcadduono commented 5 years ago

@Re4son contact me sometime if you'd like access to build.nethunter.com which comes with an assortment of random scripts for building & hosting, preferably by discord if you have it (jcadduono#6384) or I may occasionally show up on freenode, or by email

yesimxev commented 5 years ago

Same as before, when setting up fuse. Strange why happens here, cloned the repo again but still the same.

Setting up fuse (2.9.9-1) ...
libkmod: ERROR ../libkmod/libkmod-module.c:1931 kmod_module_get_holders: could not open '/sys/module/vmw_vsock_vmci_transport/holders': No such file or directory
libkmod: ERROR ../libkmod/libkmod-module.c:1931 kmod_module_get_holders: could not open '/sys/module/vsock/holders': No such file or directory
..SNIP...
libkmod: ERROR ../libkmod/libkmod-module.c:1931 kmod_module_get_holders: could not open '/sys/module/scsi_mod/holders': No such file or directory
Unknown device "/dev/fuse": No such device
dpkg: error processing package fuse (--configure):
 installed fuse package post-installation script subprocess returned error exit status 1
Errors were encountered while processing:
 fuse

Does your one end up at ~1.5GB? My build finishes too, but obviously missing the most of the fs (ends up in about 250MB)

Re4son commented 5 years ago

I don't think that message causes your build to bomb out. There must be something else going on behind the scenes. Mine throws the same errors but continues and produces a 1.3GB image; when running "dpkg --configure fuse" in the final chroot it returns "package fuse is already installed and configured".

Are there any other errors later in the build?

yesimxev commented 5 years ago

After a few tries of setting up fuse, it goes over, and wants to do the rest of the stage, but can't, most obvious errors:

Errors were encountered while processing:
 fuse
E: Sub-process /usr/bin/dpkg returned an error code (1)
/third-stage: line 38: make: command not found
./install.sh: line 108: sudo: command not found
./install.sh: line 119: wget: command not found
/usr/bin/python: can't open file 'get-pip.py': [Errno 2] No such file or directory
./install.sh: line 133: lsb_release: command not found
./install.sh: line 134: lsb_release: command not found
./install.sh: line 139: lsb_release: command not found
./install.sh: line 147: lsb_release: command not found
Unknown distro - Debian/Ubuntu Fallback
./install.sh: line 155: sudo: command not found
./install.sh: line 156: pip: command not found
./install.sh: line 157: sudo: command not found
DISTRIB_ID=Kali

Full install log: https://raw.githubusercontent.com/yesimxev/nethunterkalifs/master/log

Re4son commented 5 years ago

Thanks for posting the full log. That output is a bit different to mine. Let me sift through it in detail.

Re4son commented 5 years ago

Thanks for hanging in there. I've added a fix which looks very promising:

Selecting previously unselected package fuse. Preparing to unpack .../099-fuse_2.9.9-1_armhf.deb ... Unpacking fuse (2.9.9-1) ... Selecting previously unselected package libntfs-3g883. Preparing to unpack .../100-libntfs-3g883_1%3a2017.3.23AR.3-2_armhf.deb ... Unpacking libntfs-3g883 (1:2017.3.23AR.3-2) ... Setting up libfuse2:armhf (2.9.9-1) ... Setting up fuse (2.9.9-1) ... update-initramfs: deferring update (trigger activated)

Would you mind trying the branch "2019.2" and letting us know if that works for you?

yesimxev commented 5 years ago

Yup will try soon! Thanks

jcadduono commented 5 years ago

@Re4son I ran a build overnight based on the 2019.2 branch, you can find the output at https://build.nethunter.com/kalifs/kalifs-20190228/ I'm not seeing any major issues myself but only had the chance to glance at the build output so far. The minimal builds are a good bit smaller than usual, but perhaps the base install packages have had some requisites chopped off over the last few years. It's nice to see that all the packages are there for arm64 now, and that arm64 rootfs is now deployable. If you want to build test zips for it, you'll need to revert https://github.com/offensive-security/kali-nethunter/commit/2a1aa7101eafad2ad954144ef8e4cfbdf6fc0fa3 to get the --nightly switch working again.

yesimxev commented 5 years ago

I could build arm64 all the time (before this issue started, with apache php mod corrected). Started to build full armhf now, will let you know if it succeeds (btw libesd0-dev is not removed from depencency check)

yesimxev commented 5 years ago

Finally builds properly. Thanks! There is still a post-install issue with microsoft gpg keys, their server error?

root@kali:~# apt-get update
Get:1 https://packages.microsoft.com/repos/microsoft-debian-stretch-prod stretch InRelease [3232 B]
Hit:2 http://kali.download/kali kali-rolling InRelease
Err:1 https://packages.microsoft.com/repos/microsoft-debian-stretch-prod stretch InRelease
  The following signatures couldn't be verified because the public key is not available: NO_PUBKEY EB3E94ADBE1229CF
Reading package lists... Done
W: GPG error: https://packages.microsoft.com/repos/microsoft-debian-stretch-prod stretch InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY EB3E94ADBE1229CF
E: The repository 'https://packages.microsoft.com/repos/microsoft-debian-stretch-prod stretch InRelease' is not signed.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.

When I try to add, this happens:

root@kali:~# curl -L https://packages.microsoft.com/keys/microsoft.asc | sudot-key add -
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--       0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (60) SSL certificate problem: self signed certificate in certificate chain
More details here: https://curl.haxx.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.
gpg: no valid OpenPGP data found.
kimocoder commented 5 years ago

wget -qO - https://packages.microsoft.com/repos/microsoft-debian-stretch-prod/dists/stretch/Release.gpg | sudo apt-key add -

yesimxev commented 5 years ago

Still gpg: no valid OpenPGP data found.

jcadduono commented 5 years ago

maybe we should have nethunter app initialization of rootfs check for host keys and .gnupg and automatically generate some unique keys :/ add a regenerate keys button in custom commands too maybe since its probably a bad idea to use the same host and gpg keys for everyone using nethunter lol

for curl: (60) SSL certificate problem: self signed certificate in certificate chain uhh is it not seeing ca-certificates? did rootfs install fail for it? what does apt-get install ca-certificates say?

dont know if actually related to lack of ~/.gnupg or something else since haven't tested myself maybe try gpg --full-generate-key --expert -i
9 (ECC and ECC)
4 (NIST P-384)
0 (forever)
y (correct)
Kali Linux NetHunter (real name, as an example)
root@kali (email, as an example)
o (okay)
from here you can either add a password that it will ask every time your key is used for authentication, or leave the next 2 password entries blank and it will never ask you. it should be safe enough and convenient to leave it blank if your device has encryption and/or fingerprint id.

kimocoder commented 5 years ago

I've had problems with "base-passwd" and "ca-certificates" for some time on both the Nexus 6P and OnePlus 6. They simple make apt stall, until I remove them. Somehow, they installed correctly now the last time one the "angler" at last

yesimxev commented 5 years ago

@jcadduono it does not let me to leave the new key passphrase blank :(

yesimxev commented 5 years ago

Actually I totally forgot to enter anything, so the nh terminal was closed. It resulted in a successful blank password entry which was not allowed. Closing the issue, thanks for all the help!

yesimxev commented 5 years ago

Rebooted, and the same issue came back, but after running the gpg generate commands, leaving the password blank succeded, although needed to press ok sure many times. But the issue still persists, re-opening issue.

root@kali:~# apt-get update
Get:1 https://packages.microsoft.com/repos/microsoft-debian-stretch-prod stretch InRelease [3232 B]
Err:1 https://packages.microsoft.com/repos/microsoft-debian-stretch-prod stretch InRelease
  The following signatures couldn't be verified because the public key is not available: NO_PUBKEY EB3E94ADBE1229CF
Hit:2 http://kali.download/kali kali-rolling InRelease
Reading package lists... Done
W: GPG error: https://packages.microsoft.com/repos/microsoft-debian-stretch-prod stretch InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY EB3E94ADBE1229CF
E: The repository 'https://packages.microsoft.com/repos/microsoft-debian-stretch-prod stretch InRelease' is not signed.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.
root@kali:~#
0E800 commented 5 years ago

@yesimxev Apologies if its not my place to be offering support something I know very little on,

but would this be related?

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=867681

In source.lst is [arch=arm64] or[arch=armhf] being used?

Maybe best solution unless I am completely misunderstanding the issue.

wget -qO- https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor > microsoft.asc.gpg
sudo mv microsoft.asc.gpg /etc/apt/trusted.gpg.d/
wget -q https://packages.microsoft.com/config/debian/9/prod.list
sudo mv prod.list /etc/apt/sources.list.d/microsoft-prod.list
sudo chown root:root /etc/apt/trusted.gpg.d/microsoft.asc.gpg
sudo chown root:root /etc/apt/sources.list.d/microsoft-prod.list

Ref: https://dotnet.microsoft.com/download/linux-package-manager/debian9/runtime-2.1.2

Useful info?

Package and Repository Signing Key

Microsoft's GPG public key may be downloaded here: https://packages.microsoft.com/keys/microsoft.asc Public Key ID: Microsoft (Release signing) gpgsecurity@microsoft.com Public Key Fingerprint: BC52 8686 B50D 79E3 39D3 721C EB3E 94AD BE12 29CF

Ref: https://docs.microsoft.com/en-us/windows-server/administration/linux-package-repository-for-microsoft-software

yesimxev commented 5 years ago

It did not work as _apt did not have the permissions to read the microsoft.asc.gpg so I did

rm /etc/apt/trusted.gpg.d/microsoft.asc.gpg
wget -qO- https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor | apt-key add -

Your prod.list might have been needed to. I will try to test out before closing

0E800 commented 5 years ago

@yesimxev I got the same error when I compiled FULL. I did not receive the error from MINIMAL.

Im working with MIN for now to not get nagged, but I will troubleshoot FULL after I smooth out the Rom. Just wanted to verify and confirm the error.

yesimxev commented 5 years ago

@0E800 I see, try 2019.2 branch, that should be fine with the extra commands to add gpg key