add openwrt mipsle build

[ghost]

I want to deploy in router, many thanks!

[koolwiki]

Me too.

[klzgrad]

See if this https://github.com/klzgrad/naiveproxy/releases/tag/v78.0.3904.70-5 works. MIPS has many variants. I don't know if this covers yours.

[koolwiki]

I get this error massage:

/root/home$ ./naive-mipsle -sh: ./naive-mipsle: not found /root/home$ ldd naive-mipsle ldd: can't open cache '/etc/ld.so.cache' ld-uClibc.so.0 => /lib/ld-uClibc.so.0 (0x77789000) libdl.so.0 => /lib/libdl.so.0 (0x77775000) libc.so.0 => /lib/libc.so.0 (0x776f9000) checking sub-depends for '/opt/lib/libatomic.so.1' checking sub-depends for '/opt/lib/libdl.so.2' checking sub-depends for '/lib/libpthread.so.0' checking sub-depends for '/opt/lib/librt.so.1' checking sub-depends for 'not found' checking sub-depends for 'not found' checking sub-depends for 'not found' checking sub-depends for '/opt/lib/libm.so.6' checking sub-depends for '/opt/lib/libgcc_s.so.1' checking sub-depends for '/opt/lib/libc.so.6' libatomic.so.1 => /opt/lib/libatomic.so.1 (0x00000000) libdl.so.2 => /opt/lib/libdl.so.2 (0x00000000) libpthread.so.0 => /lib/libpthread.so.0 (0x00000000) librt.so.1 => /opt/lib/librt.so.1 (0x00000000) libnss3.so => not found (0x00000000) libnssutil3.so => not found (0x00000000) libnspr4.so => not found (0x00000000) libm.so.6 => /opt/lib/libm.so.6 (0x00000000) libgcc_s.so.1 => /opt/lib/libgcc_s.so.1 (0x00000000) libc.so.6 => /opt/lib/libc.so.6 (0x00000000) /lib/ld.so.1 => /lib/ld.so.1 (0x00000000) /lib/ld.so.1 => /lib/ld.so.1 (0x00000000)

[ghost]

mipsel doesn't work either. mipsel and mipsle should be the same thing. I'm using openwrt in router, which doesn't have many shared libraries.

root@OpenWrt:/mnt/sda1# ./naive
-ash: ./naive: not found

root@OpenWrt:/mnt/sda1# ldd naive
    /lib/ld.so.1 (0x77f03000)
Error loading shared library libatomic.so.1: No such file or directory (needed by naive)
    libdl.so.2 => /lib/ld.so.1 (0x77f03000)
    libpthread.so.0 => /lib/ld.so.1 (0x77f03000)
    librt.so.1 => /lib/ld.so.1 (0x77f03000)
Error loading shared library libnss3.so: No such file or directory (needed by naive)
Error loading shared library libnssutil3.so: No such file or directory (needed by naive)
Error loading shared library libnspr4.so: No such file or directory (needed by naive)
    libm.so.6 => /lib/ld.so.1 (0x77f03000)
    libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x77887000)
    libc.so.6 => /lib/ld.so.1 (0x77f03000)
Error loading shared library ld.so.1: No such file or directory (needed by naive)
Error relocating naive: backtrace: symbol not found
Error relocating naive: __strncat_chk: symbol not found
Error relocating naive: PR_IntervalToMilliseconds: symbol not found
Error relocating naive: SECITEM_CompareItem: symbol not found
Error relocating naive: SEC_RegisterDefaultHttpClient: symbol not found
Error relocating naive: PORT_Strdup: symbol not found
Error relocating naive: HASH_HashBuf: symbol not found
Error relocating naive: PK11_FreeSlotListElement: symbol not found
Error relocating naive: __sbrk: symbol not found
Error relocating naive: SECMOD_GetDefaultModuleList: symbol not found
Error relocating naive: __res_nclose: symbol not found
Error relocating naive: CERT_CompareCerts: symbol not found
Error relocating naive: SECITEM_FreeItem: symbol not found
Error relocating naive: PK11_GetFirstSafe: symbol not found
Error relocating naive: CERT_FindCertByDERCert: symbol not found
Error relocating naive: CERT_CreateSubjectCertList: symbol not found
Error relocating naive: PORT_GetError: symbol not found
Error relocating naive: CERT_NewCertList: symbol not found
Error relocating naive: NSS_IsInitialized: symbol not found
Error relocating naive: __res_ninit: symbol not found
Error relocating naive: CERT_DestroyCertificatePoliciesExtension: symbol not found
Error relocating naive: strtoull_l: symbol not found
Error relocating naive: CERT_NewTempCertificate: symbol not found
Error relocating naive: CERT_GetCertTrust: symbol not found
Error relocating naive: PK11_FreeSlotList: symbol not found
Error relocating naive: strtoll_l: symbol not found
Error relocating naive: CERT_DestroyCertList: symbol not found
Error relocating naive: __vsnprintf_chk: symbol not found
Error relocating naive: PK11_IsRemovable: symbol not found
Error relocating naive: PK11_HasRootCerts: symbol not found
Error relocating naive: SEC_StringToOID: symbol not found
Error relocating naive: CERT_DestroyCertificate: symbol not found
Error relocating naive: CERT_RegisterAlternateOCSPAIAInfoCallBack: symbol not found
Error relocating naive: CERT_GetDefaultCertDB: symbol not found
Error relocating naive: SECOID_AddEntry: symbol not found
Error relocating naive: PK11_FreeSlot: symbol not found
Error relocating naive: SECITEM_ItemsAreEqual: symbol not found
Error relocating naive: CERT_PKIXVerifyCert: symbol not found
Error relocating naive: CERT_FindCertExtension: symbol not found
Error relocating naive: CERT_DecodeCertificatePoliciesExtension: symbol not found
Error relocating naive: CERT_DupCertificate: symbol not found
Error relocating naive: PR_Now: symbol not found
Error relocating naive: CERT_CheckCertValidTimes: symbol not found
Error relocating naive: CERT_AddCertToListTail: symbol not found
Error relocating naive: PORT_SetError: symbol not found
Error relocating naive: SECOID_FindOIDTag: symbol not found
Error relocating naive: PK11_GetAllSlotsForCert: symbol not found
Error relocating naive: PK11_IsInternal: symbol not found
Error relocating naive: PK11_GetNextSafe: symbol not found
Error relocating naive: PK11_GetSlotName: symbol not found
Error relocating naive: PK11_FindCertInSlot: symbol not found
Error relocating naive: PK11_IsPresent: symbol not found
Error relocating naive: __register_atfork: symbol not found
Error relocating naive: PR_GetErrorTextLength: symbol not found
Error relocating naive: NSS_SetAlgorithmPolicy: symbol not found
Error relocating naive: SECMOD_GetReadLock: symbol not found
Error relocating naive: __atomic_fetch_add_8: symbol not found
Error relocating naive: PK11_GetInternalKeySlot: symbol not found
Error relocating naive: SECMOD_DestroyModule: symbol not found
Error relocating naive: SECMOD_LoadUserModule: symbol not found
Error relocating naive: PR_GetErrorText: symbol not found
Error relocating naive: NSS_NoDB_Init: symbol not found
Error relocating naive: __isnan: symbol not found
Error relocating naive: PK11_InitPin: symbol not found
Error relocating naive: PK11_GetTokenName: symbol not found
Error relocating naive: SECMOD_GetDefaultModuleListLock: symbol not found
Error relocating naive: NSS_VersionCheck: symbol not found
Error relocating naive: PR_GetError: symbol not found
Error relocating naive: PK11_NeedUserInit: symbol not found
Error relocating naive: PR_Init: symbol not found
Error relocating naive: NSS_InitReadWrite: symbol not found
Error relocating naive: PR_GetOSError: symbol not found
Error relocating naive: PK11_SetPasswordFunc: symbol not found
Error relocating naive: SECMOD_ReleaseReadLock: symbol not found
Error relocating naive: PORT_Strdup: symbol not found
Error relocating naive: __vsnprintf_chk: symbol not found
Error relocating naive: PK11_FreeSlot: symbol not found
Error relocating naive: __register_atfork: symbol not found
Error relocating naive: PR_GetErrorTextLength: symbol not found
Error relocating naive: NSS_SetAlgorithmPolicy: symbol not found
Error relocating naive: SECMOD_GetReadLock: symbol not found
Error relocating naive: __atomic_fetch_add_8: symbol not found
Error relocating naive: PK11_GetInternalKeySlot: symbol not found
Error relocating naive: SECMOD_DestroyModule: symbol not found
Error relocating naive: SECMOD_LoadUserModule: symbol not found
Error relocating naive: PR_GetErrorText: symbol not found
Error relocating naive: NSS_NoDB_Init: symbol not found
Error relocating naive: __isnan: symbol not found
Error relocating naive: PK11_InitPin: symbol not found
Error relocating naive: PK11_GetTokenName: symbol not found
Error relocating naive: SECMOD_GetDefaultModuleListLock: symbol not found
Error relocating naive: NSS_VersionCheck: symbol not found
Error relocating naive: PR_GetError: symbol not found
Error relocating naive: PK11_NeedUserInit: symbol not found
Error relocating naive: PR_Init: symbol not found
Error relocating naive: NSS_InitReadWrite: symbol not found
Error relocating naive: PR_GetOSError: symbol not found
Error relocating naive: PK11_SetPasswordFunc: symbol not found
Error relocating naive: SECMOD_ReleaseReadLock: symbol not found

I also want to compress the binary using npx so that I can fit it in router. When I try that, I got

$ upx -k --best --lzma naive
                       Ultimate Packer for eXecutables
                          Copyright (C) 1996 - 2018
UPX 3.95        Markus Oberhumer, Laszlo Molnar & John Reiser   Aug 26th 2018

        File size         Ratio      Format      Name
   --------------------   ------   -----------   -----------
upx: naive: CantPackException: DT_TEXTREL found; re-compile with -fPIC

Packed 0 files.

Could this be fixed as well?

[klzgrad]

I didn't know your distro is openwrt. You have to specify that, otherwise I can only build for Debian's mips port.

@koolwiki And what is your distro? uClibc is used by openwrt?

[koolwiki]

I didn't know your distro is openwrt. You have to specify that, otherwise I can only build for Debian's mips port.

@koolwiki And what is your distro? uClibc is used by openwrt?

My distro is padavan which maybe base on openwrt. Yes, uClibc is used by openwrt.

[klzgrad]

https://bitbucket.org/padavan/rt-n56u/wiki/EN/HowToMakeFirmware

I don't feel like doing it today.

[klzgrad]

@forever8938 Which target do you require? From https://openwrt.org/docs/techref/targets/start

There are 4 mipsel architectures mipsel_24kc, mipsel_74kc, mipsel_mips32, mipsel_mips32r2. It's not economic to build them all.

[ghost]

@klzgrad mipsel_24kc, thank you!

[klzgrad]

@forever8938 Can you test this binary? naive.zip

Need opkg install libnss libatomic1 first.

[ghost]

@klzgrad After opkg install libnss libatomic1 it does start, but when I send request to it, I got

[1201/110848.464314:INFO:naive_proxy_bin.cc(139)] Proxying via https://mydomain.com
[1201/110848.470067:INFO:naive_proxy_bin.cc(519)] Listening on 127.0.0.1:1082
[1201/111624.475437:INFO:naive_connection.cc(237)] Connection 1 to bolt.dropbox.com:443
[1201/111625.287500:INFO:naive_connection.cc(237)] Connection 2 to bolt.dropbox.com:443
[1201/111626.031559:ERROR:nss_util.cc(750)] After loading Root Certs, loaded==false: Error loading shared library libnssckbi.so: No such file or directory
[1201/111626.051094:ERROR:nss_ocsp.cc(584)] No URLRequestContext for NSS HTTP handler. host: apps.identrust.com
[1201/111626.052366:ERROR:nss_ocsp.cc(584)] No URLRequestContext for NSS HTTP handler. host: cert.int-x3.letsencrypt.org
[1201/111626.053921:ERROR:cert_verify_proc_nss.cc(1011)] CERT_PKIXVerifyCert for mydomain.com failed err=-8179
[1201/111626.063001:ERROR:ssl_client_socket_impl.cc(969)] handshake failed; returned -1, SSL error code 1, net_error -202
[1201/111626.066738:ERROR:ssl_client_socket_impl.cc(969)] handshake failed; returned -1, SSL error code 1, net_error -202
[1201/111626.068403:INFO:naive_proxy.cc(164)] Connection 2 closed: ERR_PROXY_CERTIFICATE_INVALID
[1201/111626.069104:INFO:naive_proxy.cc(164)] Connection 1 closed: ERR_PROXY_CERTIFICATE_INVALID

Here are libraries I got after installing libnss

root@OpenWrt:~# find / -name libnss* 
/lib/upgrade/keep.d/libnss
/overlay/upper/usr/lib/libnssutil3.so
/overlay/upper/usr/lib/libnss3.so
/overlay/upper/usr/lib/opkg/info/libnss.postinst
/overlay/upper/usr/lib/opkg/info/libnss.prerm
/overlay/upper/usr/lib/opkg/info/libnss.conffiles
/overlay/upper/usr/lib/opkg/info/libnss.list
/overlay/upper/usr/lib/opkg/info/libnss.control
/overlay/upper/lib/upgrade/keep.d/libnss
/usr/lib/opkg/info/libnss.postinst
/usr/lib/opkg/info/libnss.prerm
/usr/lib/opkg/info/libnss.conffiles
/usr/lib/opkg/info/libnss.list
/usr/lib/opkg/info/libnss.control
/usr/lib/libnssutil3.so
/usr/lib/libnss3.so

BTW I still cannot compress it using upx, which I usually use to compress v2ray. Is there any technical issue around here?

root@ubuntu:~/share# upx -k --best --lzma naive
                       Ultimate Packer for eXecutables
                          Copyright (C) 1996 - 2017
UPX 3.94        Markus Oberhumer, Laszlo Molnar & John Reiser   May 12th 2017

        File size         Ratio      Format      Name
   --------------------   ------   -----------   -----------
upx: naive: UnknownExecutableFormatException

Packed 0 files.

[klzgrad]

OpenWrt's libnss doesn't provide libnssckbi.so. I'll send reports upstream.

I have no problem with upx.

$ upx naive
                       Ultimate Packer for eXecutables
                          Copyright (C) 1996 - 2018
UPX 3.95        Markus Oberhumer, Laszlo Molnar & John Reiser   Aug 26th 2018

        File size         Ratio      Format      Name
   --------------------   ------   -----------   -----------
naive  5/7  [****************************************************]   18.4%  \    8999084 ->   3157564   35.09%  linux/mipsel   naive

Packed 1 file.

[ghost]

@klzgrad This is not about openwrt but it seems that libnssckbi.so is obsolete in some systems, and p11-kit-trust is an alternative. https://bugzilla.redhat.com/show_bug.cgi?id=1484449 https://p11-glue.github.io/p11-glue/trust-module.html

I tried to opkg install p11-kit and symlink libp11-kit.so to libnssckbi.so, I still got error.

[1201/131355.875420:INFO:naive_proxy_bin.cc(139)] Proxying via https://mydomain.com
[1201/131355.881164:INFO:naive_proxy_bin.cc(519)] Listening on 127.0.0.1:1082
[1201/131358.739136:INFO:naive_connection.cc(237)] Connection 1 to forum.openwrt.org:443
[1201/131358.877230:INFO:naive_connection.cc(237)] Connection 2 to client.dropbox.com:443
[1201/131359.023453:ERROR:nss_ocsp.cc(584)] No URLRequestContext for NSS HTTP handler. host: apps.identrust.com
[1201/131359.024714:ERROR:nss_ocsp.cc(584)] No URLRequestContext for NSS HTTP handler. host: cert.int-x3.letsencrypt.org
[1201/131359.026632:ERROR:cert_verify_proc_nss.cc(1011)] CERT_PKIXVerifyCert for mydomain.com failed err=-8179
[1201/131359.036057:ERROR:ssl_client_socket_impl.cc(969)] handshake failed; returned -1, SSL error code 1, net_error -202
[1201/131359.039943:INFO:naive_proxy.cc(164)] Connection 1 closed: ERR_PROXY_CERTIFICATE_INVALID
[1201/131359.088730:ERROR:ssl_client_socket_impl.cc(969)] handshake failed; returned -1, SSL error code 1, net_error -202
[1201/131359.090692:INFO:naive_proxy.cc(164)] Connection 2 closed: ERR_PROXY_CERTIFICATE_INVALID
[1201/131407.331613:INFO:naive_connection.cc(237)] Connection 3 to mtalk.google.com:443
[1201/131407.545494:ERROR:ssl_client_socket_impl.cc(969)] handshake failed; returned -1, SSL error code 1, net_error -202

Is this a different issue than "libnssckbi.so is missing"?

[klzgrad]

p11-kit is a Fedora/RHEL specific hack to hijack libnssckbi.so so they can install corporate spyware root CAs, and OpenWrt's p11-kit doesn't provide the libraries necessary to do the hijacking either, missing these libraries: https://packages.debian.org/bullseye/amd64/p11-kit-modules/filelist. You're supposed to symlink p11-kit-proxy.so or p11-kit-trust.so, according to your link.

[klzgrad]

In the mean time you can build libnssckbi.so with this

curl https://downloads.openwrt.org/releases/19.07.0-rc1/targets/ramips/rt305x/openwrt-sdk-19.07.0-rc1-ramips-rt305x_gcc-7.4.0_musl.Linux-x86_64.tar.xz | tar xJf -
cd openwrt-sdk-19.07.0-rc1-ramips-rt305x_gcc-7.4.0_musl.Linux-x86_64
./scripts/feeds update base packages
./scripts/feeds install libnss
make defconfig
for flag in ALL_NONSHARED ALL_KMODS ALL SIGNED_PACKAGES; do
  sed -i "s/CONFIG_$flag=y/# CONFIG_$flag is not set/" .config
done
make oldconfig
make -j4

Then you can fish out libnssckbi.so for mipsel_24kc from inside staging_dir.

[ghost]

In the mean time you can build libnssckbi.so with this

curl https://downloads.openwrt.org/releases/19.07.0-rc1/targets/ramips/rt305x/openwrt-sdk-19.07.0-rc1-ramips-rt305x_gcc-7.4.0_musl.Linux-x86_64.tar.xz | tar xJf -
cd openwrt-sdk-19.07.0-rc1-ramips-rt305x_gcc-7.4.0_musl.Linux-x86_64
./scripts/feeds update base packages
./scripts/feeds install libnss
make defconfig
for flag in ALL_NONSHARED ALL_KMODS ALL SIGNED_PACKAGES; do
  sed -i "s/CONFIG_$flag=y/# CONFIG_$flag is not set/" .config
done
make oldconfig
make -j4

Then you can fish out libnssckbi.so for mipsel_24kc from inside staging_dir.

Works like a charm, thanks! Here is libnssckbi.zip for mipsel_24kc.

JFYI, I found upx-compressed binary didn't work.

C:\upx-3.95-win64>upx.exe naive
                       Ultimate Packer for eXecutables
                          Copyright (C) 1996 - 2018
UPX 3.95w       Markus Oberhumer, Laszlo Molnar & John Reiser   Aug 26th 2018

        File size         Ratio      Format      Name
   --------------------   ------   -----------   -----------
Compressing naive [linux/mipsel, NRV2E/7]
   8999084 ->   2874940   31.95%  linux/mipsel   naive***********]   17.3%  -

Packed 1 file.

root@OpenWrt:/mnt/sda1# ./naive naive.json 
Illegal instruction

root@ubuntu:~/share# file naive
naive: ERROR: ELF 32-bit LSB shared object, MIPS, MIPS32 rel2 version 1 (SYSV), dynamically linked error reading (Invalid argument)

[klzgrad]

It's probably some problem in lld. I changed the linker from lld to ld for mipsel build. See https://github.com/klzgrad/naiveproxy/releases/tag/v78.0.3904.70-6.

[klzgrad]

Can you provide some performance numbers, like CPU usage and throughput? I'm curious because naiveproxy is not the most lightweight proxy and I haven't run it on an embedded box.

[ghost]

@klzgrad upx-compressed binary of your new release works, thanks. I roughly tested the performance in my router newifi d2, hope it helps. I'm using naiveproxy as a socks outbound of v2ray.

It consumes 6% of VSZ, 5-25% of CPU, as follows.

Youtube 1080P connection speed is upto around 22000kbps.

Speed test result from a test site is between 16-22Mbps.

In general I feel it's faster than ws over TLS of v2ray on embedded systems.

[klzgrad]

Any reason why you put v2ray in front of naive? I don't see the rest of your v2ray configuration, but in principle v2ray doesn't provide additional utility than naive already does, except for protocol complexity.

[ghost]

V2ray is more of a platform of proxy tools. It can sniff domains from HTTP and TLS traffic, which is an easy way to solve DNS poisoning. It also has a unique routing module that can dispatch traffic to different outbounds based on configuration. I don't need to touch dnsmasq at all. So v2ray overwrites the destination with domain name, and use the domain name to decide if the traffic should be proxyed.

[klzgrad]

I decide to not support padavan because I also need to build lines and various dependencies for it with little user base to benefit.

Opwnwrt upstream has added libnssckbi.so, thus this issue is concluded.

[kousyougi]

我回報一下，naiveproxy-v83.0.4103.61-1-openwrt-mipsel_24kc 在newifi mini (MT7620 )上面可以很順利執行。 因為newifi mini閃存只有16MB，原本以為裝不上去。用UPX壓縮， upx -k --best --lzma naive 壓縮完變1.9MB，裝得上去。