search

Zxilly / UA2F

Change User-Agent to F-string on OpenWRT router to prevent being detected.
https://learningman.top/archives/304
GNU General Public License v3.0
419 stars 85 forks source link

在 GL-ST1200 路由器上做了调整构建后，构建产物运行出现问题求助 #111

Closed Steve-xmh closed 11 months ago

Steve-xmh commented 11 months ago

（自己第一次折腾这个东西，所以会尽可能描述我的构建过程和发生的问题，如果需要什么信息我会尽力提供，还请大佬不吝赐教！）

自己有一个 GL-ST1200 的路由器，因为这个架构比较另类(mips_siflower)，然后想使用大佬的工具处理一下 HTTP 流量，因此尝试了从源码构建：

构建过程

使用的仓库对应的 Commit 是 https://github.com/Zxilly/UA2F/tree/6be399f59e15549d883701a40916a1875cfcbabe

构建的时候我使用了这个路由器开源仓库里提供的一个 SDK 套件 https://github.com/gl-inet/sdk ，通过里面描述的方法一步步进行了构建

起初是遇到了 CMake 版本不足，应该是 SDK 自己准备的 CMake 版本太低，然后我将此处的最低版本修改成了 3.12 以匹配 SDK 的 CMake 版本。 https://github.com/Zxilly/UA2F/blob/6be399f59e15549d883701a40916a1875cfcbabe/CMakeLists.txt#L1

第二次报告未知 CMAKE_C_STANDARD 值（应该也是 CMake 太旧的原因），我就又改动了 CMakeFile.txt:4 到 11。 https://github.com/Zxilly/UA2F/blob/6be399f59e15549d883701a40916a1875cfcbabe/CMakeLists.txt#L4

第三次构建在链接阶段时出现符号未定义的问题：

[ 11%] Linking C executable ua2f
/home/stevexmh/programs/sdk/sdk/1806/siflower/staging_dir/host/bin/cmake -E cmake_link_script CMakeFiles/ua2f.dir/link.txt --verbose=1
/home/stevexmh/programs/sdk/sdk/1806/siflower/staging_dir/toolchain-mipsel_mips-interAptiv_gcc-7.4.0_musl/bin/mipsel-openwrt-linux-musl-gcc -Os -pipe -mno-branch-likely -fno-caller-saves -fno-plt -fhonour-copts -Wno-error=unused-but-set-variable -Wno-error=unused-result -iremap/home/stevexmh/programs/sdk/UA2F/openwrt/..:.. -Wformat -Werror=format-security -fstack-protector -D_FORTIFY_SOURCE=1 -Wl,-z,now -Wl,-z,relro -ffunction-sections -fdata-sections -DNDEBUG  -L/home/stevexmh/programs/sdk/sdk/1806/siflower/staging_dir/target-mipsel_mips-interAptiv_musl/usr/lib -L/home/stevexmh/programs/sdk/sdk/1806/siflower/staging_dir/target-mipsel_mips-interAptiv_musl/lib -L/home/stevexmh/programs/sdk/sdk/1806/siflower/staging_dir/toolchain-mipsel_mips-interAptiv_gcc-7.4.0_musl/usr/lib -L/home/stevexmh/programs/sdk/sdk/1806/siflower/staging_dir/toolchain-mipsel_mips-interAptiv_gcc-7.4.0_musl/lib -znow -zrelro -Wl,-gc-sections CMakeFiles/ua2f.dir/src/ua2f.c.o CMakeFiles/ua2f.dir/src/statistics.c.o CMakeFiles/ua2f.dir/src/util.c.o CMakeFiles/ua2f.dir/src/cache.c.o CMakeFiles/ua2f.dir/src/handler.c.o CMakeFiles/ua2f.dir/src/cli.c.o CMakeFiles/ua2f.dir/src/config.c.o CMakeFiles/ua2f.dir/src/third/nfqueue-mnl.c.o  -o ua2f -lmnl -lnetfilter_queue -lpthread -lnfnetlink -luci
/home/stevexmh/programs/sdk/sdk/1806/siflower/staging_dir/toolchain-mipsel_mips-interAptiv_gcc-7.4.0_musl/bin/../lib/gcc/mipsel-openwrt-linux-musl/7.4.0/../../../../mipsel-openwrt-linux-musl/bin/ld: CMakeFiles/ua2f.dir/src/handler.c.o: in function `handle_packet':
handler.c:(.text.handle_packet+0x3d0): undefined reference to `nfq_tcp_mangle_ipv6'
collect2: error: ld returned 1 exit status
make[6]: *** [CMakeFiles/ua2f.dir/build.make:189: ua2f] Error 1
make[6]: Leaving directory '/home/stevexmh/programs/sdk/UA2F/build'
make[5]: *** [CMakeFiles/Makefile2:73: CMakeFiles/ua2f.dir/all] Error 2
make[5]: Leaving directory '/home/stevexmh/programs/sdk/UA2F/build'
make[4]: *** [Makefile:130: all] Error 2
make[4]: Leaving directory '/home/stevexmh/programs/sdk/UA2F/build'
make[3]: *** [Makefile:83: /home/stevexmh/programs/sdk/UA2F/openwrt/../.built] Error 2
make[3]: Leaving directory '/home/stevexmh/programs/sdk/UA2F/openwrt'
time: package/UA2F/openwrt/compile#0.10#0.02#0.17
make[2]: *** [package/Makefile:122: package/UA2F/openwrt/compile] Error 2
make[2]: Leaving directory '/home/stevexmh/programs/sdk/sdk/1806/siflower'
make[1]: *** [package/Makefile:116: /home/stevexmh/programs/sdk/sdk/1806/siflower/staging_dir/target-mipsel_mips-interAptiv_musl/stamp/.package_compile] Error 2
make[1]: Leaving directory '/home/stevexmh/programs/sdk/sdk/1806/siflower'
make: *** [/home/stevexmh/programs/sdk/sdk/1806/siflower/include/toplevel.mk:211: world] Error 2
make: Leaving directory '/home/stevexmh/programs/sdk/sdk/1806/siflower'

根据错误输出我找到了这个函数在 UA2F 的位置： https://github.com/Zxilly/UA2F/blob/6be399f59e15549d883701a40916a1875cfcbabe/src/handler.c#L305

我已经确认了这个函数对应的库 netfilters-queue 已经安装在 SDK 中了，但是好像无法通过 grep 搜索到这个函数的任何定义：

stevexmh@SteveComputer:~/programs/sdk$ grep -rl "nfq_tcp_mangle_ipv6"
UA2F/src/handler.c
UA2F/build/CMakeFiles/ua2f.dir/src/handler.c.o

作为对比我尝试搜索了 nfq_tcp_mangle_ipv4：

stevexmh@SteveComputer:~/programs/sdk$ grep -rl "nfq_tcp_mangle_ipv4"
UA2F/src/handler.c
UA2F/build/CMakeFiles/ua2f.dir/src/handler.c.o
sdk/1806/siflower/build_dir/target-mipsel_mips-interAptiv_musl/UA2F-3.10/ua2f.c
sdk/1806/siflower/build_dir/target-mipsel_mips-interAptiv_musl/libnetfilter_queue-2017-06-27-601abd1c/.pkgdir/libnetfilter-queue/usr/lib/libnetfilter_queue.so.1.4.0
sdk/1806/siflower/staging_dir/target-mipsel_mips-interAptiv_musl/root-siflower/usr/bin/ua2f
sdk/1806/siflower/staging_dir/target-mipsel_mips-interAptiv_musl/root-siflower/usr/lib/libnetfilter_queue.so.1.4.0
sdk/1806/siflower/staging_dir/target-mipsel_mips-interAptiv_musl/usr/lib/libnetfilter_queue.a
sdk/1806/siflower/staging_dir/target-mipsel_mips-interAptiv_musl/usr/lib/libnetfilter_queue.so.1.4.0
sdk/1806/siflower/staging_dir/target-mipsel_mips-interAptiv_musl/usr/include/libnetfilter_queue/libnetfilter_queue_tcp.h

然后这个是搜索出 nfq_tcp_mangle_ipv4 的那个头文件(sdk/1806/siflower/staging_dir/target-mipsel_mips-interAptiv_musl/usr/include/libnetfilter_queue/libnetfilter_queue_tcp.h)的内容：

#ifndef _LIBNFQUEUE_TCP_H_
#define _LIBNFQUEUE_TCP_H_

struct pkt_buff;

struct tcphdr *nfq_tcp_get_hdr(struct pkt_buff *pktb);
void *nfq_tcp_get_payload(struct tcphdr *tcph, struct pkt_buff *pktb);
unsigned int nfq_tcp_get_payload_len(struct tcphdr *tcph, struct pkt_buff *pktb);

struct iphdr;
struct ip6_hdr;

void nfq_tcp_compute_checksum_ipv4(struct tcphdr *tcph, struct iphdr *iph);
void nfq_tcp_compute_checksum_ipv6(struct tcphdr *tcph, struct ip6_hdr *ip6h);

int nfq_tcp_mangle_ipv4(struct pkt_buff *pkt, unsigned int match_offset, unsigned int match_len, const char *rep_buffer, unsigned int rep_len);

int nfq_tcp_snprintf(char *buf, size_t size, const struct tcphdr *tcp);

#endif

想了一下自己的环境并没有 IPv6，所以我就斗胆注释掉了这行调用了 nfq_tcp_mangle_ipv6 的函数

实际运行

最后顺利编译出来一个 ipk 产物。我就放入路由器中安装并尝试启动 UA2F，下面是我的 UCI 配置：

root@GL-SFT1200:~# uci show ua2f
ua2f.enabled=ua2f
ua2f.enabled.enabled='1'
ua2f.firewall=ua2f
ua2f.firewall.handle_tls='0'
ua2f.firewall.handle_intranet='1'
ua2f.firewall.handle_fw='1'
ua2f.main=ua2f
ua2f.main.custom_ua='Test App XD/1.0'

随后使用以下指令启动了 UA2F：

service ua2f start

但是启动之后在我的其他机器上使用 CURL 测试请求 http://httpbin.org/headers 的时候却超时了，但 HTTPS 流量还是正常的：

stevexmh@SteveComputer:~$ curl -v --connect-timeout 10 http://httpbin.org/headers
*   Trying 54.83.155.149:80...
* After 4989ms connect time, move on!
* connect to 54.83.155.149 port 80 failed: Connection timed out
*   Trying 75.101.131.185:80...
* After 2493ms connect time, move on!
* connect to 75.101.131.185 port 80 failed: Connection timed out
* Failed to connect to httpbin.org port 80 after 7507 ms: Connection timed out
* Closing connection 0
curl: (28) Failed to connect to httpbin.org port 80 after 7507 ms: Connection timed out
stevexmh@SteveComputer:~$ curl -v --connect-timeout 10 https://httpbin.org/headers
*   Trying 54.83.155.149:443...
* Connected to httpbin.org (54.83.155.149) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
* TLSv1.0 (OUT), TLS header, Certificate Status (22):
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS header, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS header, Certificate Status (22):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS header, Certificate Status (22):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS header, Certificate Status (22):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS header, Finished (20):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS header, Finished (20):
* TLSv1.2 (IN), TLS header, Certificate Status (22):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: CN=httpbin.org
*  start date: Sep 21 00:00:00 2023 GMT
*  expire date: Oct 18 23:59:59 2024 GMT
*  subjectAltName: host "httpbin.org" matched cert's "httpbin.org"
*  issuer: C=US; O=Amazon; CN=Amazon RSA 2048 M02
*  SSL certificate verify ok.
* Using HTTP2, server supports multiplexing
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* Using Stream ID: 1 (easy handle 0x561098b08e90)
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
> GET /headers HTTP/2
> Host: httpbin.org
> user-agent: curl/7.81.0
> accept: */*
>
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* Connection state changed (MAX_CONCURRENT_STREAMS == 128)!
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
< HTTP/2 200
< date: Mon, 04 Dec 2023 13:13:15 GMT
< content-type: application/json
< content-length: 173
< server: gunicorn/19.9.0
< access-control-allow-origin: *
< access-control-allow-credentials: true
<
{
  "headers": {
    "Accept": "*/*",
    "Host": "httpbin.org",
    "User-Agent": "curl/7.81.0",
    "X-Amzn-Trace-Id": "Root=1-656dd06b-2b8b7e5d1466d9f141bc5fc6"
  }
}
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* Connection #0 to host httpbin.org left intact

查阅了 OpenWRT 的系统日志后，发现几行和 UA2F 相关的日志：

Mon Dec  4 20:46:10 2023 syslog.info UA2F[13683]: Statistics initialized.
Mon Dec  4 20:46:10 2023 syslog.info UA2F[13683]: Using config user agent string: Test App XD/1.0
Mon Dec  4 20:46:10 2023 syslog.info UA2F[13683]: Handler initialized.
Mon Dec  4 20:46:10 2023 syslog.err UA2F[13683]: Netlink error: Not supported

想要解决的问题

如上，希望能在 GL-ST1200 这个路由器上正常运行 UA2F 并正确处理 HTTP 流量

非常感谢大佬！

Zxilly commented 11 months ago

找不到符号，是因为你的库版本低了，libnetfilter-queue应该用1.0.5。 1.0.4的时候改了一些东西，我也不确定之前版本的行为和现在的一不一样，也不打算去做这个兼容。

Steve-xmh commented 11 months ago

找不到符号，是因为你的库版本低了，libnetfilter-queue应该用1.0.5。 1.0.4的时候改了一些东西，我也不确定之前版本的行为和现在的一不一样，也不打算去做这个兼容。

那就难办了，这台机器的最新版本也才基于 OpenWRT 18.06，查询了一下所处的 libnetfilter-queue 也是 2017 年的旧版本了，各种工具链都也都比较旧了。 时间和难度上都不太允许我自己继续解决这些兼容性问题，可能只能放弃了，用不了作者大佬的模块太可惜了。总之非常感谢帮助！