Error with ja4tscan and zmap

[sundruid]

Anyone receive this error:

sudo python3 ja4tscan.py -p 80 204.79.197.223 adding iptable rules... zmap: invalid option -- 'I' cleaning up iptable rules...

[sundruid]

After some work on this.. There are quite a few libraries and apps that need installed to compile this. Here is a list that worked for me:

sudo apt-get install pkg-config libjson-c-dev libjudy-dev gengetopt flex byacc libgmp-dev libunistring-dev libpcap-dev

After that, I got a clean compile. But then an error when trying to test run:

sudo python3 ja4tscan.py -p 80 204.79.197.212 adding iptable rules... zmap: unrecognized option '--blacklist-file=/etc/zmap/blacklist.conf' cleaning up iptable rules...

But after grepping through the entire directory recursively, I cannot find '--blacklist-file' mentioned in any code. It appears that this option was changed to '--blocklist-file' and needs to be adjusted in order make it run. To this point, until I can figure out where that command is stored in the code, I can't make this code work.

Any help would be appreciated.

[noeltimothy]

Hi, Can you let us know which version of zmap you are using? We currently support zmap 4.1.0. You can run the build.sh file to checkout the right sources and build it.

[sundruid]

That’s what it uses. The makefile downloads and compiles zmap. You need to try an install. On Jun 20, 2024, at 1:42 PM, Timothy Noel @.***> wrote: Hi, Can you let us know which version of zmap you are using? We currently support zmap 4.1.0. You can run the build.sh file to checkout the right sources and build it.

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: @.***>

[noeltimothy]

Can you let us know what Operating System you are on? Please give us the output of "uname -a"

[IanSmith123]

What's your Python version? This script uses f-strings which are only compatible with Python 3.6 or higher. Older versions of Python might not parse -I {} successfully.

[sundruid]

python 3.12

Linux localhost 6.3.0-kali1-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.3.7-1kali1 (2023-06-29) x86_64 GNU/Linux

Also same errors on macOS

Darwin Kernel Version 22.6.0: Mon Feb 19 19:43:13 PST 2024; root:xnu-8796.141.3.704.6~1/RELEASE_ARM64_T6020 arm64

Be sure to read my errors above. It seems like you're just guessing. There is a problem with 'blacklist' being changed to 'blocklist' in zmap. Also the -I flag doesn't seem to be recognized either. I've uninstalled and reinstalled zmap and it is properly reading zmap (so it seems) in the app directory. It is installed when you run build.

ls build.sh input ja4tscan.py LICENSE module_ja4tscan.c probe_modules.c README.md zmap

[noeltimothy]

Hi @ryancmoon , if the update to probe_module.c has resolved this issue, can we close this out?

[sundruid]

Still getting error:

sudo python3 ja4tscan.py -p 80 my_probe_list.txt adding iptable rules... zmap: unrecognized option '--blacklist-file=/etc/zmap/blacklist.conf' cleaning up iptable rules...

Per my other email, it's looking for 'blocklist' not 'blacklist'

Here are the build.sh errors that come up:

build.sh fetching zmap sources Cloning into 'zmap'... remote: Enumerating objects: 7750, done. remote: Counting objects: 100% (1991/1991), done. remote: Compressing objects: 100% (622/622), done. remote: Total 7750 (delta 1719), reused 1439 (delta 1369), pack-reused 5759 Receiving objects: 100% (7750/7750), 6.38 MiB | 24.75 MiB/s, done. Resolving deltas: 100% (5555/5555), done. Note: switching to 'v4.1.0-RC1'.

You are in 'detached HEAD' state. You can look around, make experimental changes and commit them, and you can discard any commits you make in this state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may do so (now or later) by using -c with the switch command. Example:

git switch -c

Or undo this operation with:

git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at 4d4166e Output the progress once more when the whole scan process finished (#828) HEAD detached at v4.1.0-RC1 nothing to commit, working tree clean building using cmake... CMake Warning: No source or binary directory provided. Both will be assumed to be the same as the current working directory, but note that this warning will become a fatal error in future CMake releases.

CMake Deprecation Warning at CMakeLists.txt:1 (cmake_minimum_required): Compatibility with CMake < 3.5 will be removed from a future version of CMake.

Update the VERSION argument value or use a ... suffix to tell CMake that the project does not need compatibility with older versions.

-- The C compiler identification is GNU 13.2.0 -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Check for working C compiler: /usr/bin/cc - skipped -- Detecting C compile features -- Detecting C compile features - done -- Found PkgConfig: /usr/bin/pkg-config (found version "1.8.1") -- Checking for module 'json-c' -- Found json-c, version 0.17 -- Default ZMap configuration file location is /etc/zmap -- Checking if blocklist.conf exists there... CMake Warning at CMakeLists.txt:154 (message): Existing configuration file detected at /etc/zmap/blocklist.conf, blocklist.conf from sources will NOT be installed. Please check and install manually!

-- Checking if zmap.conf exists there... CMake Warning at CMakeLists.txt:154 (message): Existing configuration file detected at /etc/zmap/zmap.conf, zmap.conf from sources will NOT be installed. Please check and install manually!

-- Configuring done (0.4s) -- Generating done (0.0s) -- Build files have been written to: /root/ja4tscan/zmap [ 1%] Building C object lib/CMakeFiles/zmaplib.dir/blocklist.c.o [ 2%] Building C object lib/CMakeFiles/zmaplib.dir/cachehash.c.o [ 3%] Building C object lib/CMakeFiles/zmaplib.dir/constraint.c.o [ 4%] Building C object lib/CMakeFiles/zmaplib.dir/logger.c.o [ 4%] Building C object lib/CMakeFiles/zmaplib.dir/pbm.c.o [ 5%] Building C object lib/CMakeFiles/zmaplib.dir/random.c.o [ 6%] Building C object lib/CMakeFiles/zmaplib.dir/rijndael-alg-fst.c.o [ 7%] Building C object lib/CMakeFiles/zmaplib.dir/xalloc.c.o [ 8%] Building C object lib/CMakeFiles/zmaplib.dir/lockfd.c.o [ 9%] Building C object lib/CMakeFiles/zmaplib.dir/util.c.o [ 9%] Building C object lib/CMakeFiles/zmaplib.dir/queue.c.o [ 10%] Building C object lib/CMakeFiles/zmaplib.dir/csv.c.o [ 11%] Building C object lib/CMakeFiles/zmaplib.dir/aes128.c.o [ 12%] Linking C static library libzmaplib.a [ 12%] Built target zmaplib [ 12%] Built target git_versioning [ 13%] Generating zopt.h [ 14%] Generating lexer.c [ 14%] Generating parser.c [ 15%] Building C object src/CMakeFiles/zmap.dir/aesrand.c.o [ 16%] Building C object src/CMakeFiles/zmap.dir/cyclic.c.o [ 17%] Building C object src/CMakeFiles/zmap.dir/expression.c.o [ 18%] Building C object src/CMakeFiles/zmap.dir/fieldset.c.o [ 19%] Building C object src/CMakeFiles/zmap.dir/filter.c.o [ 19%] Building C object src/CMakeFiles/zmap.dir/get_gateway.c.o [ 20%] Building C object src/CMakeFiles/zmap.dir/iterator.c.o [ 21%] Building C object src/CMakeFiles/zmap.dir/monitor.c.o /root/ja4tscan/zmap/src/monitor.c: In function ‘export_then_update’: /root/ja4tscan/zmap/src/monitor.c:208:70: warning: ‘%s’ directive output may be truncated writing up to 19 bytes into a region of size 18 [-Wformat-truncation=] 208 | snprintf(exp->time_remaining_str, NUMBER_STR_LEN, " (%s left)", | ^~ 209 | buf); | ~~~ In file included from /usr/include/stdio.h:964, from /root/ja4tscan/zmap/src/monitor.c:20: In function ‘snprintf’, inlined from ‘export_stats’ at /root/ja4tscan/zmap/src/monitor.c:208:3, inlined from ‘export_then_update’ at /root/ja4tscan/zmap/src/monitor.c:470:3: /usr/include/x86_64-linux-gnu/bits/stdio2.h:54:10: note: ‘builtin___snprintf_chk’ output between 9 and 28 bytes into a destination of size 20 54 | return builtin_snprintf_chk (s, n, USE_FORTIFY_LEVEL - 1, | ^~~~~~~~~~~~~~~~ 55 | glibc_objsize (s), fmt, | ~~~~~~~~~ 56 | __va_arg_pack ()); | ~~~~~ [ 22%] Building C object src/CMakeFiles/zmap.dir/ports.c.o [ 23%] Building C object src/CMakeFiles/zmap.dir/recv.c.o [ 23%] Building C object src/CMakeFiles/zmap.dir/send.c.o [ 24%] Building C object src/CMakeFiles/zmap.dir/shard.c.o [ 25%] Building C object src/CMakeFiles/zmap.dir/socket.c.o [ 26%] Building C object src/CMakeFiles/zmap.dir/state.c.o [ 27%] Building C object src/CMakeFiles/zmap.dir/summary.c.o [ 28%] Building C object src/CMakeFiles/zmap.dir/utility.c.o [ 28%] Building C object src/CMakeFiles/zmap.dir/validate.c.o [ 29%] Building C object src/CMakeFiles/zmap.dir/zmap.c.o [ 30%] Building C object src/CMakeFiles/zmap.dir/zopt_compat.c.o [ 31%] Building C object src/CMakeFiles/zmap.dir/lexer.c.o [ 32%] Building C object src/CMakeFiles/zmap.dir/parser.c.o /root/ja4tscan/zmap/src/parser.c:74:23: warning: redundant redeclaration of ‘yylex’ [-Wredundant-decls] 74 | # define YYLEX_DECL() yylex(void) | ^~~~~ /root/ja4tscan/zmap/src/parser.c:79:5: note: in expansion of macro ‘YYLEX_DECL’ 79 | int YYLEX_DECL(); | ^~~~~~ In file included from /root/ja4tscan/zmap/src/parser.y:5: /root/ja4tscan/zmap/src/lexer.h:305:12: note: previous declaration of ‘yylex’ with type ‘int(void)’ 305 | extern int yylex (void); | ^~~~~ [ 33%] Building C object src/CMakeFiles/zmap.dir/probe_modules/module_ja4tscan.c.o /root/ja4tscan/zmap/src/probe_modules/module_ja4tscan.c: In function ‘num_of_digits’: /root/ja4tscan/zmap/src/probe_modules/module_ja4tscan.c:72:31: warning: cast from function call of type ‘double’ to non-matching type ‘int’ [-Wbad-function-cast] 72 | return ( n==0 ) ? 1 : (int) log10(n)+1; | ^ /root/ja4tscan/zmap/src/probe_modules/module_ja4tscan.c: At top level: /root/ja4tscan/zmap/src/probe_modules/module_ja4tscan.c:495:23: warning: initialization of ‘int ()(void , macaddr_t , macaddr_t , void )’ {aka ‘int ()(void , unsigned char , unsigned char , void )’} from incompatible pointer type ‘int ()(void , macaddr_t , macaddr_t , void )’ {aka ‘int ()(void , unsigned char , unsigned char , void )’} [-Wincompatible-pointer-types] 495 | .prepare_packet = &ja4tscan_prepare_packet, | ^ /root/ja4tscan/zmap/src/probe_modules/module_ja4tscan.c:495:23: note: (near initialization for ‘module_ja4tscan.prepare_packet’) /root/ja4tscan/zmap/src/probe_modules/module_ja4tscan.c:496:20: warning: initialization of ‘int ()(void , size_t , ipaddr_n_t, ipaddr_n_t, port_n_t, uint8_t, uint32_t , int, uint16_t, void )’ {aka ‘int ()(void , long unsigned int , unsigned int, unsigned int, short unsigned int, unsigned char, unsigned int , int, short unsigned int, void )’} from incompatible pointer type ‘int ()(void , size_t , ipaddr_n_t, ipaddr_n_t, port_n_t, uint8_t, uint32_t , int, void )’ {aka ‘int ()(void , long unsigned int , unsigned int, unsigned int, short unsigned int, unsigned char, unsigned int , int, void )’} [-Wincompatible-pointer-types] 496 | .make_packet = &ja4tscan_make_packet, | ^ /root/ja4tscan/zmap/src/probe_modules/module_ja4tscan.c:496:20: note: (near initialization for ‘module_ja4tscan.make_packet’) /root/ja4tscan/zmap/src/probe_modules/module_ja4tscan.c: In function ‘ja4tscan_process_packet’: /root/ja4tscan/zmap/src/probe_modules/module_ja4tscan.c:424:25: warning: ‘builtin_strncat’ specified bound depends on the length of the source argument [-Wstringop-overflow=] 424 | strncat(timedata->options, option_kind_str, strlen(option_kind_str)); | ^ /root/ja4tscan/zmap/src/probe_modules/module_ja4tscan.c:424:25: note: length computed here 424 | strncat(timedata->options, option_kind_str, strlen(option_kind_str)); | ^~~~~~~~~~~~~~~~ [ 33%] Building C object src/CMakeFiles/zmap.dir/probe_modules/module_icmp_echo.c.o [ 34%] Building C object src/CMakeFiles/zmap.dir/probe_modules/module_icmp_echo_time.c.o [ 35%] Building C object src/CMakeFiles/zmap.dir/probe_modules/module_tcp_synscan.c.o [ 36%] Building C object src/CMakeFiles/zmap.dir/probe_modules/module_tcp_synackscan.c.o [ 37%] Building C object src/CMakeFiles/zmap.dir/probe_modules/module_udp.c.o [ 38%] Building C object src/CMakeFiles/zmap.dir/probe_modules/module_ipip.c.o [ 38%] Building C object src/CMakeFiles/zmap.dir/probe_modules/packet.c.o [ 39%] Building C object src/CMakeFiles/zmap.dir/probe_modules/probe_modules.c.o /root/ja4tscan/zmap/src/probe_modules/probe_modules.c:7:1: error: expected identifier or ‘(’ before ‘<’ token 7 | <!DOCTYPE html> | ^ /root/ja4tscan/zmap/src/probe_modules/probe_modules.c:40:570: error: expected identifier or ‘(’ before ‘<’ token 40 | arketing_pages_search_explore_provider","react_start_transition_for_navigations","remove_child_patch","sample_network_conn_type","turbo_experiment_risky"]} | ^

/root/ja4tscan/zmap/src/probe_modules/probe_modules.c:255:168: error: expected identifier or ‘(’ before ‘<’ token 255 | pe="application/json" data-target="react-partial.embeddedData">{"props":{"docsUrl":"https://docs.github.com/get-started/accessibility/keyboard-shortcuts"}} | ^

/root/ja4tscan/zmap/src/probe_modules/probe_modules.c:540:1: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘attribute’ before ‘<’ token 540 | | ^ /root/ja4tscan/zmap/src/probe_modules/probe_modules.c:852:16: warning: missing terminating ' character 852 |

| ^ /root/ja4tscan/zmap/src/probe_modules/probe_modules.c:852:16: error: missing terminating ' character 852 |

| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /root/ja4tscan/zmap/src/probe_modules/probe_modules.c:875:280: warning: missing terminating " character 875 | ombobox" spellcheck="false" aria-expanded="false" aria-describedby="validation-edb09768-b3de-4560-8f11-6b5a381c14d4" data-target="query-builder.input" data-action=" | ^

/root/ja4tscan/zmap/src/probe_modules/probe_modules.c:875:280: error: missing terminating " character /root/ja4tscan/zmap/src/probe_modules/probe_modules.c:876:30: error: stray ‘#’ in program 876 | input:query-builder#inputChange | ^ /root/ja4tscan/zmap/src/probe_modules/probe_modules.c:877:29: error: stray ‘#’ in program 877 | blur:query-builder#inputBlur | ^ /root/ja4tscan/zmap/src/probe_modules/probe_modules.c:878:32: error: stray ‘#’ in program 878 | keydown:query-builder#inputKeydown | ^ /root/ja4tscan/zmap/src/probe_modules/probe_modules.c:879:30: error: stray ‘#’ in program 879 | focus:query-builder#inputFocus | ^ /root/ja4tscan/zmap/src/probe_modules/probe_modules.c:880:100: warning: missing terminating " character 880 | " data-view-component="true" class="FormControl-input QueryBuilder-Input FormControl-medium" /> | ^ /root/ja4tscan/zmap/src/probe_modules/probe_modules.c:880:100: error: missing terminating " character 880 | " data-view-component="true" class="FormControl-input QueryBuilder-Input FormControl-medium" /> | ^~~~ /root/ja4tscan/zmap/src/probe_modules/probe_modules.c:884:190: warning: missing terminating " character 884 | tton" id="query-builder-test-clear-button" aria-labelledby="query-builder-test-clear query-builder-test-label" data-target="query-builder.clearButton" data-action=" | ^

/root/ja4tscan/zmap/src/probe_modules/probe_modules.c:884:190: error: missing terminating " character /root/ja4tscan/zmap/src/probe_modules/probe_modules.c:885:36: error: stray ‘#’ in program 885 | click:query-builder#clear | ^ /root/ja4tscan/zmap/src/probe_modules/probe_modules.c:886:36: error: stray ‘#’ in program 886 | focus:query-builder#clearButtonFocus | ^ /root/ja4tscan/zmap/src/probe_modules/probe_modules.c:887:35: error: stray ‘#’ in program 887 | blur:query-builder#clearButtonBlur | ^ /root/ja4tscan/zmap/src/probe_modules/probe_modules.c:888:380: warning: missing terminating " character 888 | ">

/root/ja4tscan/zmap/src/probe_modules/probe_modules.c:888:380: error: missing terminating " character 888 | ">

/root/ja4tscan/zmap/src/probe_modules/probe_modules.c:1103:31: warning: missing terminating " character 1103	data-action="	^ /root/ja4tscan/zmap/src/probe_modules/probe_modules.c:1103:31: error: missing terminating " character /root/ja4tscan/zmap/src/probe_modules/probe_modules.c:1104:50: error: stray ‘#’ in program 1104	combobox-commit:query-builder#comboboxCommit	^ /root/ja4tscan/zmap/src/probe_modules/probe_modules.c:1105:44: error: stray ‘#’ in program 1105	mousedown:query-builder#resultsMousedown	^ /root/ja4tscan/zmap/src/probe_modules/probe_modules.c:1106:19: warning: missing terminating " character 1106	"	^ /root/ja4tscan/zmap/src/probe_modules/probe_modules.c:1106:19: error: missing terminating " character /root/ja4tscan/zmap/src/probe_modules/probe_modules.c:1155:75: warning: missing terminating ' character 1155		^ /root/ja4tscan/zmap/src/probe_modules/probe_modules.c:1155:75: error: missing terminating ' character 1155		^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /root/ja4tscan/zmap/src/probe_modules/probe_modules.c:1193:14: warning: missing terminating ' character 1193		^ /root/ja4tscan/zmap/src/probe_modules/probe_modules.c:1193:14: error: missing terminating ' character 1193		^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /root/ja4tscan/zmap/src/probe_modules/probe_modules.c:1320:25: error: expected identifier or ‘(’ before ‘<’ token 1320	{{ message }}	^ /root/ja4tscan/zmap/src/probe_modules/probe_modules.c:1673: error: expected identifier or ‘(’ before ‘<’ token 1673

/root/ja4tscan/zmap/src/probe_modules/probe_modules.c:1690:13: warning: character constant too long for its type 1690 |

Footer

| ^~~~~ /root/ja4tscan/zmap/src/probe_modules/probe_modules.c:1703:16: error: expected identifier or ‘(’ before numeric constant 1703 | © 2024 GitHub, Inc. | ^~~~ /root/ja4tscan/zmap/src/probe_modules/probe_modules.c:1703:37: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘attribute’ before ‘.’ token 1703 | © 2024 GitHub, Inc. | ^ /root/ja4tscan/zmap/src/probe_modules/probe_modules.c:1772:12: error: stray ‘\342’ in program 1772 | You can<U+2019>t perform that action at this time. | ^~~~ make[2]: [src/CMakeFiles/zmap.dir/build.make:494: src/CMakeFiles/zmap.dir/probe_modules/probe_modules.c.o] Error 1 make[1]: [CMakeFiles/Makefile2:205: src/CMakeFiles/zmap.dir/all] Error 2 make: *** [Makefile:156: all] Error 2 You can now run python3 ja4tscan.py