search

dreadl0ck / ja3

Go package for Ja3 TLS client and server hello fingerprints
BSD 3-Clause "New" or "Revised" License
142 stars 21 forks source link

runtime error: slice bounds out of range #6

Open snappyJack opened 3 years ago

snappyJack commented 3 years ago

when I run this /root/go/bin/goja3 -ja3s=false -json -iface eth1

the result 

[root@localhost ja3]# /root/go/bin/goja3 -ja3s=false -json -iface eth1
timestamp,source_ip,source_port,destination_ip,destination_port,ja3_digest
1600315798.710436,128.59.66.11,443,10.246.120.106,61720,dd4b012f7a008e741554bd0a4ed12920
1600315798.710710,61.129.7.21,443,10.246.52.102,62627,389ed42c02ebecc32e73aa31def07e14
1600315798.713353,61.155.222.97,443,10.246.121.186,53185,3653a20186a5b490426131a611e01992
1600315798.717188,61.155.222.97,443,10.246.121.186,53186,3653a20186a5b490426131a611e01992
1600315798.718566,10.246.92.133,2301,180.101.212.39,443,3b5074b1b5d032e5620f69f9f700ff0e
1600315798.732672,180.101.212.39,443,10.246.120.138,53336,e9a3ced07403903421d84ae7e9d92be5
1600315798.735224,10.246.142.18,41458,172.217.160.110,443,66918128f1b9b03303d77c6f2eefd128
1600315798.735948,10.246.142.18,41460,172.217.160.110,443,66918128f1b9b03303d77c6f2eefd128
1600315798.738636,10.246.152.123,43522,118.193.98.74,443,9b02ebd3a43b62d825e1ac605b621dc8
1600315798.739077,10.246.104.50,2608,180.163.32.196,443,44d502d471cfdb99c59bdfb0f220e5a8
1600315798.741837,192.168.104.159,61602,116.211.20.187,8514,455bd65d382d4741f0e48654f27cbe80
1600315798.742983,10.246.40.103,49517,117.18.232.200,443,3b5074b1b5d032e5620f69f9f700ff0e
1600315798.747841,119.23.53.168,443,10.246.92.224,61554,4ef1b297bb817d8212165a86308bac5f
1600315798.749056,10.246.152.180,38806,157.255.245.29,443,c4d9c7fc50e1fd2ebf09e3d9a2a31b33
1600315798.749270,119.23.53.168,443,10.246.92.224,61555,4ef1b297bb817d8212165a86308bac5f
1600315798.752894,139.227.253.126,6690,10.246.34.52,6672,20e0bd1c4d36e08a1f31656fb48b99a0
1600315798.756333,10.246.114.99,54261,172.217.160.74,443,b32309a26951912be7dba376398abc3b
1600315798.756440,61.174.240.228,443,10.246.138.147,39796,4cf820cab8f5a2bf61be14f5493233ae
panic: runtime error: slice bounds out of range [:15] with capacity 12

goroutine 1 [running]:
github.com/dreadl0ck/tlsx.(*ServerHelloBasic).Unmarshal(0xc00015d830, 0xc000356c6a, 0xc, 0xc, 0x4c984a, 0x0)
    /root/go/pkg/mod/github.com/dreadl0ck/tlsx@v0.0.0-20200303221230-8cb859306e07/serverHello.go:270 +0x7d5
github.com/dreadl0ck/ja3.BarePacketJa3s(0x63b980, 0xc00037cdc0, 0xee0e38, 0x0, 0x0)
    /home/morty/ja3/gopacket.go:119 +0x110
github.com/dreadl0ck/ja3.DigestHexPacketJa3s(0x63b980, 0xc00037cdc0, 0x0, 0x0)
    /home/morty/ja3/gopacket.go:58 +0x39
github.com/dreadl0ck/ja3.ReadInterfaceCSV(0x7ffc2e4667e9, 0x4, 0x6360e0, 0xc000010018, 0x5e9566, 0x1)
    /home/morty/ja3/live.go:51 +0x1846
main.main()
    /home/morty/ja3/cmd/main.go:44 +0x2e2
dreadl0ck commented 3 years ago

Thanks for reporting!

I made a new release v1.0.1 that fixes these issues and implements dumping as JSON in live mode.

There are two versions of the library, one which uses the google/gopacket package and one that uses my fork, because mixing the the imports causes symbol errors when using CGO.

To build the new version from the google-gopacket branch you could do:

git pull
git checkout v1.0.1-google-gopacket
go build -o $(go env GOPATH)/bin/goja3 -i github.com/dreadl0ck/ja3/cmd

Let me know if it worked please, then I can close the ticket.

snappyJack commented 3 years ago

I followed your guide

git pull
git checkout v1.0.1-google-gopacket
go build -o $(go env GOPATH)/bin/goja3 -i github.com/dreadl0ck/ja3/cmd

and then input like this

[root@localhost ja3]# /root/go/bin/goja3 -ja3s=false -json -iface eth0

nothing output, even the title timestamp,source_ip,source_port,destination_ip,destination_port,ja3_digest

I'm sure the netflow is functional.

dreadl0ck commented 3 years ago

Hmm weird.

What version of go are you using ?

Are you compiling within $GOPATH using go modules?

Try forcing the use of go modules:

GO111MODULE=on go build -o $(go env GOPATH)/bin/goja3 -i github.com/dreadl0ck/ja3/cmd
snappyJack commented 3 years ago

Sorry I'm not familiar with go and his compilling,the go version is go version go1.13.1 linux/amd64

and the $GOPATH is empty, and 

echo $(go env GOPATH)/bin/goja3
/root/go/bin/goja3

I install it again with go build -o $(go env GOPATH)/bin/goja3 -i github.com/dreadl0ck/ja3/cmd,and also nothing output.

Did it works in your envirment?

dreadl0ck commented 3 years ago

No problem!

It works on my machine, so let's find out why it does not on yours.

1) go 1.13.1 is outdated, I recommend to update your build environment to 1.15.2 which is currently the latest stable release.

2) To check whether the build succeeded you could do the following:

$ goja3 -h
Usage of goja3:
  -csv
        print as CSV
  -debug
        toggle debug mode
  -iface string
        specify network interface to read packets from
  -ja3s
        include ja3 server hashes (ja3s) (default true)
  -ja3s-only
        dump ja3s only
  -json
        print as JSON array (default true)
  -read string
        read PCAP file
  -separator string
        set a custom separator (default ",")
  -snaplen int
        default snaplen for ethernet frames (default 1514)
  -tsv
        print as TAB separated values

If the build succeeded you should see the new -snaplen flag to set the buffer size for ethernet frames during live capture. You would only need to adjust that if using jumbo frames, I've set it to a sane default of 1514.

1500 Bytes Ethernet MTU + 14 Bytes Ethernet Header

3) When the -json switch is active you will see no output until a handshake is found, the first line with the field names is only printed for CSV output.

4) use the -debug flag to see if traffic arrives at the interface, you should see multiple errors from trying to parse TCP packets that do not contain TLS handshakes

handshake has a malformed length
handshake is of wrong type, or not a handshake message
{
    ...
}
handshake is of wrong type, or not a handshake message
handshake is of wrong type, or not a handshake message
handshake is of wrong type, or not a handshake message
handshake is of wrong type, or not a handshake message
handshake is of wrong type, or not a handshake message
snappyJack commented 3 years ago

I found where the problem is!

the new release v1.0.1 is works fine for normal interface

but when I check the -interface to Mirroring netflow,it didn't work,but it is worked on previous version.

the Mirroring netflow interface like this 

eth1      Link encap:Ethernet  HWaddr 40:A8:F0:23:4C:01  
          inet6 addr: fe80::42a8:f0ff:fe23:4c01/64 Scope:Link
          UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
          RX packets:951354158465 errors:57 dropped:69511261 overruns:0 frame:57
          TX packets:14 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:614146790390349 (558.5 TiB)  TX bytes:2756 (2.6 KiB)
          Interrupt:36

and the debug mode in new version in Mirroring netflow interface also output nothing

and I think this is an another bug....

dreadl0ck commented 3 years ago

Interesting, might be related to gopacket, I bumped the dependency in v1.0.1.

Just pushed a few more flags to master that allow you to adjust the parameters passed to the gopacket.OpenLive call, so you can adjust additionally:

If no output is printed in debug mode, I assume opening the interface handle failed and therefore no traffic is received.

The timeout was set to -1 which means never timeout, maybe thats what lead to not receiving anything. I've set the default to 30s now.

You can recompile by doing:

git checkout master
git pull
GO111MODULE=on go build -o $(go env GOPATH)/bin/goja3 -i github.com/dreadl0ck/ja3/cmd

If attaching to the interface times out, you should get an error after 30 seconds.

Could you describe how you've setup the mirroring in detail so that I can try to reproduce it?

snappyJack commented 3 years ago

the normal interface : some times get error immediately,some times get error a few seconds

[root@localhost ja3]# /root/go/bin/goja3 -ja3s=false -json -iface eth0
panic: Timeout Expired

goroutine 1 [running]:
github.com/dreadl0ck/ja3.ReadInterface(0x7ffea5e767e9, 0x4, 0x62cdc0, 0xc000010018, 0x5e1d76, 0x1, 0xc0001d0100, 0x5ea, 0xc0000ae001, 0x6fc23ac00)
    /home/morty/ja3/live.go:46 +0x26a9
main.main()
    /home/morty/ja3/cmd/main.go:56 +0x34e

before get error ,it could get output

[root@localhost ja3]# /root/go/bin/goja3 -ja3s=false -json -iface eth0
{
    "destination_ip": "180.101.49.11",
    "destination_port": 443,
    "ja3": "771,49195-49199-158-156-49162-49172-57-107-56-53-61-49161-49171-51-103-50-47-60-5-4-22-19-10,0-65281-10-11-13,23-24-25,0",
    "ja3_digest": "3ca5d63fa122552463772d3e87d276f2",
    "ja3s": "",
    "ja3s_digest": "",
    "source_ip": "192.168.100.157",
    "source_port": 46350,
    "timestamp": 1600395647.055365
}
panic: Timeout Expired

goroutine 1 [running]:
github.com/dreadl0ck/ja3.ReadInterface(0x7ffd1668b7e9, 0x4, 0x62cdc0, 0xc000010018, 0x5e1d76, 0x1, 0xc0001d0100, 0x5ea, 0xc0000ae001, 0x6fc23ac00)
    /home/morty/ja3/live.go:46 +0x26a9
main.main()
    /home/morty/ja3/cmd/main.go:56 +0x34e

and mirror interface get error after 30s

[root@localhost ja3]# /root/go/bin/goja3 -ja3s=false -json -iface eth1
panic: Timeout Expired

goroutine 1 [running]:
github.com/dreadl0ck/ja3.ReadInterface(0x7ffd8b5e17e9, 0x4, 0x62cdc0, 0xc000010018, 0x5e1d76, 0x1, 0xc0001d0100, 0x5ea, 0xc0000ae001, 0x6fc23ac00)
    /home/morty/ja3/live.go:46 +0x26a9
main.main()
    /home/morty/ja3/cmd/main.go:56 +0x34e

the mirror interface not made by my hand, and it probably made through switch ,you can see lots of RX packets and few TX packets

          RX packets:951354158465 errors:57 dropped:69511261 overruns:0 frame:57
          TX packets:14 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:614146790390349 (558.5 TiB)  TX bytes:2756 (2.6 KiB)
dreadl0ck commented 3 years ago

I read into the docs for the timeout value (https://godoc.org/github.com/google/gopacket/pcap#hdr-PCAP_Timeouts), and discovered that I misinterpreted the meaning.

It's actually the time the kernel can wait to batch packets and avoid excessive syscalls. Setting it to 0 is explicitly not recommended, and 30 seconds is way too high and probably the reason for the timeouts you are seeing.

There is a sane default pcap.BlockForever, which I've set as a default now. It sets the value to 10 milliseconds, which should work in most cases according to the authors.

I've pushed the changes to master, you can update your build with:

git pull
GO111MODULE=on go build -o $(go env GOPATH)/bin/goja3 -i github.com/dreadl0ck/ja3/cmd

Please try again and share the results.

snappyJack commented 3 years ago

I will try that this weekend