Added bounds checking on handshake message

[altf4]

• No longer crashes on bogus data
• Also fixed the return type be signed

Here's a python script to test it: https://gist.github.com/altf4/53c7c09cd33fd02f7ed581aa25ac2c27

tested on my end. Wii crashes if you run that python script in current master. Doesn't with my patch.