Add support for UPX packing and PDFs

[austinbyers]

to: @javuto cc: @airbnb/binaryalert-maintainers size: small resolves: #17 resolves: #92

Background

This updates BinaryAlert's native dependencies and adds 2 new ones (upx and pdftotext).

Changes

• YARA Analyzer Lambda function
  • Update native analyzer dependencies
    • upgrade yara: v3.7.0 => v3.7.1
    • add upx
    • add pdftotext and its dependent .so libraries
  • Fix a bug in the analyzer - metrics are not published if there were no binaries scanned
  • All files are upx-unpacked (if possible) before the standard YARA scanning
• CLI
  • Updates analyzer build process
  • The ./manage.py live_test now tests 2 new files:
    • eicar_packed.py.upx: (tests upx) Triggers a test YARA rule once unpacked
    • eicar_text.pdf: (tests pdftotext) Contains a substring which triggers a YARA rule

Testing

• Deploy to a test account
• ./manage.py live_test:

Uploading eicar.txt to S3: ... :eicar.txt_abfcacd77b5c...
Uploading eicar.tar.gz.bz2 to S3: ... :eicar.tar.gz.bz2_abfcacd77b5c...
Uploading eicar_packed.py.upx to S3: ... :eicar_packed.py.upx_abfcacd77b5c...
Uploading eicar_text.pdf to S3: ... :eicar_text.pdf_abfcacd77b5c...
Looking up version of genericsqs_binaryalert_analyzer:Production...
    [1/15] Querying DynamoDB table for the expected YARA match entries...
    [2/15] Querying DynamoDB table for the expected YARA match entries...

SUCCESS: Expected DynamoDB entries for the test files were found!
{
    "eicar.tar.gz.bz2": [
        "yextend:eicar_av_test",
        "yextend:eicar_substring_test"
    ],
    "eicar.txt": [
        "public/eicar.yara:eicar_av_test",
        "public/eicar.yara:eicar_substring_test",
        "yextend:eicar_av_test",
        "yextend:eicar_substring_test"
    ],
    "eicar_packed.py.upx": [
        "public/eicar.yara:eicar_substring_test",
        "yextend:eicar_substring_test"
    ],
    "eicar_text.pdf": [
        "yextend:eicar_substring_test"
    ]
}
Removing test files from S3...
Removing DynamoDB match entries...
Done!

[coveralls]

Coverage increased (+0.05%) to 93.067% when pulling 15c5d71551975eb358b7526bc5df32f2d0e8e737 on austin-upx-pdf into 5a35e019c4c8ef53fb5be4bdb5a797d69fa4ef11 on master.