EasyButton failure - cif -p test fails, data not in libz format

[cjhc]

Not sure yet is a missing dependency on a Ubuntu Server 14.04 LTS build.

Failures after build w/ EasyButton and after meeting Ubuntu cif-sdk perl client deps: sudo apt-get install -y git build-essential cpanminus libmodule-install-perl zlib1g-dev

~/massive-octo-spice⟫ cif -p Data input to inflate is not in libz format at /usr/local/share/perl/5.18.2/CIF/SDK/Client.pm line 189.

[cjhc]

cif@massive-octo-spice:~⟫ cif -p -d [2016-12-21T17:09:17,708Z][INFO][main:268]: starting up client... [2016-12-21T17:09:17,709Z][INFO][main:279]: pinging: https://localhost... [2016-12-21T17:09:17,709Z][DEBUG][CIF::SDK::Client:229]: generating ping... [2016-12-21T17:09:17,710Z][DEBUG][CIF::SDK::Client:170]: uri created: https://localhost/ping?gzip=1 [2016-12-21T17:09:17,710Z][DEBUG][CIF::SDK::Client:171]: making request... [2016-12-21T17:09:18,290Z][INFO][CIF::SDK::Client:175]: status: 200 [2016-12-21T17:09:18,290Z][INFO][CIF::SDK::Client:181]: response size: < 1MB [2016-12-21T17:09:18,290Z][DEBUG][CIF::SDK::Client:184]: decoding content..

[2016-12-21T17:09:18,291Z][DEBUG][CIF::SDK::Client:193]: Data input to inflate is not in libz format at /usr/local/share/perl/5.18.2/CIF/SDK/Client.pm line 189. Data input to inflate is not in libz format at /usr/local/share/perl/5.18.2/CIF/SDK/Client.pm line 189.

[cjhc]

gunzip is confirmed as being installed.

Code in question:

    if($resp->{'headers'}->{'content-type'} && $resp->{'headers'}->{'content-ty$
        $Logger->debug('decoding content..');
        if($resp->{'content'} !~ /^\[/){
            try {
                $Logger->debug('decompressing...');
                my $ret = decode_base64($resp->{'content'});
                **$ret = gunzip($ret);**
                $resp->{'content'} = $ret;
            } catch {
                my $err = shift;
                $Logger->debug($err);
                unless($err =~ /Data input to gunzip is not in gzip format/){
                    die($err);
                }
            };

sudo cpanm IO::Uncompress::Gunzip doesn't help here, and IO::Compress::Gzip was already installed.

[cjhc]

Tweaked the code to test an alternate gunzip in CIF/SDK/Client.pm:

# use Gzip::Faster;
use IO::Uncompress::Gunzip qw(gunzip);      

And I get this:

cif@cif:~/massive-octo-spice/hacking/platforms/ubuntu⟫ cif -p -d
[2016-12-23T16:13:51,925Z][INFO][main:271]: starting up client...
[2016-12-23T16:13:51,925Z][INFO][main:282]: pinging: https://localhost...
[2016-12-23T16:13:51,925Z][DEBUG][CIF::SDK::Client:230]: generating ping...
[2016-12-23T16:13:51,926Z][DEBUG][CIF::SDK::Client:171]: uri created: https://localhost/ping?gzip=1
[2016-12-23T16:13:51,926Z][DEBUG][CIF::SDK::Client:172]: making request...
[2016-12-23T16:13:52,481Z][INFO][CIF::SDK::Client:176]: status: 200
[2016-12-23T16:13:52,481Z][INFO][CIF::SDK::Client:182]: response size: < 1MB
[2016-12-23T16:13:52,482Z][DEBUG][CIF::SDK::Client:185]: decoding content..
[2016-12-23T16:13:52,482Z][DEBUG][CIF::SDK::Client:188]: decompressing...
Use of uninitialized value $_[1] in string eq at /usr/local/share/perl/5.18.2/IO/Compress/Base/Common.pm line 280.
malformed JSON string, neither tag, array, object, number, string or atom, at character offset 0 (before "(end of string)") at /usr/local/share/perl/5.18.2/CIF/SDK/Client.pm line 201.

Perhaps of note, sudo cpanm MIME::Base64 installed that missing module, but it doesn't make a difference.

                $ret = gunzip($ret);
                $resp->{'content'} = $ret;
            } catch {
                my $err = shift;
                $Logger->debug($err);
                unless($err =~ /Data input to gunzip is not in gzip format/){
#                    die($err);
                }

Commenting out the code to die results in successful pings, and a testmode run of the cif client appears to be processing events.

[2016-12-23T16:27:25,093Z][28985][INFO]: processed events: 9000 [2016-12-23T16:27:36,043Z][28985][INFO]: processing: -r /etc/cif/rules/default/alexa.yml -f top10 [2016-12-23T16:27:45,528Z][28985][INFO]: processed events: 10 [2016-12-23T16:27:48,062Z][28985][INFO]: processing: -r /etc/cif/rules/default/alienvault.yml -f reputation [2016-12-23T16:29:18,305Z][28985][INFO]: processed events: 32887

[cjhc]

For some reason the p5-cif-sdk cpanm install isn't working via script, but working when done manually post-easybutton:

 * Restarting daemon monitor monit                                    [ OK ] 
cif@cif:~/massive-octo-spice-2.00.06$ sudo chown `whoami`:`whoami` ~/.cif.yml
cif@cif:~/massive-octo-spice-2.00.06$ cif -p
Data input to inflate is not in libz format at /usr/local/share/perl/5.18.2/CIF/SDK/Client.pm line 189.
cif@cif:~/massive-octo-spice-2.00.06$ sudo cpanm https://github.com/csirtgadgets/p5-cif-sdk/archive/2.00.tar.gz
--> Working on https://github.com/csirtgadgets/p5-cif-sdk/archive/2.00.tar.gz
Fetching https://github.com/csirtgadgets/p5-cif-sdk/archive/2.00.tar.gz ... OK
Configuring p5-cif-sdk-2.00 ... OK
Building and testing CIF-SDK-2.00 ... OK
Successfully installed CIF-SDK-2.00
1 distribution installed
cif@cif:~/massive-octo-spice-2.00.06$ cif -p
roundtrip: 0.536087 ms
roundtrip: 0.538367 ms
roundtrip: 0.533054 ms
roundtrip: 0.509839 ms

[drcruft]

Just did an easybutton.sh install on Ubuntu 14.04LTS server, and the same issue after install. Fixed after a sudo cpanm https://github.com/csirtgadgets/p5-cif-sdk/archive/2.00.tar.gz post installation...

[R3MRUM]

had the same issue and this fixed it as well! thank you!