packetloop / packetpig

Packetpig - Open Source Big Data Security Analytics
www.packetloop.com
298 stars 85 forks source link

lib/run_emr LoadError #18

Open dmuhs opened 8 years ago

dmuhs commented 8 years ago

The command run in packetpig/lib/ with all environment variables set correctly

./run_emr -w -f s3://<my_bucket>/packetpig/pig/examples/snort.pig -r s3://<my_bucket>/packetpig/data/web.pcap -o s3://<my_bucket>/output/ -l s3://<my_bucket>/logging/

results in the following output:

/usr/bin/env ruby lib/elastic-mapreduce-ruby/elastic-mapreduce --create --name "spoons's pig jobflow" --bootstrap-action s3://elasticmapreduce/bootstrap-actions/configure-daemons --args --namenode-opts=-XX:GCTimeRatio=19 --bootstrap-action s3://elasticmapreduce/bootstrap-actions/configure-hadoop --args -m,mapreduce.task.timeout=0 --args -m,mapred.task.timeout=0 --args -m,tasktracker.http.threads=40 --args -m,mapred.map.tasks.speculative.execution=false --args -s,mapred.map.tasks.speculative.execution=false --bootstrap-action s3://packetpig/bootstrap.sh --instance-group master --instance-count 1 --instance-type m1.large --instance-group core --instance-count 1 --instance-type m1.large --alive --pig-script s3://<my_bucket>/packetpig/pig/examples/snort.pig --step-name snort.pig --args -p,includepath=s3://packetpig/pig/include-emr.pig,-p,snortconfig=/mnt/var/lib/snort-2931/etc/snort.conf,-p,dnspath=/mnt/var/lib/packetpig/lib/scripts/dns_parser.py,-p,tcppath=/mnt/var/lib/packetpig/lib/scripts/tcp.py,-p,pcap=s3://<my_bucket>/packetpig/data/web.pcap,-p,output=s3://<my_bucket>/output/output-1464276028 --ami-version 2.2 --hadoop-version 1.0.3 --pig-version 0.9.2.2
/usr/bin/ruby-mri: No such file or directory -- lib/elastic-mapreduce-ruby/elastic-mapreduce (LoadError)

./run_emr:146:in `create_cluster': undefined method `[]' for nil:NilClass (NoMethodError)
    from ./run_emr:20:in `run'
    from ./run_emr:444:in `<main>'

ruby -v provides ruby 2.2.5p319 (2016-04-26 revision 54774) [x86_64-linux] and the elastic-mapreduce gem is installed. Is the script deprecated by now? Or is this a bug on my side?