KCL not doing what I'm expecting it to do

[ghost]

I'm running the sample code.

I'm running

sample_kinesis_wordputter.py --stream words -w cat -w dog -w bird -w lobster

and I get some words pushed

Connecting to stream: words in us-east-1
Put word: cat into stream: words
Put word: dog into stream: words
Put word: bird into stream: words
Put word: lobster into stream: words

but when I run python sample_kclpy_app.py, I don't have any output.

I'm expecting python sample_kclpy_app.py to ouput "getting word: cat..." or something along those lines.

Am I missing something? It looks like the stream is not being listened to.

[knorwood]

The sample app doesn't actually do anything with the records, but you can add something like

print data, partition_key, sequence_number

here https://github.com/awslabs/amazon-kinesis-client-python/blob/master/samples/sample_kclpy_app.py#L94:L96

[ghost]

thanks for the guidance.

doing what you mentioned produces no output.

I re-ran the word putter from command line, that finishes with no problems.

[ghost]

could this be that the producer is using a different region than the consumer? I can see that the region is set int sample.properties, would this possibly related to this issue https://github.com/awslabs/amazon-kinesis-client-ruby/issues/4?

[ksmin23]

If you install amazon_kclpy-1.1.0-py2.7.egg as follows,

virtualenv /tmp/kclpy-sample-env

source /tmp/kclpy-sample-env/bin/activate

pip install amazon_kclpy

you can find sample_kclpy_app.py file in the following directory

/tmp/kclpy-sample-env/lib/python2.7/site-packages/amazon_kclpy-1.1.0-py2.7.egg/EGG-INFO/scripts/sample_kclpy_app.py 

if you want to test sample_kclpy_app.py working correctly, you change sample_kclpy_app.py in the above directory, and then rerun.

[linusthe3rd]

Another issue i found was that the MultiLangDaemon was running, but I was getting zero output to stdout or to a local text file whenever process_record was called.

After stumbling upon this SO post, the following got my custom code to work correctly:

1. ensure that sample_kclpy_app.py is on your $PATH
2. in sample.properties, make sure that executableName is set to the following:

executableName = python sample_kclpy_app.py

Doing that resulted in the following output when messages were read:

INFO: Skipping unexpected line on STDOUT for shard shardId-000000000000: TACO: batman

where "TACO:" is coming from my custom code in process_record:

def process_record(self, data, partition_key, sequence_number):
    print("TACO: {}".format(data))
    return

[sheldonkreger]

In addition to what is outlined in previous comment, I also had to re-run python setup.py install

[sonnyrajagopalanuptycs]

Not a single one of these suggestions work for me. Isn't there a single, decent way to read from a kinesis stream?