Documentation enhancement - Message format/content

[shane-od]

I think it would be beneficial to outline the format of the SQS/SNS message format in your documentation. I couldn't find it and even checked your test-logs for output of the SQS message.

I eventually figured out the issue I was having was NOT due to using an S3 notification to SQS, but rather due to the from_sns option being left out instead of being set to false.

I did locate the block of code to verify my format against here:

def preprocess(message)
    @logger.debug("Inside Preprocess: Start", :event => message)
    payload = JSON.parse(message.body)
    payload = JSON.parse(payload['Message']) if @from_sns
    @logger.debug("Payload in Preprocess: ", :payload => payload)
    return nil unless payload['Records']
    payload['Records'].each do |record|
      @logger.debug("We found a record", :record => record)
      # in case there are any events with Records that aren't s3 object-created events and can't therefore be
      # processed by this plugin, we will skip them and remove them from queue
      if record['eventSource'] == EVENT_SOURCE and record['eventName'].start_with?(EVENT_TYPE) then
        @logger.debug("record is valid")
        bucket  = CGI.unescape(record['s3']['bucket']['name'])
        key     = CGI.unescape(record['s3']['object']['key'])
        size    = record['s3']['object']['size']
        yield({
          bucket: bucket,
          key: key,
          size: size,
          folder: get_object_path(key)
        })
      end
    end

At the very least, I think you should reassure people that using a notification that sends the file info to SQS should work fine.