Method to verify data made it to kafka

[gshif]

Need to implement a method to verify data made it to kafka from gateway

[gshif]

The Smoke Test (and other tests as well) would benefit from having a method/function to verify if data was written to kafka. There could be different ways of writing the data, but the one, that is used by Smoke Test, is using gateway write endpoint to write a single point.

• One way to check if data madd it to kafka would be to use a kubectl, i.e.:

  kubectl --kubeconfig=" --context=influx-internal exec kafka-0 -c k8skafka -n -- bash -c "(/opt/kafka/bin/kafka-console-consumer.sh --bootstrap-server kafka-svc:9093 --topic ingress --from-beginning > /tmp/out.log 2>&1 &) && sleep 2 && egrep -a \"[0]{4}\" '/tmp/out.log"

Basically, execute one of the kafka's shell commands, kafka-console-consumer.sh, and send data to a log file and later grep the parts of the sent data. The challenge with using this approach is that executing kafka-console-consumer.sh some times crashes the kafka container.

[gshif]

Another way is to use the same `kubectl command, but ... grep for a tag value from a topics directory in the kafka container : https://github.com/influxdata/idpe/pull/1971

[gshif]

Added a step to Smoke Test to verify data point made to kafka using the above method

[gshif]

Gave it up to 10 seconds for data to make it to kafka from gateway in case there is a latency (The threshold was added to a smoke test)