Support for Consumer lag metric

[RohanAshar]

Summary

For kafka based source pipelines, it would be great if a metric like consumer lag would be supported. This would give insight to the current lag of the pipeline and give a better picture of the current state.

Use Cases

Monitoring the current state of the system.

---

Message from the maintainers:

If you wish to see this enhancement implemented please add a 👍 reaction to this issue! We often sort issues this way to know what to prioritize.

[vigith]

Aren't you able to get the consumer lag from the Kafka metrics? The consumer group and its delay should be there.