Recalculate lag on heartbeat

[DeeChau]

Pull Request Template

Description

Fixes #4, introduces partition_last_offsets that is used to recalculate consumer lag on heartbeat.

Type of change

Please delete options that are not relevant.

• [x] Bug fix (non-breaking change which fixes an issue)
• [ ] This change requires a documentation update

How Has This Been Tested?

• [ ] TBD - Publish messages to kafka while consumer is not consuming? Pending dev tests, updated unit tests

  Checklist:

• [x] My code follows the style guidelines of this project

• [x] I have performed a self-review of my own code

• [x] I have commented my code, particularly in hard-to-understand areas

• [x] I have made corresponding changes to the documentation

• [x] My changes generate no new warnings

• [x] I have added tests that prove my fix is effective or that my feature works

• [x] New and existing unit tests pass locally with my changes

• [x] Any dependent changes have been merged and published in downstream modules

[DeeChau]

Revisited the way lag is calculated, For the following ActiveSupport::Notifications used for this operation,

• start_process_batch, use the last_offset option
• start_process_message use the offset option

to measure the last known offset for certain partitions upon message_processed

Use this current offset in each heartbeat operation and get the partitions last offset for the topic to calculate the lag.

[dorner]

It's a lot easier to follow now - good work!