How to capture changes from specific tables?

[rahuljoshi123]

I have specified mysql tables(LogicalSources) I want to capture changes from in sources.json . When databus is getting an event for a table which is not specified in logical sources, I am getting null pointer exception in ORListener as I have not added schemas for these tables and have not configured these logical sources. These are the exact lines - VersionedSchema vs = _schemaRegistryService.fetchLatestVersionedSchemaBySourceName(_tableUriToSrcNameMap.get(tableName)); Schema schema = vs.getSchema(); // Here NullPointer

The application is running fine after this exception. Major concern here is, it is updating scn as -1 after handling non specified table events. So if the application stops after parsing events for unspecified tables, in the next run it will start from SCN -1.

Is there any good way to register specific tables and ignore others?

[chavdar]

Thanks for reporting this.

I have a fix which I will merge.

On Fri, Jan 20, 2017 at 7:56 AM, rahuljoshi123 notifications@github.com wrote:

I have specified mysql tables(LogicalSources) I want to capture changes from in sources.json . When databus is getting an event for a table which is not specified in logical sources, I am getting null pointer exception in ORListener. These are the exact lines - VersionedSchema vs = _schemaRegistryService.fetchLatestVersionedSchemaBySo urceName(_tableUriToSrcNameMap.get(tableName)); Schema schema = vs.getSchema(); // Here NullPointer

The application is running fine after this exception but its updating scn as -1 after handling non specified table events. So if the application stops after parsing events for unspecified tables. It will start from SCN as -1.

Is there any way to register specific tables and ignore others?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/linkedin/databus/issues/84, or mute the thread https://github.com/notifications/unsubscribe-auth/AA4sG1iH2qPIGKRw0HMtO6QffRYyKVjgks5rUNkygaJpZM4LpdE6 .

[rahuljoshi123]

Hi, Thanks for the reply. Please let me know once you merge the fix. Meanwhile if you share the fix branch, I can help in testing it.

Thanks, Rahul

[chavdar]

Done.

On Tue, Jan 24, 2017 at 1:43 AM, rahuljoshi123 notifications@github.com wrote:

Hi, Thanks for the reply. Please let me know once you merge the fix. Meanwhile if you share the fix branch, I can help in testing the fix.

Thanks, Rahul

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/linkedin/databus/issues/84#issuecomment-274755462, or mute the thread https://github.com/notifications/unsubscribe-auth/AA4sG51HnpmCHs5wfoITxdlh0K4_KbiLks5rVcfCgaJpZM4LpdE6 .

[rahuljoshi123]

Hi,

Fix you did is avoiding framing of avro records and the earlier exception of 'schema not available'.

But SCN is updated as -1 after processing a ignored source events. If relay restart after this, it will try to read events from scn -1.

I have done a fix - https://github.com/linkedin/databus/pull/85. Please let me know your thoughts on this.

[chavdar]

Merged. Thanks for the fix!

On Wed, Feb 1, 2017 at 2:01 AM, rahuljoshi123 notifications@github.com wrote:

Hi,

Fix you did is avoiding framing of avro records and the earlier exception of 'schema not available'.

But SCN is updated as -1 after processing a ignored source events. If relay restart after this, it will try to read events from scn -1.

I have done a fix - #85 https://github.com/linkedin/databus/pull/85. Please let me know your thoughts on this.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/linkedin/databus/issues/84#issuecomment-276617700, or mute the thread https://github.com/notifications/unsubscribe-auth/AA4sG1hBD2p9cdJ2SqsvX2ZYelHAlsz9ks5rYFf4gaJpZM4LpdE6 .