narcisoguillen / kafka-node-avro

ISC License
26 stars 13 forks source link

AVRO Schema needs to be first registered #22

Closed remy-lab closed 4 years ago

remy-lab commented 4 years ago

Hello, Quick question on how the producer works. When I use a Java producer producing AVRO contents, the producer handles sending the schema first to the schema registry and then validate my messages from this schema. Here, when I read all the steps and the code, I see that the code fetch schema but never send it to the schema registry. (I'm maybe wrong)

Do I need to register the schema first in the registry manually ? What is the best way to proceed ?

Thanks.

narcisoguillen commented 4 years ago

Hi @remy-lab sorry for taking so long to answer I have had a lot of work and has being hard to free my self up, unfortunately as you mention , this library only gets/fetches the schemas from the registry but does not create or manipulates them anyhow.

Yeah you would have to create the schema beforehand, I would try to see if I can spend some time and this could be a nice feature on the package.

Thanks for using this library and try to make it better.

Damonio commented 3 years ago

Hello,

I think there could be a quick fix for this, following confluentinc recommendation, their java client will try to register the schema using it's API every time you try to send a message, so I think trying to register for every message should be ok, or it can be cached until there is some change in the schema.

They have defined 3 ways to handle subject registration, but for now I have just used the one we use in our company which is topic registration, this will register the schema using their Rest-API, (check more details for key and value meaning in the subject

const axios = require('axios')

async function registerSchema(schemaRegistryURL, topic, schema) {
    const url = `${schemaRegistryURL}/subjects/${topic}-value/versions`;
    const payload = {schema: JSON.stringify(schema)};
    await axios.post(url, payload)
}