Allow for IDL imports from different dirs/classpath

[Miro-Marinov]

Let me start by complementing you on the very awesome and handy library!

A tiny request - correct me if I am wrong but right now, it looks like the only option to import another avro schema in IDL is to have the file being imported in the same directory as that of the importing avdl (e.g. https://github.com/julianpeeters/avrohugger/blob/master/avrohugger-core/src/test/avro/import.avdl).

This ultimately leads to a flat hierarchy where all of my (40+) .avsc and .idl files must be placed in the same folder. However, I would like to be able to group these events/entities nicely in different sub-directories to improve readability and most importantly to clearly define the boundaries of the different sub-domains (the so called bounded contexts) in our system.

I saw that there is a similar (resolved) request in Apache Avro: https://issues.apache.org/jira/browse/AVRO-971 - not sure if this is going to be of any help.

Thanks in advance and happy holidays! 🎅

[julianpeeters]

Thank you @Miro-Marinov ! What a pleasant comment, and the link is much appreciated. I'll see what I can do to match that behavior in the next week or so.

In the meantime, I expect that you'll be able to import from other directories as long as they are in the avroSource directory that avrohugger is configured to use, e.g. https://github.com/julianpeeters/sbt-avrohugger/blob/master/src/test/resources/avro/c.avdl#L3. You should even be able to do something like import idl "../bar/a.avdl"; as long as bar is known to avrohugger as an avroSource directory, e.g. avroSourceDirectories in Compile += (sourceDirectory in Compile).value / "bar". Still, not as nice as it could be, so big thanks for this report.

[Miro-Marinov]

Apologies for the mega late reply - took a needed long ass vacation : P This is exactly what I was looking for, thanks a bunch! (this can be closed now)

As a side note - I know that it is possible to form an ADT by defining a protocol and several records inside. From what I can see the generated sealed trait serves only as a marker interface, as well telling the compiler for all possible subtypes (very useful when pattern matching). But what about other advantages of inheritance and polymorphism?

For example, say, I have several user specific events - I can place them in a protocol called UserEvent. So far so good, but what if I want to say that each user event has a string-based id field. Is it possible to have this base field at the sealed trait level so that consumers of these events don't need to be concerned with the specific UserEvent implementation if they only care about the event id? I don't think Avro supports that - at least I couldn't find anything in the docs or a way to do this with avrohugger (or the maven code generation plugin for that matter).

Any input would be greatly appreciated!

[julianpeeters]

took a needed long ass vacation : P

hehe, sounds great. Very interesting inquiry, thank you, I could see how this would be useful in general, but there's no support for that I'm afraid.

Is it possible to have this base field at the sealed trait level so that consumers of these events don't need to be concerned with the specific UserEvent implementation if they only care about the event id?

not sure I follow, but in case it's useful: AFAIK if a consumer only cares about a single field sent by a consumer, then the consumer could take advantage of avro's schema evolution rules and read the data using a very basic schema. Not sure if that helps here.

I don't think Avro supports that - at least I couldn't find anything in the docs or a way to do this with avrohugger

No, not AFAIK. The immediate problem is that avro doesn't have a very rich syntax, and there's no built-in way to specify such a definition. We might be able to hijack the messages definition in idls/protos, and maybe get something to work, but a few issues come to mind:

It would no longer be avro. This question is making me rethink using the protocol name as the ADT root, that's not avro either. I'll consider making a subproject, which interprets avro as some SuperAvro of some sorts, but it would be

• very experimental
• major refactor
• very long-term side-project

[julianpeeters]

I'm still punting on putting members on the trait.

The classpath dependencies, however, is now expected to be working in avrohugger 2.0.0-RC16, if it is still of interest.