php-kafka / php-avro-schema-generator

PHP avro subschema merger and experimental PHP Class avro schema generator
BSD 3-Clause "New" or "Revised" License
5 stars 5 forks source link

Generator: nested record types generated incorrectly #52

Open Hubbitus opened 2 years ago

Hubbitus commented 2 years ago

Said run your example of generation we got output schema PhpKafka.PhpAvroSchemaGenerator.Example.SomeTestClass.avsc (part):

    {
      "name": "someOtherTestClass",
      "type": "PhpKafka.PhpAvroSchemaGenerator.Example.SomeOtherTestClass"
    },
    {
      "name": "someOtherTestClasses",
      "type": {
        "type": "array",
        "items": "PhpKafka.PhpAvroSchemaGenerator.Example.SomeOtherTestClass"
      }
    },
    {
      "name": "blaaaaaaaa",
      "type": [
        "int",
        "string"
      ]
    },
    {
      "name": "unicorn",
      "type": "PhpKafka.PhpAvroSchemaGenerator.Example.Wonderland"
    }

Which is incorrect! AVRO does not have types like PhpKafka.PhpAvroSchemaGenerator.Example.Wonderland!

In your example it is not very representative, because no any inner classes have fields, but said if Wonderland will have single field like:

**                                                                                                                                                                                                                                                                           
* Country of miracles
**/
class Wonderland                                                                                                                                                                                                                                                              
{                                                                                                                                                                                                                                                                             
    private string $land;                                                                                                                                                                                                                                                     
}

That generated part instead of

    {
      "name": "unicorn",
      "type": "PhpKafka.PhpAvroSchemaGenerator.Example.Wonderland"
    }

should look like:

{
  "name": "unicorn",
  "type": "record",
  "doc": "Country of miracles",
  "fields": [
    {
      "name": "land",
      "type": "string"
    }
  ]
}
nick-zh commented 2 years ago

This project contains a schema merger, this helps you split up your schema in sub schemas to have a better overview. So the generator as well will create templates that you can merge which will result in the outcome you posted last :v:

Hubbitus commented 2 years ago

But generated schemas should be correct? An example may show how to create templates and schemes...

Could you please provide an example of what I should call after:

$schemas = $generator->generate();
$generator->exportSchemas($schemas);

To get correct AVRO schemes?

nick-zh commented 2 years ago

Just to elaborate a bit more, let's say you have a class:

class User {
  public Location $location;
}

class Location {
  public string street;
  etc.
}

Then generate will create two templates:

Merge will unify that, but this way you will not be overwhelmed if you have complicated schema. You can find example how to merge your templates here Basically just pass the output directory of generate, and it will merge your generated templates into proper schema. Hope this helps :v:

Hubbitus commented 2 years ago

So, essentially $schemas = $generator->generate(); generates templates and not schemas?

nick-zh commented 2 years ago

That is correct Edit: I updated the readme, to make this a bit more clear

Hubbitus commented 2 years ago

Very contr-intuitive on my mind. I think at least extending examples with comments and full procedure to get correct schemes would be very helpful

nick-zh commented 2 years ago

I totally agree, i should have probably added an example which does both in one go. The initial generator was not so good, so manual adjustment was needed, but with your input and contributions, the generator has improved a lot. It would be ok, to generate & directly merge now :v:

I should probably also add cli examples in the examples folder :smile: