cheekybits / genny

Elegant generics for Go
MIT License
1.71k stars 167 forks source link

use text/scanner to tokenize input documents for better handling of complex types #42

Closed coryb closed 6 years ago

coryb commented 7 years ago

The problem is complex types like maps are not correctly generated.

For demonstration create sample input for a generated struct with a corresponding map of that struct:

$ cat <<EOM > input
package main
import "github.com/cheekybits/genny/generic"

type MyType generic.Type

type MyTypeThing struct {
    thing MyType
}
type MapMyType map[string]MyTypeThing
EOM

Then pass the file through genny:

$ cat input | genny gen MyType=string
// This file was automatically generated by genny.
// Any changes will be lost if this file is regenerated.
// see https://github.com/cheekybits/genny

package main

type StringThing struct {
        thing string
}
type MapString map[string]stringThing

Note the generated map type is map[string]stringThing with incorrect capitalization on stringThing. The reason for this is that the current code treats the map[string]stringThing as a single token and it thinks stringThing should be lower case because map is lower case.

This PR tweaks how genny tokenizes the input, using the text/scanner codebase which is designed to parse go code. The changes simply tokenize everything looking only for identifiers. If it is not an identifier it goes into the output stream, otherwise it is processed to substitute in the generated type. This will turn map[string]MyTypeThing into these tokens: map, [, string, ], MyTypeThing which are processed individually. Where only the map, string and MyTypeThing would be marked as identifiers.