Added the "Explode" transform

[brettcvz]

This will be useful for the mongo->redshift pipeline. Right now, we can't create useful "many to many" Foreign Key tables such as school admin IDs linked to the schools they work with. We can't create such tables, because we have no way of expanding (or "exploding") the array of schools associated with the admin object into a set of rows for sql

Other places this transform be useful in bringing data into redshift:

• Coteachers for sections
• Students for sections
• Auth types for launchpads
• Grants for oauthtokens
• Permissions for students/teachers/sections
• Tags for oauthclients

[brettcvz]

My first Go PR :raised_hands: Perhaps a bit ambitious (reflection, type assertions, attempts at basically using generics in go...) but it was fun

[natebrennand]

Nice! This is an ambitious first Go PR!

---

Thoughts on a more generalized function like this?

func MapMany(transform func(optimus.Row) ([]optimus.Row, error)) optimus.TransformFunc

I don't think this should be tied quite as closely to having an array in the row. MapMany would allow expanding rows based on embedded maps as well, or whatever the user wants to use. Or maybe we should do both? Explode would be useful as a convenience function.

[brettcvz]

Re. MapMany, I'm not sure I understand the use case, but it's definitely possible. One thing that held me back from adding too much functionality to Explode (for instance, could you pass in a list of keys and explode on each?) was that you can always compose functions. So if you wanted to explode multiple keys, explode one and then the other. Or if you wanted other behavior, you could do a Explode(Map(source)) or visa versa.

[azylman]

Haven't looked at the code yet, but my personal opinion is that if we only have one or two places we want this right now, we should leave it out of optimus and define the transform in those places.

[brettcvz]

Ok ok fine.. :)

Still was fun though