umpirsky / Extraload

Powerful ETL library.
MIT License
34 stars 5 forks source link

Adding a MappingFieldsTransformer #18

Open programarivm opened 6 years ago

programarivm commented 6 years ago

Hi there,

Thanks for sharing Extraload with the community. It would be great if some more examples and documentation could be added to the library.

Why would we need a MappingFieldsTransformer?

I noticed that many times we'll go like this.

Source books:

isbn, title, author

Destination my_books:

foo, bar

In this example we're loading the contents of books.isbn into my_books.foo; the same thing goes for books.author (we load its content into my_books.bar). And note the books.title field is not used in the target table.

The example above shows a 3 -> 2 mapping between the tables books and my_books because we want to (1) change the names of the fields in the target table and (2) be able to choose which source table fields are actually loaded into the destination table.

More specifically, this 3 -> 2 mapping in particular could be described like this in the MappingFieldsTransformer:

$mapping = [
    'author' => 'foo',
    'title' => 'bar'
];

I believe this is very common -- I'll soon share a MappingFieldTransformer if all this makes sense. In fact it would be good to have a bunch of common transformers:

  1. MappingFieldsTransfomer
  2. TrimTransformer ...

And write a few more examples on using them.

programarivm commented 6 years ago

Just added a MapFieldsTransformer as described at this README.md in section Doctrine query with field transformation loaded into a table

umpirsky commented 6 years ago

I like the idea, posted my thoughts in https://github.com/umpirsky/Extraload/pull/17#discussion_r183299408. :+1: