ceumicrodata / mETL

mito ETL tool
161 stars 41 forks source link

"transform: ConvertType" converting too soon? #4

Open korenmiklos opened 11 years ago

korenmiklos commented 11 years ago

I have a chain of record modifiers, as below. mETL fails saying "float does not have a Strip attribute"

bfaludi commented 11 years ago

The field type is probably float or you have a finalType: Float attribute in the source section. Take consider to fix your config because I found some error!

I have tried the following:

source: 
  source: Static
  sourceRecords:
    - [' pb: 12.12','1st']
    - ['23.122 pb:','2nd']
    - ['11,22 pb:','3rd']
  fields:
    - name: population
      map: 0
      type: String
      finalType: Float
      transforms: 
        - transform: Strip 
          chars: ' pb:'
        - transform: ConvertType 
          fieldType: Float
    - name: position
      map: 1

target:
  type: JSON
  compact: false
  resource: result.json

It is worked fine for me. After that I tried to move the ConvertType and TransformField into manipulation step. This config is worked well for me too:

source: 
  source: Static
  sourceRecords:
    - [' pb: 12.12','1st']
    - ['23.122 pb:','2nd']
    - ['11,22 pb:','3rd']
  fields:
    - name: population
      map: 0
    - name: position
      map: 1

manipulations:
  - modifier: TransformField
    fieldNames: population 
    transforms: 
      - transform: Strip 
        chars: ' pb:'
      - transform: ConvertType 
        fieldType: Float

target:
  type: JSON
  compact: false
  resource: result.json

Important: Population field has not got finalType attribute in this scenario, because the type conversion will be in the manipulation section, not the reader part.

Please publish your configuration file in this issue and I will check the error.