Proposal for: imposm3 "Custom field/filter functions"

[ImreSamu]

Hello,

Sometimes I only need unique "custom field" and I want just drop the hand made source code. but maybe others can reuse some code.

So my vision:

• We will create a community repository for "Custom field/filter functions"
• everybody can upload code ( without full golang testing , this repository less strict )
• we will create a fieldtype_install program, and anybody can install any "custom fieldtype function"
• we create a file and function name convention like
  • file: "fieldtype_xxxxx.go"
  • function: "getField_xxxx( )"
  • documentation rules
  • simple test files ( if somebody want ) or input - output data
• If the mapping code battle tested - then we can move to the imposm3 base code.
• And any "bool" return type "Custom field functions" can be used as a filter ( need investigate )
• from the fieldType metadata we can generate documentation.

The suggested procedure for install will be similar:

git clone https://github.com/omniscale/imposm3  ...
// fieldtype_install command: only download the Golang code and copy to the mapping directory
fieldtype_install "current_dateutc"
fieldtype_install "coalesce"
fieldtype_install "hstore_tags_keep"
fieldtype_install "hstore_tags_drop"
fieldtype_install "jsonb_example"
fieldtype_install "sleep"
godep go install ... 

I have created some draft proof of concept code:

Patch for the current system: https://github.com/ImreSamu/imposm3/tree/fieldtypes_addon

• create some NEW PostgreSQL field types (single byte internal char, jsonb, date, timestamp, ... )
• create a new function ( registerFieldTypes )
• define FieldType metadata structure. ( FieldTypeDoc )

Example function ( very draft ) https://github.com/ImreSamu/imposm3_fieldtypes/blob/master/mapping/fieldtype_current_dateutc.go

Example repository : https://github.com/ImreSamu/imposm3_fieldtypes draft examples ( later I will create a better documentation )

• "coalesce"
• "hstore_tags_keep"
• "hstore_tags_drop"
• "jsonb_example"
• "sleep"
• ...

I like the idea of 1 Golang file = 1 FieldType Definition

My question:

• This direction is accepted or not? I don't know your plan about this topic.

[olt]

Having custom field types (similar to Imposm 2) is something I'd really like to see in Impsom 3. Embedding Lua might be the best way to allow a user to define custom field types. But this will require a lot more work than manually adding a few missing types to the Imposm Go code.

Adding more types (timestamp etc.) and having some utility functions to make this more easy would be ok for me, but I don't like to introduce an external repository infrastructure with FieldTypeDoc, fieldtype_install, etc.

[ImreSamu]

Thank you for your answer.

I have created a first proposal : https://github.com/omniscale/imposm3/pull/86

Later, I would like to discuss & add some new types :

• "coalesce"
• "hstore_tags_keep"
• "hstore_tags_drop"
• "current_datetimeutc"
• "current_dateutc"
• "jobstart_datetimeutc"

and more later, I would like to testing some "geos-mapping" types like:

• GEOSPointOnSurface
• GEOSGetCentroid
• GEOSTopologyPreserveSimplify
• ...

Thank you in advance!