Custom fields

[tadjik1]

Hello, since this PR #68 it's impossible to create custom fields for documents (because they are ignoring). So what's the official way for creating custom properties?

Let's say we want to keep with every document timestamp of related import.

[missinglink]

hi @tadjik1, I actually didn't consider that when working on this feature.

how would you feel about overloading the toESDocument method by either replacing the function with the old behavior or creating a new class which extended Document?

for planet-size builds, this change makes a big impact on our build times so I would prefer not to revert it if possible.

[missinglink]

something like this?

const util = require('util');
const Document = require('pelias-model').Document;

function TimestampedDocument() {
  Document.call(this); // or use apply() to pass arguments to the super constructor
}

util.inherits(TimestampedDocument, Document);

TimestampedDocument.prototype.toESDocument = function() {
  var esdoc = Document.prototype.toESDocument.apply(this);
  esdoc.timestamp = xxx
  return esdoc;
};

sorry for the hasty code, I didn't test this, it's just to demonstrate how it could work :)

[missinglink]

you can also use the 'meta' property to store the timestamp.

[tadjik1]

@missinglink it will be enough, thanks!

[diego-kuik]

Hi @missinglink / @tadjik1 ,do you guys know if is it possible to show meta properties in the queries JSON response?

[orangejulius]

Hi everyone, This issue is growing in importance. Lots of people have requested the ability to store custom data in Pelias and return it in the API. Thus far we haven't really provided a good way.

The existing meta field was never intended to allow for storing arbitrary data in Elasticsearch for later display[1].

We should add support for this. The most important consideration is that the field should not slow down general API queries. This means:

• The field we store custom data in should NOT be indexed by Elasticsearch, but only stored. (see here for the difference).
• The resulting custom data should probably only be displayed on calls to the /v1/place endpoint, unless we build a way to ensure the amount of data stored is very small

To fit with the rest of the Pelias API, storing a custom JSON object feels like a decent approach. Anyone else have thoughts on the implementation?

[1] It is only used in the OSM importer to store data needed at different stages of the import process, but not after import. We should probably refactor that importer to store that information another way.

[orangejulius]

I've taken some of the discussion above and put into a top-level Pelias ticket over in https://github.com/pelias/pelias/issues/747, so this is now a duplicate. Any custom data discussion can happen in the new ticket!