Using the standard parse function for creating entities does not generate an entity_id

alephdata / memorious

Lightweight web scraping toolkit for documents and structured data.

https://docs.alephdata.org/developers/memorious

MIT License

310 stars 59 forks source link

Using the standard parse function for creating entities does not generate an entity_id #187

Open Rosencrantz opened 3 years ago

Rosencrantz commented 3 years ago

the parse_ftm function should include the create of an entity_id value. This should be able to be defined within the yml but if it is not then it should be created from a hash of the url

sunu commented 3 years ago

I would suggest we use the same mechanism that we use in mapping - let the user supply a list of keys and use those properties to construct a unique and idempotent entity id.