Internal terms lists grow indefinitely when populated from registry

[lukasgraf]

In vocabulary.py:56 and vocabulary.py:76 the internal lists VocabularyFactory.visible_terms and VocabularyFactory.hidden_terms get updated with values from the registry.

However, terms are only ever appended to those lists, and no check for duplicates is being done. This will cause those lists to grow indefinitely, because on ever __call__() the terms from the registry will be appended again. In production sites running for a couple weeks we had vocabularies with those lists having 28'000 items, most of them duplicates.

Proposed fix: Either

• iterate over record and only append() items that aren't already in the lists

or

• extend() the list by record, make it a set and turn it into a list again.

I slightly prefer the first solution because it preserves order (shouldn't matter for the implementation, but makes for writing nicer tests).

[garbas]

yup ... first one it is, will look into it

[phgross]

:+1: looks good for me.

[lukasgraf]

Rebased on master after merging #2

[phgross]

:+1: