samvera / questioning_authority

Question your authorities
Other
54 stars 30 forks source link

Optimize Local File Vocab Load #153

Open no-reply opened 7 years ago

no-reply commented 7 years ago

Local vocabularies in a YAML file are read from disk fresh for each call to the #all, #search, and #find operations. This can mean that the full file is read, and the in-memory YAML structure allocated/gc'd multiple times in a typical form operation. This is especially an issue for larger vocabularies

We should optimize this to hold the YAML data structures in memory while the application running. I think this is slightly more complex than a simple instance variable, since the Authority instances may be initialized repeatedly. Something like a singleton wrapping the YAML is probably called for.

See: https://github.com/samvera/questioning_authority/blob/master/lib/qa/authorities/local/file_based_authority.rb#L28

elrayle commented 6 years ago

The configuration yml files associated with the linked data processing side of QA are read in once.

See config/initializers/linked_data_authorities.rb.

This looks in the QA's config/authorities/linked_data directory and the same directory in the app that includes QA. The downside of reading in through an initializer is that it requires a server restart to pick up configuration changes and new configurations. But since these don't change very often, that hasn't been an issue thus far.