Introduce L2 cache for high usage APIs

[mahalakshme]

Currently for caching we are using only in-memory Map which is the default cache provider if not configured one. Eg: To retrieve from virtual_catchment_address_mapping table via the method getAddressLevelsForCatchmentAndMatchingAddressLevelTypeIds. But it still takes 3-11 secs. Hence looks like cache misses are higher.

Hence, to reduce the performance issues raised as part of https://github.com/avniproject/avni-server/issues/607 - we can add caching to fetching of users, concepts, concept answers and virtual_catchment_address_mapping table. All these calls though made frequently are necessary and hence cant be removed.

So for our app, looks like caching in memory might not be sufficient and a more robust cache provider like Ehcache which can overflow the entries to disk is needed. Unlike Redis it is also simple to configure. In comparison to other cache providers Ehcache seems to be suitable for our usecase.

Acceptance criteria:

• Remove in-memory map caching for address levels, if you think this is not any more necessary. The decision can be made based on whether caching of the existing and new one happens at different levels and if so, will retaining both provide any improved performance when compared to having just the l2 cache.
• Other concurrentmap caching related to glific can be retained
• fetching of users, concepts, concept answers and virtual_catchment_address_mapping table, - fetching of these can be configured to use Ehcache
• READ_WRITE concurrency strategy can be used

Out of scope:

Multi-level caches with different cache providers

References:

https://medium.com/vedity/spring-boot-caching-mechanism-8ef901147e60 https://www.baeldung.com/hibernate-second-level-cache https://gosunaina.medium.com/cache-redis-ehcache-or-caffeine-45b383ae85ee#:~:text=You%20can%20think%20of%20Redis,retrieve%20it%20in%20another%20language. https://medium.com/@shahto/scaling-spring-boot-with-hibernate-2nd-level-cache-on-redis-54d588fc8b06#:~:text=Level%202%20Cache%3A%20is%20a,is%20therefore%20disabled%20by%20default.

[vinayvenu]

probably caching of users as well might be necessary.

[vinayvenu]

Requests chosen to be optimized

1. eligiblePrograms, eligibleProgramEncounters etc (large number of encounters to be pulled in, with lots of calls to concept and concept_answer). Cache works within request
2. api/subject and api/encounter POST where multiple calls are made by the same user in a short while
3. retrieving User (across all calls). Since users are
4. AddressLevel retrievals for api/subject GET requests. Multiple calls to addressLevel happen over a single call

Any change to concepts/addressLevels/users will take a maximum of 10 minutes to take effect with this change.

[vinayvenu]

Verify concept, concept answer changes and how fast they are reflected in the system

[petmongrels]

debug has become true in application.properties

[vinayvenu]

fixed. moving to qa ready.

[mahalakshme]

Things to test:

• entries of users, concepts, concept answers and virtual_catchment_address_mapping table when updated should retrieve the right value next time.