[names] Incremental Web service caching and timeouts

[re1]

Timed out Web service caches are to be deleted if a new response was cached from the common names Web service. As the new caching implementation was not yet thoroughly tested with the OpenUp! database and data security is of high concern, elements are not yet deleted.

This issue will be closed after caching functionality is both fully functional and tested completely.

[re1]

Webservice caches are to be implemented incrementally in order to reduce storage used on service updates, Details will follow soon.

[re1]

The most popular library for calculating the difference of strings and patching them is Google's diff-match-patch tool. It might still be a good idea to consider an alternative as the Java implementation is rarely updated and not available on Maven.

java-diff-utils started as a fork of java-diff-utils from Google Code Archive and is now a still active independent project available from the Maven repository. The package is only available for Java 8 and above.

Other alternatives are mostly outdated or not intended for full diffs and patching.

[re1]

Google's solution can be used to save diffs as strings more easily while java-diff-utils' is still actively maintained and also more powerful. The letter will likely be chosen for the caching of differences between web service responses.

[re1]

The following chart shows the data flow specific to incremental caching. The full caching strategy is documented in the respective issue [names] Use OpenUp! database for caching (#15).

[re1]

As diffs are not serializable it might be a good idea to use Pherialize for share serialization. This might also improve portability as the format can also be used from PHP and JavaScript.

[re1]

An example of using java-diff-utils unified diff format to patch in older versions can be found here: https://github.com/java-diff-utils/java-diff-utils/wiki/Examples#generate-a-file-in-unified-diff-format-import-it-and-apply-the-patch

In our case the original is the Web service response and the patch shows the changes needed in order to build the previous cached response.

//importing unified diff format from file or here from memory to a Patch
Patch<String> latestResponsePatch = UnifiedDiffUtils.parseUnifiedDiff(latestResponseDiff);

//apply patch to original list
List<String> lastCachedResponse = DiffUtils.patch(cachedResponse, latestResponsePatch);

System.out.println(lastCachedResponse);

[re1]

The following DDL can be used to create the Web service cache diffs table:

create table openup.tbl_webservice_cache_diffs
(
    id int auto_increment
        primary key,
    tbl_webservice_cache_id int not null,
    diff longtext not null,
    timestamp int not null,
    constraint tbl_webservice_cache_id
        foreign key (tbl_webservice_cache_id) references openup.tbl_webservice_cache (id)
            on update cascade on delete cascade
);

This change is required on both production and OpenUp! databases.