The main logic is here:
1 - gets3Object that includes getLastModified() (just contains a summary, do not download the whole metadata.json file.)
2- check the condition (cache contains up-to-date metadata)
3- If the cache contains up-to-date metadata, get it;
Otherwise, download it, set it to the cache, and return it.
Motivation and Context
How Has This Been Tested?
Screenshots (if appropriate):
Types of changes
[ ] Bug fix (non-breaking change which fixes an issue)
[x] Code improvements with no or little impact
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
Checklist:
[x] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
issue --> https://github.com/JohnSnowLabs/spark-nlp/issues/14221 Cache mechanism implementation for metadata.json. Nowadays, metadata.json is more than 10MB and it will increase in the future.
Description
The main logic is here: 1 - gets3Object that includes getLastModified() (just contains a summary, do not download the whole metadata.json file.) 2- check the condition (cache contains up-to-date metadata) 3- If the cache contains up-to-date metadata, get it; Otherwise, download it, set it to the cache, and return it.
Motivation and Context
How Has This Been Tested?
Screenshots (if appropriate):
Types of changes
Checklist: