search

heyyy / espa

Automatically exported from code.google.com/p/espa
0 stars 0 forks source link

Replace the Scene Cache #101

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
The scene_cache currently runs on the webservers and holds a full listing of 
every scene currently on the online cache.  This listing is rebuilt every 4 
hours and introduces a potential latency of 4 hours into the processing time in 
a worst case scenario.

This should be replaced with the lsrd multiprocess xmlrpc server that will run 
on 140 & 67, which has already been benchmarked and shown to be more than able 
to handle the request load.  

At the specified order production interval (currently 7 minutes) the webserver 
will create a single call with a list of all scenes that are on order and 
submit this to the xmlrpc server, which will return only the scene ids that are 
on cache.  The webserver can then update the database accordingly.  This is a 
drop in replacement for the current scene_cache... just requires running server 
code on the online cache.

Potential issue with unencrypted & unauthenticated POST/PUT calls from the 
webserver to the cache...  this may or may not need to be handled.

Original issue reported on code.google.com by davehil...@gmail.com on 2 Jan 2014 at 3:41

GoogleCodeExporter commented 9 years ago 
The scene_cache server is checked into the codebase under 
prototype/new_scene_cache.  Need to add another method to it to check for nlaps 
scenes, then it needs to be formally put onto 140/67.  Probably need to 
coordinate this with landsat.

Original comment by davehil...@gmail.com on 11 Feb 2014 at 3:17

GoogleCodeExporter commented 9 years ago 
Everything is working with the new scene cache implementation and it is 
deployed.  Only remaining item is to clean up the codebase location for the new 
scene cache and ensure all references to the former implementation are removed 
from the codebase and the system.

Original comment by davehil...@gmail.com on 2 Apr 2014 at 9:27

GoogleCodeExporter commented 9 years ago 
Scene cache has been replaced with xmlrpc daemons running on the online cache 
boxes and client calls from the web tier.

Original comment by davehil...@gmail.com on 21 Apr 2014 at 5:26

GoogleCodeExporter commented 9 years ago

Original comment by davehil...@gmail.com on 21 Apr 2014 at 5:27

GoogleCodeExporter commented 9 years ago

Original comment by davehil...@gmail.com on 21 Apr 2014 at 5:27

GoogleCodeExporter commented 9 years ago

Original comment by davehil...@gmail.com on 23 Apr 2014 at 5:01