search

azavea / osmesa

OSMesa is an OpenStreetMap processing stack based on GeoTrellis and Apache Spark
Apache License 2.0
79 stars 26 forks source link

StreamingAOIMonitor INTERVAL environment variable #172

Closed CloudNiner closed 4 years ago

CloudNiner commented 4 years ago

Allows interval to be set via env instead of CLI flag.

Demo

export INTERVAL=weekly
./sbt analytics/test:runMain osmesa.analytics.oneoffs.StreamingAOIMonitor --augmented-diff-source s3://bucket/path/to/aug-diffs

yields: [info] 13:47:02 AOIMonitorUtils: Processing stream for interval "Weekly" and sequence [3767750, 3777830]

Then adding the --interval param back:

export INTERVAL=weekly
./sbt analytics/test:runMain osmesa.analytics.oneoffs.StreamingAOIMonitor --augmented-diff-source s3://bucket/path/to/aug-diffs --interval d

yields a "daily" run again (since cli arg overrides environment):

[info] 13:48:51 AOIMonitorUtils:                                                                                                         
[info]  Running stream process from:                                                                                                     
[info]    3758846 (2019-11-05 09:21:00.0)                                                                                                
[info]  to:                                                                                                                              
[info]    3777831 (2019-11-18 13:46:00.0)                                                                                                
[info]  in replication stream                                                                                                            
[info] 13:48:51 AOIMonitorUtils: WHILE RUNNING DAILY UPDATE: catching up on too many days (13.184027777777779) of logs!