Catch invalid regular expressions in the configuration

[OXYAMINE]

Daemon doesn't start.

Running command:

dynamic-dynamodb -c /etc/dynamic_dynamoDB.conf --log-config-file dynamic_dynamoDB_logging.conf --daemon foreground

Output is:

2014-03-11 00:54:46,691 - dynamic-dynamodb - INFO - Test_Item - Consumed read units: 12%
2014-03-11 00:54:46,714 - dynamic-dynamodb - INFO - Test_Item - Read throttle count: 0
2014-03-11 00:54:46,772 - dynamic-dynamodb - INFO - Test_Item - Consumed write units: 0%
2014-03-11 00:54:46,790 - dynamic-dynamodb - INFO - Test_Item - Write throttle count: 0
2014-03-11 00:54:46,790 - dynamic-dynamodb - INFO - Test_Item - No need to change provisioning
Traceback (most recent call last):
  File "/usr/local/bin/dynamic-dynamodb", line 27, in <module>
    dynamic_dynamodb.main()
  File "/usr/local/lib/python2.6/dist-packages/dynamic_dynamodb/__init__.py", line 128, in main
    check_interval=config['global']['check_interval'])
  File "/usr/local/lib/python2.6/dist-packages/dynamic_dynamodb/__init__.py", line 52, in run
    if re.match(gsi_key, gsi_name):
  File "/usr/lib/python2.6/re.py", line 137, in match
    return _compile(pattern, flags).match(string)
  File "/usr/lib/python2.6/re.py", line 245, in _compile
    raise error, v # invalid expression
sre_constants.error: nothing to repeat

What is this?

[sebdah]

Please attach your configuration. I would assume that you have an invalid regular expression in it, given the last rows in the exception. On Mar 11, 2014 5:56 AM, "OXYAMINE" notifications@github.com wrote:

Daemon doesn't start.

Running command: dynamic-dynamodb -c /etc/dynamic_dynamoDB.conf --log-config-file dynamic_dynamoDB_logging.conf --daemon foreground

Output is: 2014-03-11 00:54:46,691 - dynamic-dynamodb - INFO - Test_Item - Consumed read units: 12% 2014-03-11 00:54:46,714 - dynamic-dynamodb - INFO - Test_Item - Read throttle count: 0 2014-03-11 00:54:46,772 - dynamic-dynamodb - INFO - Test_Item - Consumed write units: 0% 2014-03-11 00:54:46,790 - dynamic-dynamodb - INFO - Test_Item - Write throttle count: 0 2014-03-11 00:54:46,790 - dynamic-dynamodb - INFO - Test_Item - No need to change provisioning Traceback (most recent call last): File "/usr/local/bin/dynamic-dynamodb", line 27, in dynamic_dynamodb.main() File "/usr/local/lib/python2.6/dist-packages/dynamic_dynamodb/init.py", line 128, in main check_interval=config['global']['check_interval']) File "/usr/local/lib/python2.6/dist-packages/dynamic_dynamodb/init.py", line 52, in run if re.match(gsi_key, gsi_name): File "/usr/lib/python2.6/re.py", line 137, in match return

_compile(pattern, flags).match(string) File "/usr/lib/python2.6/re.py", line 245, in _compile raise error, v # invalid expression sre_constants.error: nothing to repeat

What is this?

Reply to this email directly or view it on GitHubhttps://github.com/sebdah/dynamic-dynamodb/issues/125 .

[OXYAMINE]

[global]
# AWS access keys
aws-access-key-id: removed
aws-secret-access-key-id: removed

# AWS region to use
region: us-east-1

# How often should Dynamic DynamoDB monitor changes (in seconds)
check-interval: 300

# Circuit breaker configuration
# No provisioning updates will be made unless this URL returns
# a HTTP 200 OK status code
#circuit-breaker-url: http://my.service.com/v1/is_up
#circuit-breaker-timeout: 500

[logging]
# Log level [debug|info|warning|error]
log-level: info

# Log file (comment out to get only console output)
log-file: /var/log/dynamic-dynamodb.log

# External Python logging configuration file
# Overrides both log-level and log-file
# log-config-file: /path/to/logging.conf

[table: Test_Item]
#
# Read provisioning configuration
#

# Thresholds for scaling up or down the provisioning (%)
reads-upper-threshold: 90
reads-lower-threshold: 30

# How many percent should Dynamic DynamoDB increase/decrease provisioning with (%)
increase-reads-with: 10
decrease-reads-with: 10

# Units to increase or decrease reads with, must be either percent or units
increase-reads-unit: percent
decrease-reads-unit: percent

# Maximum and minimum read provisioning
# Dynamic DynamoDB will not provision any more or less reads than this
min-provisioned-reads: 5
max-provisioned-reads: 500

#
# Write provisioning configuration
#

# Thresholds for scaling up or down the provisioning (%)
writes-upper-threshold: 90
writes-lower-threshold: 30

# How many percent should Dynamic DynamoDB increase/decrease provisioning with (%)
increase-writes-with: 10
decrease-writes-with: 10

# Units to increase or decrease writes with, must be either percent or units
increase-writes-unit: percent
decrease-writes-unit: percent

# Maximum and minimum write provisioning
# Dynamic DynamoDB will not provision any more or less writes than this
min-provisioned-writes: 5
max-provisioned-writes: 500

#
# Maintenance windows (in UTC)
#
#maintenance-windows: 22:00-23:59,00:00-06:00

#
# Other settings
#

# Allow down scaling when at 0% consumed reads
allow-scaling-down-reads-on-0-percent: true
allow-scaling-down-writes-on-0-percent: true

# Restric scale down to only happen when BOTH reads AND writes are in need
# of scaling down. Set this to "true" to minimize down scaling.
#always-decrease-rw-together: true

[gsi: * table: Test_Item]
#
# Read provisioning configuration
#

# Thresholds for scaling up or down the provisioning (%)
reads-upper-threshold: 90
reads-lower-threshold: 30

# How many percent should Dynamic DynamoDB increase/decrease provisioning with (%)
increase-reads-with: 50
decrease-reads-with: 50

# Units to increase or decrease reads with, must be either percent or units
increase-reads-unit: percent
decrease-reads-unit: percent

# Maximum and minimum read provisioning
# Dynamic DynamoDB will not provision any more or less reads than this
min-provisioned-reads: 4
max-provisioned-reads: 500

#
# Write provisioning configuration
#

# Thresholds for scaling up or down the provisioning (%)
writes-upper-threshold: 90
writes-lower-threshold: 30

# How many percent should Dynamic DynamoDB increase/decrease provisioning with (%)
increase-writes-with: 50
decrease-writes-with: 50

# Units to increase or decrease writes with, must be either percent or units
increase-writes-unit: percent
decrease-writes-unit: percent

# Maximum and minimum write provisioning
# Dynamic DynamoDB will not provision any more or less writes than this
min-provisioned-writes: 4
max-provisioned-writes: 500

#
# Maintenance windows (in UTC)
#
#maintenance-windows: 22:00-23:59,00:00-06:00

#
# Other settings
#

# Allow down scaling when at 0% consumed reads
allow-scaling-down-reads-on-0-percent: true
allow-scaling-down-writes-on-0-percent: true

# Restric scale down to only happen when BOTH reads AND writes are in need
# of scaling down. Set this to "true" to minimize down scaling.
#always-decrease-rw-together: true

[OXYAMINE]

[global]
# AWS access keys
aws-access-key-id: XXX
aws-secret-access-key-id: XXXX

# AWS region to use
region: us-east-1

# How often should Dynamic DynamoDB monitor changes (in seconds)
check-interval: 300

# Circuit breaker configuration
# No provisioning updates will be made unless this URL returns
# a HTTP 200 OK status code
#circuit-breaker-url: http://my.service.com/v1/is_up
#circuit-breaker-timeout: 500

[logging]
# Log level [debug|info|warning|error]
log-level: info

# Log file (comment out to get only console output)
log-file: /var/log/dynamic-dynamodb.log

# External Python logging configuration file
# Overrides both log-level and log-file
# log-config-file: /path/to/logging.conf

[table: Test_Item]
#
# Read provisioning configuration
#

# Thresholds for scaling up or down the provisioning (%)
reads-upper-threshold: 90
reads-lower-threshold: 30

# How many percent should Dynamic DynamoDB increase/decrease provisioning with (%)
increase-reads-with: 10
decrease-reads-with: 10

# Units to increase or decrease reads with, must be either percent or units
increase-reads-unit: percent
decrease-reads-unit: percent

# Maximum and minimum read provisioning
# Dynamic DynamoDB will not provision any more or less reads than this
min-provisioned-reads: 5
max-provisioned-reads: 500

#
# Write provisioning configuration
#

# Thresholds for scaling up or down the provisioning (%)
writes-upper-threshold: 90
writes-lower-threshold: 30

# How many percent should Dynamic DynamoDB increase/decrease provisioning with (%)
increase-writes-with: 10
decrease-writes-with: 10

# Units to increase or decrease writes with, must be either percent or units
increase-writes-unit: percent
decrease-writes-unit: percent

# Maximum and minimum write provisioning
# Dynamic DynamoDB will not provision any more or less writes than this
min-provisioned-writes: 5
max-provisioned-writes: 500

#
# Maintenance windows (in UTC)
#
#maintenance-windows: 22:00-23:59,00:00-06:00

#
# Other settings
#

# Allow down scaling when at 0% consumed reads
allow-scaling-down-reads-on-0-percent: true
allow-scaling-down-writes-on-0-percent: true

# Restric scale down to only happen when BOTH reads AND writes are in need
# of scaling down. Set this to "true" to minimize down scaling.
#always-decrease-rw-together: true

[gsi: * table: Test_Item]
#
# Read provisioning configuration
#

# Thresholds for scaling up or down the provisioning (%)
reads-upper-threshold: 90
reads-lower-threshold: 30

# How many percent should Dynamic DynamoDB increase/decrease provisioning with (%)
increase-reads-with: 50
decrease-reads-with: 50

# Units to increase or decrease reads with, must be either percent or units
increase-reads-unit: percent
decrease-reads-unit: percent

# Maximum and minimum read provisioning
# Dynamic DynamoDB will not provision any more or less reads than this
min-provisioned-reads: 4
max-provisioned-reads: 500

#
# Write provisioning configuration
#

# Thresholds for scaling up or down the provisioning (%)
writes-upper-threshold: 90
writes-lower-threshold: 30

# How many percent should Dynamic DynamoDB increase/decrease provisioning with (%)
increase-writes-with: 50
decrease-writes-with: 50

# Units to increase or decrease writes with, must be either percent or units
increase-writes-unit: percent
decrease-writes-unit: percent

# Maximum and minimum write provisioning
# Dynamic DynamoDB will not provision any more or less writes than this
min-provisioned-writes: 4
max-provisioned-writes: 500

#
# Maintenance windows (in UTC)
#
#maintenance-windows: 22:00-23:59,00:00-06:00

#
# Other settings
#

# Allow down scaling when at 0% consumed reads
allow-scaling-down-reads-on-0-percent: true
allow-scaling-down-writes-on-0-percent: true

# Restric scale down to only happen when BOTH reads AND writes are in need
# of scaling down. Set this to "true" to minimize down scaling.
#always-decrease-rw-together: true

[OXYAMINE]

OMG! why the font is so large? anyway... it's just a test config. Table - Test_item, all indexes

[OXYAMINE]

you're 100% right! changing * to .* seems solve the issue. Could you consider to check config before execution and output user friendly message? I feel myself stupid a little bit :)

[sebdah]

Alright :).

I'll definitely add a more human readable error message.

Happy hacking and thanks for the report! On Mar 11, 2014 6:21 AM, "OXYAMINE" notifications@github.com wrote:

you're 100% right! changing * to .* seems solve the issue. Could you consider to check config before execution and output user friendly message? I feel myself stupid a little bit :)

Reply to this email directly or view it on GitHubhttps://github.com/sebdah/dynamic-dynamodb/issues/125#issuecomment-37264432 .

[sebdah]

Forgot to say, GitHub uses Markdown for formatting the comments, that's why it looks a bit messed up. I have fixed the formatting above :).

[OXYAMINE]

Thanks a lot

[sebdah]

This has been fixed in version 1.9.1

[OXYAMINE]

thanks a lot.