Make config keys optional

[bilalshaikh42]

The majority of keys in the conifg.yaml are not relevant for every installation. However, they must still be specified in the yaml leading to clutter.

For example, I ran into the following error since I forgot to specify the aws_dynamodb_gateway even though it is not used.

hsds entrypoint
node type:  sn
running hsds-servicenode
INFO> Service node initializing
INFO> service node initializing
Error applying command line override value for key: head_port: invalid literal for int() with base 10: ''
Traceback (most recent call last):
  File "/opt/env/hsds/bin/hsds-servicenode", line 8, in <module>
    sys.exit(main())
  File "/opt/env/hsds/lib/python3.8/site-packages/hsds/servicenode.py", line 244, in main
    app = create_app()
  File "/opt/env/hsds/lib/python3.8/site-packages/hsds/servicenode.py", line 220, in create_app
    initUserDB(app)
  File "/opt/env/hsds/lib/python3.8/site-packages/hsds/util/authUtil.py", line 194, in initUserDB
    if config.get("aws_dynamodb_gateway") and \
  File "/opt/env/hsds/lib/python3.8/site-packages/hsds/config.py", line 183, in get
    raise KeyError(f"config value {x} not found")
KeyError: 'config value aws_dynamodb_gateway not found'

This can be fixed by changing the following line to remove the check for default being null. @jreadey would returning None if the default is None cause any issues?

https://github.com/HDFGroup/hsds/blob/546de9de21943f73df191a876f59cc89cf53b197/hsds/config.py#L173-L184

[jreadey]

I think that would be ok - I don't see any code where there's a catch for KeyError with the handler doing something other than setting the value to be None.

But is the yaml clutter a big concern? Having all the keys defined in config.yml is documentation (of a sort) of the available config options. Specific deployments can use override.yml for those keys that need to be changed.