Error with configuration file parser?

mturilli commented 9 years ago

The following might produce a silent error when the configuration file is parsed:

cluster_type=slurm hostname=stampede.tacc.xsede.org username="name"

Note the "" around the user name. Would it be possible to verify and confirm whether this is a bug?

Francis-Liu commented 9 years ago

In this case, bundle manager will use ("name") to try to connect to the cluster, which will result in a authentication failure exception. Bundle manager will call logging.exception to print the exception.

Did you get a silent error of this while running SC14 tutorial?

mturilli commented 9 years ago

Yadu did - silent meaning that bundles did not considered having a name between commas an error. Should we ignore the single/double commas? I don't think they are valid symbols for login names or resource FQDN.

Francis-Liu commented 9 years ago

In my opinion, Bundle should not ignore single/double commas. If we did, then we should also support things like cluster_type="slurm", or that checking whether a username is valid or not. I suggest we leave all these checking to cluster. If double-quoted username is wrong, then connection to the cluster will generate a exception.

So for this issue, my idea is that we do not make changes to Bundle code.

However, I do notice that bundle_manager's logging does not work properly. Because even though logging.exception is called, the output is not shown on terminal.

I will create another issue about "logging" of Bundle.

Once the logging works properly, user will be able to see error message complaining of authentication error once username="user" is provided.

andre-merzky commented 9 years ago

Hi Francis :),

we have a json parser in radical.utils (see https://github.com/radical-cybertools/radical.utils/blob/master/radical/utils/read_json.py) which we use for all our config files (see for example https://github.com/radical-cybertools/radical.pilot/blob/master/src/radical/pilot/configs/xsede.json). That json parser allows for # type comments (for complete lines), which is convenient. You may want to consider using that parser, to? That would free us from using different configuration formats, and makes detection of quotation errors etc. much simpler...

On a related note, you may also want to consider using other parts of our stack, such as the config files from radical pilot, or the ssh interactions from SAGA. For example, the following will run an ssh command on the target host, using the same credentials setups we use in SAGA and RADICAL-Pilot:

        import saga.utils.pty_shell as sup
        shell = sup.PTYShell('ssh://stampede.tacc.utexas.edu')
        ret, out, err = shell.run_sync ("echo $HOME")
        if ret == 0:
           print "home: %s" % out
       else:
           print "homeless: %s" % err

The same will work for gsissh acces (and of course localhost).

Let me know if you are interested -- happy to share code examples and documentation :)

Francis-Liu / aimes.bundle

Error with configuration file parser? #1