Support For Dynamic Options

[ericnewton76]

Issue by issafram Wednesday Aug 28, 2013 at 14:46 GMT Originally opened as https://github.com/gsscoder/commandline/issues/96

---

I have an application that will have 1 to many different arguments being passed in. The argument names will always be different depending on configuration (dynamic). So I will never know the name of the argument names as the designer of the application.

Does this library support something like that? It is great for parsing but it requires the options parameter to be defined. I have tried using an ExpandoObject but I don't think it works properly because attributes are required on the properties.

[ericnewton76]

Comment by nemec Wednesday Aug 28, 2013 at 15:32 GMT

---

No, I don't believe it's possible with this library. How "dynamic" are you talking?

Do you accept a completely different "set" of parameters based on a config value? If so, maybe you could create one Options class per config value and conditionally load it.

Or are there "flags" in the config that enable or disable a specific parameter? If so, it may be better to create one "master" Options class and just ignore any extraneous parameters.

[ericnewton76]

Comment by issafram Wednesday Aug 28, 2013 at 15:40 GMT

---

The application will be used as a framework. So there could be hundreds to thousands of configurations. I could not realistically create an Options class per config.

A configuration file will allow a user to add/remove a set of values that get loaded up at run-time with values for "key" and "description".

The values in the configuration file need to be able to be "overridable" by passing in values through the command line arguments. In order to validate the arguments passed in, I was thinking of using this library. But in order to do so, I need a valid Options class. That is why it needs to be dynamic.

Any possible workarounds or a possible enhancement for the future?

[ericnewton76]

Comment by nemec Wednesday Aug 28, 2013 at 17:12 GMT

---

If the keys are dynamic, would you be trying to parse the values as strongly typed objects (ints, Uris, etc.)? If not, you may be better off with a different approach, since this library is aimed more toward automatically populating an options object into strongly typed values.

I'm not sure if you could reuse the parser to do what you want without a bit of work first. On Aug 28, 2013 10:40 AM, "Issa Fram" notifications@github.com wrote:

The application will be used as a framework. So there could be hundreds to thousands of configurations. I could not realistically create an Options class per config.

A configuration file will allow a user to add/remove a set of values that get loaded up at run-time with values for "key" and "description".

The values in the configuration file need to be able to be "overridable" by passing in values through the command line arguments. In order to validate the arguments passed in, I was thinking of using this library. But in order to do so, I need a valid Options class. That is why it needs to be dynamic.

Any possible workarounds or a possible enhancement for the future?

— Reply to this email directly or view it on GitHubhttps://github.com/gsscoder/commandline/issues/96#issuecomment-23424596 .

[ericnewton76]

Comment by issafram Wednesday Aug 28, 2013 at 17:27 GMT

---

They are essentially a collection of strings. Much like a dictionary. I wanted to use the parser to validate that all arguments were passed through since I do not want to have to worry about the logic for parsing it myself.

[ericnewton76]

Comment by nemec Wednesday Aug 28, 2013 at 17:32 GMT

---

Nope, I don't think that's possible here. On Aug 28, 2013 12:27 PM, "Issa Fram" notifications@github.com wrote:

They are essentially a collection of strings. Much like a dictionary. I wanted to use the parser to validate that all arguments were passed through since I do not want to have to worry about the logic for parsing it myself.

— Reply to this email directly or view it on GitHubhttps://github.com/gsscoder/commandline/issues/96#issuecomment-23432379 .

[ericnewton76]

Comment by issafram Wednesday Aug 28, 2013 at 18:52 GMT

---

Perhaps at a simpler level, would it perhaps be feasible to make a method that parses the arguments into a dictionary of elements? That would provide the value of the library without any strongly typed requirements.

[ericnewton76]

Comment by akfish Thursday Aug 29, 2013 at 03:44 GMT

---

Well. Just passing by. You can actually do this without requiring features from the library. You can:

1. Generate source code from your configuration file to define a 'static' Option class.
2. Compile it at run-time. A reference here Or you can use Emitter if you do not feel like to generate source code and compile.

[ericnewton76]

Comment by issafram Thursday Aug 29, 2013 at 04:16 GMT

---

That compiling at run-time example seemed strange. You have to basically generate a text file, then programmatically invoke the compiler at run-time, and finally Assembly.Load on the new assembly. This would have to be done on each run. It seems like it isn't an elegant solution.

I previously tried using an ExpandoObject but I couldn't find a way to add attributes which are required by the library for parsing. The Emitter seems like a very interesting idea. I will try that approach.

However, I still believe that the extra functionality I defined up above could have some use cases.

I'm thinking the method call could be something like

public Dictionary<string, string> ParseArguments(string[] args)

That allows the library to do the parsing and the user to do whatever they deem appropriate with the return value.

[ericnewton76]

Comment by akfish Thursday Aug 29, 2013 at 04:31 GMT

---

Glad to help. And FYI, you don't need to compile each time. It gives you an option to generate a .dll file on disk.

[ericnewton76]

Comment by nemec Thursday Aug 29, 2013 at 05:12 GMT

---

You can use Roslyn too if you want to avoid creating and compiling new files.

So basically what you're looking for is taking an argument string like "--name team --profession oranges --tagline go team" and turn it into three key-value pairs? I would think the effort needed to rework commandline to do that isn't worth the cost of just writing a parser.

Something like this would work for -- arguments:

var parts = "--name tim --occupation oranges --tagline cool things".Split();
var elems = new Dictionary<string, string>();

var en = parts.AsEnumerable().GetEnumerator();
en.MoveNext();
var key = en.Current;

while(en.MoveNext())
{
    var builder = new StringBuilder();
    do
    {
        if(en.Current.StartsWith("-"))
        {
            break;
        }
        builder.Append(en.Current);
        builder.Append(" ");
    } while(en.MoveNext());
    elems[key] = builder.ToString();

    try
    {
        key = en.Current;
    }
    catch(InvalidOperationException)
    {
        break;
    }
}

[ericnewton76]

Comment by issafram Thursday Aug 29, 2013 at 05:21 GMT

---

At a very basic level, yes I agree. I thought some of the value from this library was handling all the logic for parsing from a command line. Maybe I am confused, but I thought a user didn't have to use dashes, or they could use dashes, or they could do something like -keyValue with no space in between. All of the logic for the actual parsing I did not want to recreate as much time has been spent on it already for this library.

But if you recommend just doing a simple split and looping through the values, then perhaps that is a route to take. Roslyn seems interesting as well. I will try to see if I can dynamically create a Options class using Roslyn.

In totality, there are workarounds to get to the end goal. Perhaps I was just looking for some flexibility. This isn't a high priority, but something to consider.

[ericnewton76]

Comment by nemec Thursday Aug 29, 2013 at 05:42 GMT

---

By "not using dashes" you're referring to positional arguments, right? How would those get parsed into a dictionary since they have no key?

You're right about the others, those would have to be coded too.

The biggest issue here is that all of this parsing is predicated upon having a spec of what is/is not a valid parameter name.

You may be able to hook into the Tokenizer and TokenPartitioner to get the behavior you want (although you will probably need to join the Name and Value tokens together yourself).

[ericnewton76]

Comment by issafram Thursday Aug 29, 2013 at 05:53 GMT

---

I just think there are many possible variations, like so: app.exe arg1 value arg2 value app.exe arg1:value arg2:value app.exe arg1=value arg2=value app.exe arg1="value" arg2="value"

Some of these examples could be invalid input, but that just goes to show that the parsing logic should not be something the consumer (developer) should be trying to re-create.

If the spec is required for the parser, then I do believe there should be a way to define a spec at run-time. That has really been the scope of the conversation. I fully understand that you need a spec in order for the parser to behave and operate correctly, but requiring a class at design time seems like a limitation in my eyes.

[ericnewton76]

Comment by nemec Thursday Aug 29, 2013 at 06:17 GMT

---

Oh! I think in all of this discussion about Dictionaries and "dynamic" I forgot that you have a spec, it's just that the spec isn't known until runtime.

We've had a discussion before about adding a fluent builder to the library that would do exactly that. We had some ideas, but I don't think anything was ever implemented.

[ericnewton76]

Comment by issafram Thursday Aug 29, 2013 at 06:39 GMT

---

Yes I think we are on the same page now. There is a spec, but because it is configurable, the spec is not known until run-time.

A Fluent Builder sounds interesting. I always thought of fluent libraries as extra. I believe out of the box, the minimal functionality would be to just allow the option. Then fluent would be the next step.

So perhaps an overload for the ParseArguments method. Some ideas could be:

ParseArguments(string[] args, IEnumerable<Option> options) ParseArguments(string[] args, OptionSet optionSet)

There is already an OptionAttribute class that would be similar to how I am thinking the Option class would be structured. The difference is that these are not attributes. You basically want to be able to instantiate a new instance of an Option class on the fly and add it to an IEnumerable collection which would result in a spec that would be usable by the parser.

What do you think?

[ericnewton76]

Comment by nemec Friday Aug 30, 2013 at 03:59 GMT

---

Yeah, you're right. You'd probably want to extract most of the stuff in OptionAttribute into an interface that both OptionAttribute and Option will implement so that they'd be interchangeable when it comes to parsing.

[ericnewton76]

Comment by nemec Sunday Sep 01, 2013 at 08:28 GMT

---

@issafram Would you be willing to design a "superclass" that contains all possible arguments for any config value? I created my own command line parsing library a while ago in the style of Python's argparse and it has a fluent interface you might be able to use until something like this is merged into the library.

class ConditionalOptions
{
    public string Filename { get; set; }
    public string Url { get; set; }
}

var destinationFromConfig = "http";
var opt = new ConditionalOptions();
var parser = new CliParser<ConditionalOptions>(opt);

switch (destinationFromConfig)
{
    case "file":
        parser.HasNamedArgument(c => c.Filename)
                .WithShortName('f');
        break;
    //case "http":
    default:
        parser.HasNamedArgument(c => c.Url)
                .WithShortName('u');
        break;
}

parser.Parse(args);

If you try to use -u when destinationFromConfig is file, you'll get an "unknown parameter -u" error. It's not in a dictionary, though (I don't think that's possible without some major modifications to the library).

[ericnewton76]

Comment by issafram Monday Sep 02, 2013 at 01:01 GMT

---

@nemec the problem with that is you are assuming that I actually know the combination options at design time. We need to make it a little bit more flexible (my opinion).

I'm thinking you could perhaps do something similar to an OptionBuilder class (similar to StringBuilder). I'm thinking one of it's methods could be AddOption(Option option) which can be called at run-time.

[ericnewton76]

Comment by nemec Monday Sep 02, 2013 at 04:03 GMT

---

So you could do that to configure the parser, but how will you specify that the parsed output should be a Dictionary<string, string> and that the parser should use the dictionary indexer set method instead of PropertyInfo.Set like the existing one does? I did some experimenting, but both of our libraries require that the values being bound are properties and dictionary assignment is converted to a method call behind the scenes.

Also, my plan doesn't assume you know the combination at design time, it requires that you know the union of all possible configurations. For example, if config setting A adds a filename property and config setting B adds a timeoutproperty, the "master" config would contain both filename and timeout. If setting B isn't used at runtime, that value will be 0 (or null if you use a nullable) and the parser will not allow it to be specified on the command line (if you use the fluent interface).

If this still doesn't fix your issue, maybe it would be helpful if you gave a very specific scenario that you want covered.

[ericnewton76]

Comment by issafram Monday Sep 02, 2013 at 05:33 GMT

---

Let us pretend that there is a file called input.txt

public static void Main(string[] args)
{

OptionsBuilder optionsBuilder = new OptionsBuilder();

foreach (string line in File.ReadLines(@"input.txt"))
{
    optionsBuilder.AddOption(new Option(line, false));
}

using (var parser = new Parser())
{
    parser.ParseArguments(args, optionsBuilder);
}

}

Notice how as the developer I have no clue what is in the file input.txt I simply want to add each line (in this case assuming it is one word), as an option. I made up a constructor for the Option class where the first value is the string representation and the second one was whether or not it is required.

This is what I had in my mind. Keep in mind that this is just a high level prototype and the constructors/classes/design will need to be thought out more. What do you think?

[ericnewton76]

Comment by nemec Monday Sep 02, 2013 at 06:14 GMT

---

Ah, so the keys can potentially be any string at all? Makes sense. I'll take a look sometime and see if it's feasible to factor out the PropertyInfo into an interface so that I can build a backend for dict indexers...

Although the library still wants to know what type to convert the values into, so it'a pretty limited in what the output can be (all values will need to be of the same type). Not a problem for you, since you're using strings though.

[ericnewton76]

Comment by nemec Monday Sep 02, 2013 at 09:48 GMT

---

So, uh, it's possible but very brittle. If you stick to Dictionary<string, string> parsing you shouldn't have any problems though. To use it, check out the code on my dictbackend branch.

To use it:

var opt = new Dictionary<string, string>();
var parser = new CliParser<Dictionary<string, string>>(opt);
parser.HasNamedArgument(c => c["name"])
        .WithShortName('n');

parser.Parse("-n frank".Split());

Console.WriteLine("Parsed Keys:");
foreach (var kv in opt)
{
    Console.WriteLine("\t{0}: {1}", kv.Key, kv.Value);
}
// Parsed Keys:
//    name: frank

This is what you were looking for, right?

[ericnewton76]

Comment by issafram Sunday Sep 15, 2013 at 02:56 GMT

---

That branch does not exist. Also, how is CliParser related to the CommandLine library? Are they different code bases?

[ericnewton76]

Comment by nemec Sunday Sep 15, 2013 at 03:58 GMT

---

Oh yeah I merged it into master a week ago, so I deleted the branch. They're not related, other than the fact that they're both command line parsing libraries. I wanted to experiment with a slightly different configuration syntax than CommandLine, so I wrote my own.

[ericnewton76]

Comment by issafram Sunday Sep 15, 2013 at 22:03 GMT

---

Perhaps that is my confusion? I put this issue in for CommandLine. I am glad that you put in an enhancement in for your Clipr codebase though. I am sure that I can check it out via NuGet at some point.