Ability to get and set maxBytes param using DROID CLI

[Dclipsham]

Request via Google Group - https://groups.google.com/forum/#!topic/droid-list/YiV7NQE2olk

[nishihatapalmer]

This is really wider than just max bytes to scan. We really need to be able to pass any configurable profile property into the CLI.

At present, we have to edit the profile properties in the GUI (not workable if you want to script different things) or use scripting hacks to edit the properties file just to control what the CLI does.

[nishihatapalmer]

A simple method might be to offer a command line option that allows a bunch of properties to be set, using some kind of delimited string. It might be a bit geeky vs. having dedicated command line options for each setting, but it would provide a default capability for any configurable property.

Essentially you just provide what you need to override in the properties file as a string argument to the command line.

Or we could have an option giving a path to a profile property file you want to use.

[nishihatapalmer]

I'm having the issue of injecting runtime configuration into a new profile in some other work I'm doing.

When DROID initialises a profile, it reads configurable properties from the global config from the properties file in the profile working directory. This is hard coded. I guess we need to pass in additional overrides when we assemble the properties somehow.

[nishihatapalmer]

Right - I think the ProfileManager interface needs a new create method which takes sigs AND a set of properties to override any default profile properties. The current method only lets you create a profile with sigs passed in (so all runtime configuration comes from default properties).

[nishihatapalmer]

I have a branch on my own fork which allows any profile properties to be set, either directly on the command line, or by specifying an alternate property file to use.

Still needs more testing, but can be seen here: https://github.com/nishihatapalmer/droid/tree/ProfileProperties

To override a profile property such as max bytes to scan, you specify it like this:

droid -a "/home/user/Documents" -R -p "/home/user/docprofile.droid" -Pr "profile.maxBytesToScan=1024"

You can set more than one property by just specifying each as additional arguments:

droid -a "/home/user/Documents" -R -p "/home/user/docprofile.droid" -Pr "profile.maxBytesToScan=1024" "profile.processZip=false"

If you have a lot of properties you want to override all the time, or different sets of properties you want to re-use, you can put them in a property file and tell DROID to use that:

droid -a "/home/user/Documents" -R -p "/home/user/docprofile.droid" -Pf "/home/user/my.properties"

You can also specify properties on the command line with -Pr and use a property file with -Pf. In this case, any properties specified directly on the command line take precedence over the ones in the file (which in turn has precedence over properties defined in the droid global property file).

droid -a "/home/user/Documents" -R -p "/home/user/docprofile.droid" -Pr "profile.maxBytesToScan=1024" "profile.processZip=false" -Pf "/home/user/my.properties"

To see all the profile properties that can be set, look at the droid.properties file in the droid home folder. This is normally located in the .droid6 folder under the user's home folder.

[nishihatapalmer]

One slightly annoying limitation of the current code is that a few properties are not used when a profile is created, but are rather used when it is opened (and taken from the global property file at that point).

All the normal profile properties like whether it processes archive files, or max bytes to scan can be set on creation. These properties tend to be prefixed by profile. But there are other properties defined in the global property file too, like database.durability. These will still be taken from the global property file when the profile is actually opened, so they can't currently be overridden.

To fix that, we'd need a profile to remember all the properties it is given when created (not just the ones that directly affect the profile creation), so that they could be re-used when it is opened. This is harder than it first seems - I've experimented with this and it runs into some tricky issues, for example, how persistent those profile settings are.

[nishihatapalmer]

I've submitted a PR here with the ability to set profile properties (but not to get them so far).

https://github.com/digital-preservation/droid/pull/407

[nishihatapalmer]

Propose this issue is closed - PR above allows setting max bytes. Is there any demand for doing a "get"? Which would only return what the profile default setting is (you can just look in the file or run DROID GUI and look at the default properties sheet).

[OliverHannan]

After investigation PR above https://github.com/digital-preservation/droid/pull/407 allows for original issue.