Improvements for long-running scripts

[vcohen-irvinecompany]

I have a few questions and feature requests but can't find a place to post them so creating an issue. Please let me know if you want to split this into multiple issues, or take the questions elsewhere. (Sorry for the cross-post; I posted this in the old icfnext repo and only now saw the noticed that this one is current.)

I'm trying to solve timeout problems with long-running scripts. The Groovy Console only writes its output at the end of the request, so if it halts for any reason -- a back-end error, or an HTTP timeout -- I see no output at all so can't tell how much was processed successfully.

There are three areas (in no particular order) that could be improved to help with this problem: logging, streaming output, and script parameters.

Logging: I could work around this if I could log my progress, but have not been able to do so. I see the console exposes a log but there's no information on where it writes or how to configure it. My attempts at configuring it using the Sling Log services in the Felix console didn't work. How do I configure logging? It would be useful to have:

1. clear documentation on the default setting, so I know without changing anything where it will attempt to write,
2. detailed documentation on how to change the configuration. Pointing to the JavaDoc on the SL4J site is not useful, nor would another link to SL4J as it describes generic setup. AEM is a very particular environment; rather than making users research this and figure it out for themselves over and over, please provide and document a sensible default and how to change it.
3. best would be a field in the Groovy Console's OSGi properties, so I can just specify a log file path (and perhaps a logging pattern) to override the default.

Streaming: It would be great if the output streamed back to the browser rather than writing at the end of the request. That would solve timeout problems and also let me monitor progress without having to configure and tail log files. print and println statements in the Groovy code would go to the HTTP response stream and appear in the output panel as they're written, rather than when the whole script completes.

Parameters: Finally, does the groovy console support querystring parameters, or any other way of passing data to the script via the URL? If I could do that then I could call the console in a loop from e.g. a shell script, providing different parameters on each run. I'd still need to guess at batch sizes that would not time out, but once that's done I could automate the batches.

Right now, without any of these facilities, I have to test my scripts and see where they fail, then guess at a lower limit and try again until they run successfully. Then I have to run in batches, manually updating the search offset on each run. Recently I spent nearly 4 hours running a script 200 times, each run taking about 40 seconds and processing 100 search results, then manually changing some params and running again. A few enhancements would make working with batches much easier.

[Jakub-Izbicki]

Groovyconsole does support input parameters. There is an injected data variable that is being populated with data passed to groovyconsole when executing a script.

This data can be typed in on one of the accordion sections on groovyconsle web UI, or it can be passed as part of form data params when using http api directly.

[vcohen-irvinecompany]

Hi @Jakub-Izbicki Thanks for the info. I'm familiar with the data section -- can you point to documentation about (or just tell me) specifically how to pass the data param over HTTP? What's the param key? I'd prefer not to guess.