search

icfnext / aem-groovy-console

The AEM Groovy Console provides an interface for running Groovy scripts in the AEM container. Scripts can be created to manipulate content in the JCR, call OSGi services, or execute arbitrary code using the CQ, Sling, or JCR APIs.
Other
159 stars 94 forks source link

Improvements for long-running scripts #114

Closed vcohen-irvinecompany closed 3 years ago

vcohen-irvinecompany commented 3 years ago

I have a few questions and feature requests but can't find a place to post them so creating an issue. Please let me know if you want to split this into multiple issues, or take the questions elsewhere.

I'm trying to solve timeout problems with long-running scripts. The Groovy Console only writes its output at the end of the request, so if it halts for any reason -- a back-end error, or an HTTP timeout -- I see no output at all so can't tell how much was processed successfully.

There are three areas (in no particular order) that could be improved to help with this problem: logging, streaming output, and script parameters.

Logging: I could work around this if I could log my progress, but have not been able to do so. I see the console exposes a log but there's no information on where it writes or how to configure it. My attempts at configuring it using the Sling Log services in the Felix console didn't work. How do I configure logging? It would be useful to have:

  1. clear documentation on the default setting, so I know without changing anything where it will attempt to write,
  2. detailed documentation on how to change the configuration. Pointing to the JavaDoc on the SL4J site is not useful, nor would another link to SL4J as it describes generic setup. AEM is a very particular environment; rather than making users research this and figure it out for themselves over and over, please provide and document a sensible default and how to change it.
  3. best would be a field in the Groovy Console's OSGi properties, so I can just specify a log file path (and perhaps a logging pattern) to override the default.

Streaming: It would be great if the output streamed back to the browser rather than writing at the end of the request. That would solve timeout problems and also let me monitor progress without having to configure and tail log files. print and println statements in the Groovy code would go to the HTTP response stream and appear in the output panel as they're written, rather than when the whole script completes.

Parameters: Finally, does the groovy console support querystring parameters, or any other way of passing data to the script via the URL? If I could do that then I could call the console in a loop from e.g. a shell script, providing different parameters on each run. I'd still need to guess at batch sizes that would not time out, but once that's done I could automate the batches.

Right now, without any of these facilities, I have to test my scripts and see where they fail, then guess at a lower limit and try again until they run successfully. Then I have to run in batches, manually updating the search offset on each run. Last night I spent nearly 4 hours running a script 200 times, each run taking about 40 seconds and processing 100 search results. A few enhancements would make working with batches much easier.

vcohen-irvinecompany commented 3 years ago

After I posted here I saw the notice that the repo had moved, so I opened the same issue there: https://github.com/CID15/aem-groovy-console/issues/8 Closing this one