feat: inserting and running sub-tasks dynamically

[privatenumber]

Problem

I'm creating a CLI to collect benchmarks, and I want to run a set of tasks n times. However, I don't want to create a new task for each one as n could be as high as 100.

Instead, I want to use one task that repeats it's sub-task n times, and updates the output message to Iteration ${i}.

Feature request

An API like this where I can insert/run arbitrary Listr instances dynamically at run-time:

const listr = new Listr<Options>([
    {
        title: 'Benchmarking',
        async task(context, task) {
            for (let i = 0; i < 10; i++) {
                task.output = `Iteration ${i}`;

                // Run arbitrary listr instances as a sub-task
                await task.run(task.newListr([
                    {
                        title: 'Task A',
                        task() { ... }
                    },
                    {
                        title: 'Task B',
                        task() { ... }
                    },
                    {
                        title: 'Task C',
                        task() { ... }
                    }
                ]));
            }
        }
    },

    ...
]);

[cenk1cenk2]

Yes you can do this already. Its not per say dynamic but works like it. The key is returning a subtask mapped inside a parent task. Something like this https://github.com/cenk1cenk2/servicecmd/blob/5d13c9616ca70753f7b9d458f10f1fdb75631b1a/packages/cli/src/commands/docker/index.ts#L235 . Sorry for not complete mobile answer but tell me if this works in your case.

I guess you can create a subtask for each iteration and in that subtask you can even create more subtasks or something like that.

[privatenumber]

Returning a subtask is what I'm trying to avoid because the list of tasks in the subtask could be very large.

Correct me if I'm wrong, but as I understand, this is what you're suggesting right?

const listr = new Listr<Options>([
    {
        title: 'Benchmarking',
        task(context, task) {

            return task.newListr(
                Array.from({ length: 100 }, (_, iteration) => ({
                    title: `Iteration ${iteration}`,
                    async task() { ... },
                }))
            );
        },
    },

    ...,
]);

This makes my output look like this:

Instead, I'd like to repeat a subtask 100 times and update the parent title to say "This is the n th iteration".

[cenk1cenk2]

I guess what you can do this not give it a task title and use the parent output or title through the subtasks. So subtasks are not actually rendered, how do you think of that approach?

const listr = new Listr<Options>([
    {
        title: 'Benchmarking',
        task(context, task) {

            return task.newListr((parent) => {
                Array.from({ length: 100 }, (_, iteration) => ({
                    async task() { 
                                            parent.output = `Iteration ${iteration}`
                                        },
                }))
            );
        },
    },

    ...
]);

[privatenumber]

Close, but I would like to run a subtask for each iteration.

The closest I have been able to get to that behavior is with faking retries:

const listr = new Listr<Options>([
    {
        title: 'Benchmarking',
        async task(context, task) {

            return task.newListr([
                {
                    title: 'Repeat task',
                    async task() {
                        ...

                        throw new Error('fake error to re-run');
                    },
                    retry: 100,
                }
            ], {
                rendererOptions: {
                    suffixRetries: true
                }
            });
        },
    },

    ...
]);

I will try to get a working visual output with this code to further demonstrate my end goal.

[cenk1cenk2]

You might be able to set no title to the subtasks then set the title dynamically, then set it to null while the task finishes so they can disappear from the task list for the iteration case.

[cenk1cenk2]

I can add a flag like rerun that replicates the behavior of retry without the need for failing tomorrow if none of this gives you what you need.

[privatenumber]

Looks like the fake retry approach doesn't work either because retries don't render subtasks (and there doesn't seem to be an option to show it).

I'm able to accomplish something close with this, but the output is completely corrupted because the renderer isn't inherited:

const listr = new Listr<Options>([
    {
        title: 'Benchmarking',
        async task(context, task) {
            for (let i = 0; i < 100; i++) {
                task.title = `Benchmarking (${i})`;

                const listr = task.newListr(context.artifacts.map(
                    artifact => ({
                        title: artifact.moduleName,
                        task: () => getAllBenchmarks(artifact.modulePath),
                    }),
                ));

                const { results } = await listr.run({
                    results: [],
                });
            }
        },
    },

    ...
]);

---

I looked into the source code a bit, and I'm wondering if it's possible to call something like this.listr.runTask() to run arbitrary tasks inline. This doesn't work right now, but I'm trying something like this:

const listr = new Listr<Options>([
    {
        title: 'Benchmarking',
        async task(context, task) {
            const taskInstance = new Task(this.listr, {
                title: 'Some arbitrary task',
                async task() {
                    await sleep(10000);
                },
            }, listr.options, {
                ...listr.rendererClassOptions,
                ...this.options,
            });

            await this.listr.runTask(taskInstance);
        },
    },

    ...
]);

A rerun option would work but sounds specific to my use-case. A method that can be called with an arbitrary task would be more accommodating of more use-cases I think.

[cenk1cenk2]

No, it is actually not possible to run two listr at the same time because the problem is log-update takes control of the stdout and rest of the subtasks are always silent, so only the parent tasks get rendered. Elsewise the stdout gets corrupted as you say two instances taking control of it.

Oh, because in retries it won't render the successful tasks that run before I suppose, I have to reset them to it should be a bug.

I will have a proper look at this tomorrow since it is kind of getting late here. Will prompt you how we can proceed if all is okay for you. But in the worst-case scenario we can have the rerun flag. But I think that this should somehow be possible to create something like this with the current form with some trickery.

[privatenumber]

I did a little more exploration and I decided to take another approach.

I withdraw my feature request but feel free to keep it open if you want to accommodate the potential use-case.

Thanks so much for your fast responses and support.

[cenk1cenk2]

Thank you for the kind words, wish I could be more helpful having access to a computer at the time to prototype it. Have a fun day.