Update of `Last update` value when subtasks are updated

[formacube]

Hi @jedie,

I realized that when tasks are running with subtasks, the Last update value is not updated when one of the sub-task is updated (as shown by the screenshot below).

Therefore, it is hard without opening the subtask to know if something has happened or not, or to get any sense of progress.

It would be great if the Last update value could also be updated each time one of the sub-task is updated, and not only when the main-task is updated (could be days in my case).

Thanks in advance

[jedie]

Good point...

I see two solutions:

A) every sub task will update the timestamp on main task B) the admin collects all timestamps and display the newest one

Think B is the best solution. Because A results in more write queries...

[formacube]

Totally agree with B probaly being the best solution

[formacube]

in addition, with B, we do not loose the information about the timestamp on last update of main task progression

[jedie]

I take a quick look into the code...

It seems that Option B is not a good idea: The change list order depends on main task "update_dt" timestamp. Think it's expensive to order the change list correctly...

Think there is another kind of a bug: A old main task will be sorted wrong, if sub task has a newer timestamp. Isn't it?

So if we implement Option A, than we also fix the wrong order of the changelist!

[jedie]

I take a deeper look: Think we already update the main task timestamp here:

https://github.com/boxine/django-huey-monitor/blob/f44622bf7ae7ddfb0af80f57243d55d21eb44386/huey_monitor/tqdm.py#L80-L96

Do you use ProcessInfo()and provide parent_task_id ?

[formacube]

I do use both ProcessInfo() and provide 'parent_task_id':

@huey.db_task(context=True)
def start_puzzle_research(puzzle_exploration_id=None, task=None):
      ...
      process_info = ProcessInfo(
              task,
              desc=task_name,
              total=task_length,  # info on task size
              unit= ' explored' if exploration.is_snake else ' pieces explored',
          )
      ...
      research_piece_depart(exploration, parent_task_id=task.id, parent_process_info = process_info)

and

@huey.db_task(context=True)
def research_piece_depart(exploration=None, task=None, parent_task_id=None, parent_process_info = None):
    ...
    if parent_task_id:
        TaskModel.objects.set_parent_task(
            main_task_id=parent_task_id,
            sub_task_id=task.id,
        )
    else:
        raise ValueError('missing parent_task_id')
    ...
    process_info = ProcessInfo(
        task,
        desc=task_name,
        total=task_length,  # info on task size - needed to enable progression display
        unit=' configurations'
    )
    ...
    # Update of task monitor:
    process_info.update(n=1)

[formacube]

I also checked that my huey_monitor code corresponds well with what you showed here.

[formacube]

OK, looking deeper into your and my code, I think I understood what is happening.

I use a parent_task_id on the sub-task (TaskModel) I don't use it on the ProcessInfo object of the sub-task (I didn't saw it I guess in the docs, and I didn't saw we could add one). I assumed that the info was directly taken from the sub-task itself.

What your code do as I understand it:

if self.parent_task_id:
    # Store information for main task, too:
    ids.append(self.parent_task_id)
    objects.append(
        TaskProgressModel(
            task_id=self.parent_task_id,
            progress_count=n,
            create_dt=now
        )
    )

is when parent_task_id is specified in the ProcessInfo object, it updates the parent task.

I would like the progress_count on the parent to stay the same. So I don't know about the part :

objects.append(
        TaskProgressModel(
            task_id=self.parent_task_id,
            progress_count=n,
            create_dt=now
        )
    )

I guess a new option could be easily added to decide if one wants to increase as well the main_task progress or not

[jedie]

Think the process count will be cumulated already. Think there is no need to change anything about this, isn't it?

If you find some information missing in the docs about all of this, just create a PR.

I will close this issues, now. Think there is no real problem here, or?

[formacube]

Yes. The process count will be cumulated ... which is a problem for a lot of applications of sub-tasks (such as mine).

That's why I suggest a new option

[jedie]

Hm. I have then not understand your problem with this. Can you explain a little bit more?

[formacube]

I'll implement directly and propose you the code

[formacube]

OK, I've submitted to you a PR with change + documentation

Best

[formacube]

FYI It is working well on my production system