OCA / queue

Asynchronous Job Queue
GNU Affero General Public License v3.0
175 stars 451 forks source link

[15.0] Queue job in chain does not see changes from previous job #614

Closed oerp-odoo closed 1 month ago

oerp-odoo commented 8 months ago

Module

queue_job

Describe the bug

I am using queue job that consists of a chain having inside single jobs and group of jobs.

So my chain looks like this: Job1 -> Job2 -> JobsGroup -> Job4

When Job2 completes, it does some changes in database, where JobsGroup depends on. Every time I run JobsGroup, it sees outdated data as if Job2 changes did not happen. Though after all jobs are complete, I can see that changes did happen.

At first I thought that maybe they were run in parallel (as Job2 completed very fast). But when I added sleep time on Job2 I can clearly see that JobsGroup waits for Job2 to complete.

As a side note, I use these jobs with odoo import functionality. So maybe it also has something do to with it (as odoo uses savepoints and so on)? (Job2 imports/updates record that JobsGroup depend on and then JobsGroup imports child records after waiting for parent record import/update. Basically Job2 is parent record and JobsGroup consists of child records with one2many relationship).

Though if I change logic, to only run JobsGroup -> Job4, then it properly sees changes that happened before.

To Reproduce

Affected versions: 15.0

Here is snippet of my logic, how I create delayables and how jobs are started.

    def _import_file(
        self,
        date_start,
        dryrun=False,
    ):
        """Extend to use queue jobs for importing."""
        if not self.is_delayed_import():
            return super()._import_file(
                date_start,
                dryrun=dryrun,
                date_start=date_start,
            )
        pricing_id = self.pricing_id.id
        delayables_list = [
            self.delayable(
                description=get_preprocess_import_file_description(dryrun, pricing_id)
            ).preprocess_import_file(),
            # This is Job2
            self.delayable(
                description=get_pricing_import_description(dryrun, self.pricing_id.id)
            )._import_pricing(dryrun=dryrun),
        ]
        lanes_delayables = self._import_lanes_delayed(dryrun=dryrun)
        # This is JobsGroup
        lanes_delayable_group = group(*lanes_delayables)
        delayables_list.append(lanes_delayable_group)
        delayables_list.append(
            self.delayable(
                description=get_postprocess_import_file_description(dryrun, pricing_id)
            ).postprocess_import_file(
                # Placeholders for pricing/lanes results, because at
                # this point we don't know as jobs have not run yet.
                None,
                None,
                date_start,
                dryrun=dryrun,
            )
        )
        chain(*delayables_list).delay()

Expected behavior I would expect other jobs that are run in chain, to see changes that happened before them.

Additional context Add any other context about the problem here. (e.g. OS, Python version, ...)

github-actions[bot] commented 2 months ago

There hasn't been any activity on this issue in the past 6 months, so it has been marked as stale and it will be closed automatically if no further activity occurs in the next 30 days. If you want this issue to never become stale, please ask a PSC member to apply the "no stale" label.