Unable to add custom attribute to Celery trace

[schallis]

Description I've been attempting to add a custom attribute to my Celery traces as per the docs.

When using the following snippet (as described in Issue 215), I do NOT see the expected attribute:

class CustomBaseTask(Task):
    """
    Annotates the new relic trace with args and kwargs.
    """

    def __call__(self, *args, **kwargs):
        if newrelic.agent.current_transaction():
            newrelic.agent.add_custom_parameter('custom_parameter': 'value')

        return super().__call__(*args, **kwargs)

However I am able to get the custom attribute to appear when I use the following snippet, however this appears as a second trace so I have a trace without the attribute and one with the attribute which is confusing:

class CustomBaseTask(Task):
    """
    Annotates the new relic trace with args and kwargs.
    """

    def __call__(self, *args, **kwargs):
        application = newrelic.agent.application()
        with newrelic.agent.BackgroundTask(application, name=self.name, group='Celery'):
            newrelic.agent.add_custom_parameter(
                    'custom_parameter',
                    'value',
                )

        return super().__call__(*args, **kwargs)

newrelic.agent.add_custom_parameter also works perfectly in my Web transactions.

Expected Behavior Under Traces > Trace Groups > Trace Name > Attributes for my instrumented non-web transactions I expect to see my custom attribute appear but it does not with the first example, and for the second example it seems to create a new trace instead of modifying the existing one resulting in duplication.

Steps to Reproduce See custom Celery Task class above

Your Environment newrelic==8.5.0 python 3.9.6

Additional context N/A

[workato-integration[bot]]

https://new-relic.atlassian.net/browse/NR-332298