[BUG] SMP mode - vTaskCoreAffinitySet does not yield

[WojciechJasko-BB]

Describe the bug vTaskCoreAffinitySet does not yield when the modified task is in a ready state but not executing.

Scenario: FreeRTOS is running on 4 cores. Core 0 is executing:

for (size_t i = 0u; i < 4U; ++i)
    {
        // TaskHandle_t handle = xTaskCreateStaticAffinitySet(
        //     core_task, "core", MAIN_TASK_SIZE, NULL, MAIN_TASK_PRI, gCoreTaskStack[i], &gCoreTaskObj[i], (1 << i));
        // configASSERT(handle != NULL);
        TaskHandle_t handle = xTaskCreateStatic(
            core_task, "core", MAIN_TASK_SIZE, NULL, MAIN_TASK_PRI, gCoreTaskStack[i], &gCoreTaskObj[i]);
        configASSERT(handle != NULL);

        vTaskCoreAffinitySet(handle, (1 << i));
    }

No other tasks are running.

Outcome: Sometimes one task is not executed (5% times), but it is on ReadyList.

From the code instrumentation, I see that:

• when the core is executed then if( taskTASK_IS_RUNNING( pxTCB ) == pdTRUE ) was true
• when the core is not executing then both if( taskTASK_IS_RUNNING( pxTCB ) == pdTRUE ) and if( ( uxPrevNotAllowedCores & uxCoreAffinityMask ) != 0U ) were false and nobody yield assigned core.

It makes sense because:

• task was not running (it was in ready state)
• we did not added a new core

In our case, there was no other job for this core.

if( ( uxPrevNotAllowedCores & uxCoreAffinityMask ) != 0U ) is wrong, because it assumes that the task was blocked (the ready state was forgotten).

[chinglee-iot]

@WojciechJasko-BB The tasks created by xTaskCreateStatic will be scheduled to run when the following conditions are met:

• The task's core affinity settings permit it to run on the core ( default no core affinity )
• The task has a higher priority than the currently running task on the core.
• configUSE_PREEMPTION is enabled.

Restricting the core affinity of a task in the ready state does not alter its state. This is because the task should be able to run already with it's original core affinity mask. Therefore, yield for the task is only required when new core is added to it's core affinity mask.

I use the following code for explanation:

for (size_t i = 0u; i < 4U; ++i)
{
    /* By default, task is created with no core affinity. Task 0 is able to run on core 0, 1, 2, 3.
     * So does task 1, 2 and 3. The only reason that the task is not running in this situation is that
     * it doesn't have higher priority than any current running task.
     * Let's assume that task 0, 1, 2 are running and task 3 is in ready state due to some higher priority
     * task is running on core 3. */
    TaskHandle_t handle = xTaskCreateStatic( core_task, 
                                             "core",
                                             MAIN_TASK_SIZE,
                                             NULL,
                                             MAIN_TASK_PRI,
                                             gCoreTaskStack[i],
                                             &gCoreTaskObj[i]);

    /* Setting core affinity of task 3 to core 3 only doesn't change it's state cause the same reason that
     * core 3 is running a task with higher priority than task 3. Therefore, this task can't be preempted by task 3. */
    vTaskCoreAffinitySet(handle, (1 << i));
}

We will need your help to provide the following information to investigate this problem.

• Is timer task enabled ( configUSE_TIMERS )? What is the priority of the timer task?
• What is the current running task on the core when the created task is in ready state?

[WojciechJasko-BB]

@chinglee-iot Is timer task enabled ( configUSE_TIMERS )? What is the priority of the timer task? Yes, timers are enabled but not used. Timer task has piority configMAX_PRIORITIES - 1. It is the same priority as newly created task.

What is the current running task on the core when the created task is in ready state? There is no other task that can be run on that core.

There are 6 task started by user code (excluding timer task). 4x tasks that are assigned per core for counters incrementation. 1 task for printing counters values assigned to core 0. 1 task for serving uart assigned to core 0.

These 4 tasks are created after start of scheduler. Other 2 before.

In 5% cases, one counter is not incrementing. When I check it with debugger, it is on ready list. I also instrumented portYIELD_CORE and it is not called for specific core.

[chinglee-iot]

Thank you for your reply. From your information, it looks like that the task is in ready state and never get scheduled to run in 5% cases and the scheduler doesn't request the core to yield for this task.

I will need more information from you to investigate this problem. Assuming that your task in ready state is supposed to run on core x. Can you help to provide pxCurrentTCBs[ x ] information with your debugger? If you are using GDB, you can just print the *pxCurrentTCBs[ x ] to dump the fields in TCB_t.

Example dump:

(gdb) print *pxCurrentTCBs[0] $1 = {pxTopOfStack = 0x5555555e6390, uxCoreAffinityMask = 18446744073709551615, xStateListItem = {xItemValue = 0, pxNext = 0x5555555e64e0, pxPrevious = 0x555555596140 <pxReadyTasksLists+96>, pvOwner = 0x5555555e6280, pvContainer = 0x555555596130 <pxReadyTasksLists+80>}, xEventListItem = {xItemValue = 5, pxNext = 0x0, pxPrevious = 0x0, pvOwner = 0x5555555e6280, pvContainer = 0x0}, uxPriority = 2, pxStack = 0x5555555e6390, xTaskRunState = 0, uxTaskAttributes = 0, pcTaskName = "SMP Task\000\000\000", xPreemptionDisable = 0, pxEndOfStack = 0x5555555e64a4, uxCriticalNesting = 0, uxTCBNumber = 1, uxTaskNumber = 0, uxBasePriority = 2, uxMutexesHeld = 0, pxTaskTag = 0x0, ulNotifiedValue = {0, 0, 0, 0, 0}, ucNotifyState = "\000\000\000\000", ucDelayAborted = 0 '\000'}