By the semantics of the AXL API, AXL_Test returning AXL_SUCCESS is meant to indicate that calling AXL_Wait will not block (i.e., the transfer is complete, either with success or error, but it is not ongoing). For pthread, it turns out that we are returning AXL_SUCCESS before the transfer may actually be complete. An AXL user will then call AXL_Wait and block until the transfer finishes, so that the transfer does not really happen in the background.
One way to fix this would be to have each thread decrement a counter when it finishes (with a lock). The main thread could set that counter to the number of work items before starting any threads, and during pthread_test, the main thread could check whether the value of that counter has hit 0 to verify that all work items are done.
By the semantics of the AXL API, AXL_Test returning AXL_SUCCESS is meant to indicate that calling AXL_Wait will not block (i.e., the transfer is complete, either with success or error, but it is not ongoing). For pthread, it turns out that we are returning AXL_SUCCESS before the transfer may actually be complete. An AXL user will then call AXL_Wait and block until the transfer finishes, so that the transfer does not really happen in the background.
The condition in the
pdata->headline below checks whether all work items have been started, but it doesn't indicate whether all items have been completed: https://github.com/ECP-VeloC/AXL/blob/1090a6e2cde45af051f0d6ea90c9a4c40b733ba1/src/axl_pthread.c#L392-L404One way to fix this would be to have each thread decrement a counter when it finishes (with a lock). The main thread could set that counter to the number of work items before starting any threads, and during pthread_test, the main thread could check whether the value of that counter has hit 0 to verify that all work items are done.