DynamoRIO / dynamorio

Dynamic Instrumentation Tool Platform
Other
2.63k stars 557 forks source link

tool.drcov.eintr failing on ubuntu-20-arm64-sve #6761

Closed abhinav92003 closed 6 months ago

abhinav92003 commented 6 months ago

The tool.drcov.eintr test seems to have started failing on ubuntu-20-arm64-sve. This was observed on #6756, #6757, and #6746.

The test logs mention an output mismatch, but I don't see it.

2024-04-04T21:14:23.5101279Z 206: CMake Error at /opt/actions-runner/_work/dynamorio/dynamorio/clients/drcov/runtest.cmake:95 (message):
2024-04-04T21:14:23.5102190Z 206:   tool output
2024-04-04T21:14:23.5102819Z 206:   SF:/opt/actions-runner/_work/dynamorio/dynamorio/suite/tests/linux/eintr.c
2024-04-04T21:14:23.5103404Z 206: 
2024-04-04T21:14:23.5103627Z 206:   DA:52,1
2024-04-04T21:14:23.5103973Z 206: 
2024-04-04T21:14:23.5104272Z 206:   DA:53,1
2024-04-04T21:14:23.5112911Z 206: 
2024-04-04T21:14:23.5113309Z 206:   DA:56,1
2024-04-04T21:14:23.5113565Z 206: 
2024-04-04T21:14:23.5113778Z 206:   DA:57,1
2024-04-04T21:14:23.5114021Z 206: 
2024-04-04T21:14:23.5114233Z 206:   DA:58,1
2024-04-04T21:14:23.5114469Z 206: 
2024-04-04T21:14:23.5114684Z 206:   DA:59,1
2024-04-04T21:14:23.5114917Z 206: 
2024-04-04T21:14:23.5115152Z 206:   DA:60,1
2024-04-04T21:14:23.5115389Z 206: 
2024-04-04T21:14:23.5115602Z 206:   DA:64,1
2024-04-04T21:14:23.5115840Z 206: 
2024-04-04T21:14:23.5116046Z 206:   DA:65,1
2024-04-04T21:14:23.5116283Z 206: 
2024-04-04T21:14:23.5116502Z 206:   DA:67,1
2024-04-04T21:14:23.5116738Z 206: 
2024-04-04T21:14:23.5116956Z 206:   DA:69,1
2024-04-04T21:14:23.5117190Z 206: 
2024-04-04T21:14:23.5117402Z 206:   DA:70,1
2024-04-04T21:14:23.5117641Z 206: 
2024-04-04T21:14:23.5117852Z 206:   DA:71,1
2024-04-04T21:14:23.5118086Z 206: 
2024-04-04T21:14:23.5118305Z 206:   DA:72,1
2024-04-04T21:14:23.5118540Z 206: 
2024-04-04T21:14:23.5118752Z 206:   DA:74,1
2024-04-04T21:14:23.5118988Z 206: 
2024-04-04T21:14:23.5119197Z 206:   DA:75,1
2024-04-04T21:14:23.5119435Z 206: 
2024-04-04T21:14:23.5119650Z 206:   DA:76,0
2024-04-04T21:14:23.5119887Z 206: 
2024-04-04T21:14:23.5120103Z 206:   DA:78,1
2024-04-04T21:14:23.5120335Z 206: 
2024-04-04T21:14:23.5120549Z 206:   DA:79,1
2024-04-04T21:14:23.5120783Z 206: 
2024-04-04T21:14:23.5121010Z 206:   DA:80,1
2024-04-04T21:14:23.5121250Z 206: 
2024-04-04T21:14:23.5121818Z 206:   DA:81,1
2024-04-04T21:14:23.5122086Z 206: 
2024-04-04T21:14:23.5122301Z 206:   DA:85,1
2024-04-04T21:14:23.5122534Z 206: 
2024-04-04T21:14:23.5122749Z 206:   DA:90,1
2024-04-04T21:14:23.5122986Z 206: 
2024-04-04T21:14:23.5123198Z 206:   DA:91,0
2024-04-04T21:14:23.5123432Z 206: 
2024-04-04T21:14:23.5123643Z 206:   DA:92,0
2024-04-04T21:14:23.5123876Z 206: 
2024-04-04T21:14:23.5124086Z 206:   DA:95,1
2024-04-04T21:14:23.5124321Z 206: 
2024-04-04T21:14:23.5124924Z 206:   DA:96,1
2024-04-04T21:14:23.5125168Z 206: 
2024-04-04T21:14:23.5125376Z 206:   DA:98,1
2024-04-04T21:14:23.5125612Z 206: 
2024-04-04T21:14:23.5125822Z 206:   DA:99,0
2024-04-04T21:14:23.5126058Z 206: 
2024-04-04T21:14:23.5126272Z 206:   DA:100,0
2024-04-04T21:14:23.5126513Z 206: 
2024-04-04T21:14:23.5126726Z 206:   DA:103,1
2024-04-04T21:14:23.5126968Z 206: 
2024-04-04T21:14:23.5127181Z 206:   DA:104,1
2024-04-04T21:14:23.5127428Z 206: 
2024-04-04T21:14:23.5127752Z 206:   DA:105,1
2024-04-04T21:14:23.5127996Z 206: 
2024-04-04T21:14:23.5128210Z 206:   DA:106,1
2024-04-04T21:14:23.5128451Z 206: 
2024-04-04T21:14:23.5128661Z 206:   DA:107,1
2024-04-04T21:14:23.5128909Z 206: 
2024-04-04T21:14:23.5129124Z 206:   DA:114,1
2024-04-04T21:14:23.5129367Z 206: 
2024-04-04T21:14:23.5129579Z 206:   DA:115,1
2024-04-04T21:14:23.5129814Z 206: 
2024-04-04T21:14:23.5130027Z 206:   DA:118,1
2024-04-04T21:14:23.5130267Z 206: 
2024-04-04T21:14:23.5130481Z 206:   DA:121,1
2024-04-04T21:14:23.5130724Z 206: 
2024-04-04T21:14:23.5130936Z 206:   DA:122,1
2024-04-04T21:14:23.5131179Z 206: 
2024-04-04T21:14:23.5131390Z 206:   DA:124,1
2024-04-04T21:14:23.5131633Z 206: 
2024-04-04T21:14:23.5131845Z 206:   DA:125,1
2024-04-04T21:14:23.5132086Z 206: 
2024-04-04T21:14:23.5132299Z 206:   DA:126,0
2024-04-04T21:14:23.5132539Z 206: 
2024-04-04T21:14:23.5132753Z 206:   DA:127,1
2024-04-04T21:14:23.5132998Z 206: 
2024-04-04T21:14:23.5133210Z 206:   DA:128,1
2024-04-04T21:14:23.5133606Z 206: 
2024-04-04T21:14:23.5133831Z 206:   DA:130,1
2024-04-04T21:14:23.5134073Z 206: 
2024-04-04T21:14:23.5134285Z 206:   DA:131,0
2024-04-04T21:14:23.5134528Z 206: 
2024-04-04T21:14:23.5134738Z 206:   DA:133,1
2024-04-04T21:14:23.5135139Z 206: 
2024-04-04T21:14:23.5135354Z 206:   DA:134,0
2024-04-04T21:14:23.5135592Z 206: 
2024-04-04T21:14:23.5135805Z 206:   DA:136,1
2024-04-04T21:14:23.5136046Z 206: 
2024-04-04T21:14:23.5136259Z 206:   DA:137,1
2024-04-04T21:14:23.5136500Z 206: 
2024-04-04T21:14:23.5136707Z 206:   DA:139,1
2024-04-04T21:14:23.5136951Z 206: 
2024-04-04T21:14:23.5137162Z 206:   DA:140,1
2024-04-04T21:14:23.5137401Z 206: 
2024-04-04T21:14:23.5137614Z 206:   DA:142,1
2024-04-04T21:14:23.5137850Z 206: 
2024-04-04T21:14:23.5138061Z 206:   DA:143,1
2024-04-04T21:14:23.5138302Z 206: 
2024-04-04T21:14:23.5138523Z 206:   end_of_record
2024-04-04T21:14:23.5138790Z 206: 
2024-04-04T21:14:23.5139068Z 206:    failed to match expected DA:52,1
2024-04-04T21:14:23.5139444Z 206: 
2024-04-04T21:14:23.5139665Z 206:   DA:53,1
2024-04-04T21:14:23.5139906Z 206: 
2024-04-04T21:14:23.5140115Z 206:   DA:56,1
2024-04-04T21:14:23.5140355Z 206: 
2024-04-04T21:14:23.5140566Z 206:   DA:57,1
2024-04-04T21:14:23.5140804Z 206: 
2024-04-04T21:14:23.5141020Z 206:   DA:58,1
2024-04-04T21:14:23.5141255Z 206: 
2024-04-04T21:14:23.5141467Z 206:   DA:59,1
2024-04-04T21:14:23.5141705Z 206: 
2024-04-04T21:14:23.5141919Z 206:   DA:60,1
2024-04-04T21:14:23.5142155Z 206: 
2024-04-04T21:14:23.5142366Z 206:   DA:64,1
2024-04-04T21:14:23.5142606Z 206: 
2024-04-04T21:14:23.5142822Z 206:   DA:65,1
2024-04-04T21:14:23.5143060Z 206: 
2024-04-04T21:14:23.5143276Z 206:   DA:67,1
2024-04-04T21:14:23.5143511Z 206: 
2024-04-04T21:14:23.5143725Z 206:   DA:69,1
2024-04-04T21:14:23.5143962Z 206: 
2024-04-04T21:14:23.5144173Z 206:   DA:70,1
2024-04-04T21:14:23.5144410Z 206: 
2024-04-04T21:14:23.5144625Z 206:   DA:71,1
2024-04-04T21:14:23.5144864Z 206: 
2024-04-04T21:14:23.5145077Z 206:   DA:72,1
2024-04-04T21:14:23.5145315Z 206: 
2024-04-04T21:14:23.5145531Z 206:   DA:74,1
2024-04-04T21:14:23.5145772Z 206: 
2024-04-04T21:14:23.5145987Z 206:   DA:75,1
2024-04-04T21:14:23.5146223Z 206: 
2024-04-04T21:14:23.5146436Z 206:   DA:76,0
2024-04-04T21:14:23.5146669Z 206: 
2024-04-04T21:14:23.5146885Z 206:   DA:78,1
2024-04-04T21:14:23.5147120Z 206: 
2024-04-04T21:14:23.5147332Z 206:   DA:79,1
2024-04-04T21:14:23.5147570Z 206: 
2024-04-04T21:14:23.5147777Z 206:   DA:80,1
2024-04-04T21:14:23.5148013Z 206: 
2024-04-04T21:14:23.5148228Z 206:   DA:81,1
2024-04-04T21:14:23.5148465Z 206: 
2024-04-04T21:14:23.5148931Z 206:   DA:85,1
2024-04-04T21:14:23.5149177Z 206: 
2024-04-04T21:14:23.5149390Z 206:   DA:90,1
2024-04-04T21:14:23.5149629Z 206: 
2024-04-04T21:14:23.5149843Z 206:   DA:91,0
2024-04-04T21:14:23.5150074Z 206: 
2024-04-04T21:14:23.5150285Z 206:   DA:92,0
2024-04-04T21:14:23.5150522Z 206: 
2024-04-04T21:14:23.5150736Z 206:   DA:95,1
2024-04-04T21:14:23.5150973Z 206: 
2024-04-04T21:14:23.5151182Z 206:   DA:96,1
2024-04-04T21:14:23.5151417Z 206: 
2024-04-04T21:14:23.5151747Z 206:   DA:98,1
2024-04-04T21:14:23.5151984Z 206: 
2024-04-04T21:14:25.4880740Z 206:   DA:99,0
2024-04-04T21:14:25.4881850Z 206: 
2024-04-04T21:14:25.4882489Z 206:   DA:100,0
2024-04-04T21:14:25.4882803Z 206: 
2024-04-04T21:14:25.4883057Z 206:   DA:103,1
2024-04-04T21:14:25.4883305Z 206: 
2024-04-04T21:14:25.4883524Z 206:   DA:104,1
2024-04-04T21:14:25.4883772Z 206: 
2024-04-04T21:14:25.4883990Z 206:   DA:105,1
2024-04-04T21:14:25.4884235Z 206: 
2024-04-04T21:14:25.4884447Z 206:   DA:106,1
2024-04-04T21:14:25.4884689Z 206: 
2024-04-04T21:14:25.4884917Z 206:   DA:107,1
2024-04-04T21:14:25.4885161Z 206: 
2024-04-04T21:14:25.4885374Z 206:   DA:114,1
2024-04-04T21:14:25.4885615Z 206: 
2024-04-04T21:14:25.4885830Z 206:   DA:115,1
2024-04-04T21:14:25.4886076Z 206: 
2024-04-04T21:14:25.4886292Z 206:   DA:118,1
2024-04-04T21:14:25.4886533Z 206: 
2024-04-04T21:14:25.4886752Z 206:   DA:121,1
2024-04-04T21:14:25.4886996Z 206: 
2024-04-04T21:14:25.4887218Z 206:   DA:122,1
2024-04-04T21:14:25.4887868Z 206: 
2024-04-04T21:14:25.4888181Z 206:   DA:124,1
2024-04-04T21:14:25.4888446Z 206: 
2024-04-04T21:14:25.4888659Z 206:   DA:125,1
2024-04-04T21:14:25.4888902Z 206: 
2024-04-04T21:14:25.4889115Z 206:   DA:126,1
2024-04-04T21:14:25.4889357Z 206: 
2024-04-04T21:14:25.4889729Z 206:   DA:127,1
2024-04-04T21:14:25.4889981Z 206: 
2024-04-04T21:14:25.4890201Z 206:   DA:128,1
2024-04-04T21:14:25.4890444Z 206: 
2024-04-04T21:14:25.4890811Z 206:   DA:130,1
2024-04-04T21:14:25.4891058Z 206: 
2024-04-04T21:14:25.4891274Z 206:   DA:131,0
2024-04-04T21:14:25.4891517Z 206: 
2024-04-04T21:14:25.4891740Z 206:   DA:133,1
2024-04-04T21:14:25.4891983Z 206: 
2024-04-04T21:14:25.4892199Z 206:   DA:134,0
2024-04-04T21:14:25.4892442Z 206: 
2024-04-04T21:14:25.4892663Z 206:   DA:136,1
2024-04-04T21:14:25.4892910Z 206: 
2024-04-04T21:14:25.4893122Z 206:   DA:137,1
2024-04-04T21:14:25.4893365Z 206: 
2024-04-04T21:14:25.4893581Z 206:   DA:139,1
2024-04-04T21:14:25.4893823Z 206: 
2024-04-04T21:14:25.4894039Z 206:   DA:140,1
2024-04-04T21:14:25.4894302Z 206: 
2024-04-04T21:14:25.4894522Z 206:   DA:142,1
2024-04-04T21:14:25.4894770Z 206: 
2024-04-04T21:14:25.4894986Z 206:   DA:143,1
2024-04-04T21:14:25.4895228Z 206: 
2024-04-04T21:14:25.4895461Z 206:   end_of_record
2024-04-04T21:14:25.4895749Z 206: 
2024-04-04T21:14:25.4895957Z 206: 
2024-04-04T21:14:25.4896169Z 206: 
2024-04-04T21:14:25.4896739Z 207/377 Test #206: code_api|tool.drcov.eintr ........................................***Failed    1.22 sec
abhinav92003 commented 6 months ago

Oh I think it's the SF:/opt/actions-runner/_work/dynamorio/dynamorio/suite/tests/linux/eintr.c line written by

https://github.com/DynamoRIO/dynamorio/blob/2587672112ef84b7eddf1c8b14e98da3205f1d4d/clients/drcov/postprocess/drcov2lcov.cpp#L1269

jackgallagher-arm commented 6 months ago

The difference is this line: output:

2024-04-04T21:14:23.5132299Z 206:   DA:126,0

expected:

2024-04-04T21:14:25.4889115Z 206:   DA:126,1

which corresponds to this code:

124:     pthread_mutex_lock(&lock);
125:     while (!child_ready)
126:         pthread_cond_wait(&condvar, &lock); // <-- Is this line not executed for some reason?
127:     child_ready = false;
128:     pthread_mutex_unlock(&lock);
jackgallagher-arm commented 6 months ago

Is this just a racy test? If the handler() has been called before the main thread gets to the while loop child_ready will already be set to true and the loop body won't be executed.

/*
 * test of restarted syscalls
 */

#include "tools.h"

#include <assert.h>
#include <stdio.h>
#include <string.h>
#include <signal.h>
#include <stdlib.h>
#include <pthread.h>

static pthread_cond_t condvar;
static bool child_ready;
static pthread_mutex_t lock;

static void
handler(int sig)
{
    print("in handler %d\n", sig);

    /* potentially unsafe but we risk it: we should be in our read syscall */
    pthread_mutex_lock(&lock);
    child_ready = true;
    pthread_cond_signal(&condvar);
    pthread_mutex_unlock(&lock);
}

/* snip */

int
main(int argc, char **argv)
{
    /* snip */

    /* test signal w/ handler */
    print("sending SIGUSR1\n");
    pthread_kill(thread, SIGUSR1);
                  /* <-- handler() is executed here, setting child_ready = true */
    pthread_mutex_lock(&lock);
    while (!child_ready) /* child_ready == true so the loop gets skipped */
        pthread_cond_wait(&condvar, &lock);
    child_ready = false;
    pthread_mutex_unlock(&lock);

    /* snip */

    return 0;
}
abhinav92003 commented 6 months ago

You're right -- looks like I was chasing a red herring.

I can modify the expected output to expect either a 0 or 1 for that line.