Segmentation fault on certain rawacfs

[asreimer]

Testing the master branch, I found that certain rawacf files will cause the code to segfault. The rawacf that I used is: 20090101.0000.12.sas.rawacf. I cannot upload this to git because the file is >10MB even when compressed using BZIP2.

[kkotyk]

Did you try with backscatter too?

On Sep 19, 2016 20:33, "Ashton Reimer" notifications@github.com wrote:

Testing the master branch, I found that certain rawacf files will cause the code to segfault. The rawacf that I used is: 20090101.0000.12.sas.rawacf. I cannot upload this to git because the file is >10MB even when compressed using BZIP2.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/SuperDARNCanada/fitacf.3.0/issues/2, or mute the thread https://github.com/notifications/unsubscribe-auth/AGxtWsDXpqVIUy5IwEPYxBXIfY0UrD4Mks5qr0XggaJpZM4KBKZ8 .

[asreimer]

Not yet. I'm getting the same problem on my lmfit2 software. I'll post a detailed report here once I fix it on my end. Same thing is happening to both and will probably be a problem in backscatter too.

[kkotyk]

Does fitacf2 fail as well? Like standard make fit?

On Sep 19, 2016 20:37, "Ashton Reimer" notifications@github.com wrote:

Not yet. I'm getting the same problem on my lmfit2 software. I'll post a detailed report here once I fix it on my end. Same thing is happening to both and will probably be a problem in backscatter too.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/SuperDARNCanada/fitacf.3.0/issues/2#issuecomment-248185979, or mute the thread https://github.com/notifications/unsubscribe-auth/AGxtWkCHKXfoVjH2Uz-l2OMAonW-dOtFks5qr0bhgaJpZM4KBKZ8 .

[asreimer]

No.

[kkotyk]

Alright ill look into this at some point. Just let me know what rawacfs it is.

On Sep 19, 2016 20:49, "Ashton Reimer" notifications@github.com wrote:

No.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/SuperDARNCanada/fitacf.3.0/issues/2#issuecomment-248187433, or mute the thread https://github.com/notifications/unsubscribe-auth/AGxtWrSueX2SsdJPVNmCnKOWZ5c8dZ9sks5qr0mdgaJpZM4KBKZ8 .

[asreimer]

I posted the name of the rawacf that's causing this problem in my OP. I'll let you know the fix pretty soon. I've almost got it figured out.

[kkotyk]

Okay. Thats the only one?

On Sep 19, 2016 20:52, "Ashton Reimer" notifications@github.com wrote:

I posted the name of the rawacf that's causing this problem in my OP. I'll let you know the fix pretty soon. I've almost got it figured out.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/SuperDARNCanada/fitacf.3.0/issues/2#issuecomment-248187808, or mute the thread https://github.com/notifications/unsubscribe-auth/AGxtWtd7dbNS-vDFnQtS9tRulKCIa8aIks5qr0pmgaJpZM4KBKZ8 .

[asreimer]

There are others but I think that they have the same problem. Also, turns out the problem I'm having in lmfit2 is not the same as the problem that fitacf3.0 has.

Here's the output from gdb for fitacf3.0 after running run -new 20090101.0000.12.sas.rawacf > 20090101.0000.12.sas.fitacf

Program received signal SIGSEGV, Segmentation fault.
0x0000000000402cd5 in Copy_Fitting_Prms (radar_site=0x6218a0, 
    radar_prms=0x61e010, raw_data=0x61e0c0, fit_prms=0x624f10)
    at src/fitacftoplevel.c:195
195             fit_prms->acfd[i*fit_prms->mplgs+j][0]=raw_data->acfd[0][i*fit_prms->mplgs+j];

and here's the bt:

(gdb) bt
#0  0x0000000000402cd5 in Copy_Fitting_Prms (radar_site=0x6218a0, 
    radar_prms=0x61e010, raw_data=0x61e0c0, fit_prms=0x624f10)
    at src/fitacftoplevel.c:195
#1  0x0000000000402542 in main (argc=3, argv=0x7fffffffd448)
    at make_fitacf.3.0/make_fit.c:257

The problem I was having in lmfit2 was related to the acfd array have 0 length, but this was manifesting in the preprocessing, not in the Copy_Fit_Prms function.

[asreimer]

Not sure if this will be helpful, but here's what I did to fix my problem in lmfit2: https://github.com/asreimer/lmfit2/commit/e0386867b4b4215be6706bc3d6fcbbb641953198

My code didn't segfault in Copy_Fit_Prms though.

[asreimer]

Alright, now I'm getting the same problem in lmfit2 as in fitacf3.0... Damn it.

[asreimer]

Ok, so the problem is that the raw_data array is a null pointer for some reason (no raw acf data in the file at that point for some reason):

(gdb) print raw_data->acfd[0][i*fit_prms->mplgs+j]
Cannot access memory at address 0x0
(gdb) print raw_data->acfd
$7 = {0x0, 0x0}

[kkotyk]

Hmm interesting. That must be a broken file. I've added a similar check for when XCFs are null. I guess I need to add that for ACFs too.

On Mon, Sep 19, 2016 at 9:36 PM, Ashton Reimer notifications@github.com wrote:

Ok, so the problem is that the raw_data array is a null pointer for some reason (no raw acf data in the file at that point for some reason):

(gdb) print raw_data->acfd[0][i*fit_prms->mplgs+j] Cannot access memory at address 0x0 (gdb) print raw_data->acfd $7 = {0x0, 0x0}

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/SuperDARNCanada/fitacf.3.0/issues/2#issuecomment-248192834, or mute the thread https://github.com/notifications/unsubscribe-auth/AGxtWvPtnPvKTRr5iDGSGGnqiR1-xYozks5qr1TDgaJpZM4KBKZ8 .

-Keith Kotyk

[asreimer]

Ya, there just needs to be NULL checking everywhere I guess. Good ol' dmap library.

[asreimer]

This is what I did to fix the segfault in lmfit2. It should also fix the problem in fitacf3.0. I replaced the lines that copy the acfd and xcfd from the raw_data pointer with the following:

    if (raw_data->acfd != NULL){
        for (i=0;i<fit_prms->nrang;i++) {
            if (raw_data->acfd[0] != NULL){
                for (j=0;j<fit_prms->mplgs;j++) {
                    fit_prms->acfd[i*fit_prms->mplgs+j][0]=raw_data->acfd[0][i*fit_prms->mplgs+j];
                    fit_prms->acfd[i*fit_prms->mplgs+j][1]=raw_data->acfd[1][i*fit_prms->mplgs+j];
                }
            }
        } 
    }
    if (raw_data->xcfd != NULL){
        for (i=0;i<fit_prms->nrang;i++) {
            if (raw_data->xcfd[0] != NULL){
                for (j=0;j<fit_prms->mplgs;j++) {
                    fit_prms->xcfd[i*fit_prms->mplgs+j][0]=raw_data->xcfd[0][i*fit_prms->mplgs+j];
                    fit_prms->xcfd[i*fit_prms->mplgs+j][1]=raw_data->xcfd[1][i*fit_prms->mplgs+j];
                }
            }
        }
    }

[asreimer]

Just tested it, can confirm my code fixes fitacf3.0 for this issue.

[ksterne]

Sorry I've been a little late to this game...and maybe it's already done though I don't see any commits anywhere.

I can confirm that the 20090101.00sas rawacf file input causes a segmentation fault. However, 20090101.00bks rawacf file doesn't.

Let me know if I can help out with anything here.

[kkotyk]

Ashton used that code to fix a fork of fitacf.3.0 he used to make his own fitting. I've been busy and haven't looked at fixing this issue yet. Ashton's code snippet does look correct in determining if the acfd arrays are null.

[ksterne]

Ah, good ole acfd arrays being null...I chased that issue down in the VTRST a while ago. It's a good one!

[asreimer]

Ya, really quite annoying. Thankfully it happened on the first file I tried so it was easy to chase down.

[kkotyk]

I've added slightly different code that what posted ashton, but this issue should be fixed now. The only difference is that I fill the arrays with zeros if those pointers are NULL