lsalamon / slimgen

Automatically exported from code.google.com/p/slimgen
MIT License
0 stars 0 forks source link

Debugger Using 2.9gigs of Memory #7

Open GoogleCodeExporter opened 8 years ago

GoogleCodeExporter commented 8 years ago
I have no idea how or why but just for documentation the debugger.exe process 
shot up to 2.9gigs of memory which forced my memory to be paged which in turn 
pretty much froze my computer.

Original issue reported on code.google.com by Jorgy...@gmail.com on 13 Jul 2010 at 2:21

GoogleCodeExporter commented 8 years ago
I was able to do some primitive debugging and it appears that the 'main' loop 
in Main.cpp of the Debugger project is infinite looping. It appears that the 
loop is not reading off all of the data from the input stream. This is caused 
by me trying to inject some matrix multiply code (which is roughly 15 lines of 
assembly since I wasn't able to get very far yet). The code in question is here:

    while (std::cin.peek() != '\n') {
        SlimGen::MethodInformation method;
        std::getline(std::wcin, method.Assembly);
        std::getline(std::wcin, method.Method);

        int chunks;
        std::cin >> chunks >> std::ws;
        method.CompiledData = std::vector<std::vector<char>>(chunks);

        for (int i = 0; i < chunks; i++) {
            int size;
            std::cin >> size >> std::ws;

            method.CompiledData[i] = std::vector<char>(size);
            std::cin.read(&method.CompiledData[i][0], size);
        }
        methods.push_back(method);
    }

and the part that I believe is really causing the infinite loop is the for loop 
there. It doesn't seem to be reading all of the data being passed to it. I 
wasn't able to get beyond that. The assembly code that causes this is below 
(don't make fun of the extremely primitive assembly code [I have decided that a 
transpose multiply will be faster anyways]).

BITS 64

    ;Column1
    movss xmm0,[rdx+(3*4*4)]
    shufps xmm0,xmm0,93h
    movss xmm0,[rdx+(2*4*4)]
    shufps xmm0,xmm0,93h
    movss xmm0,[rdx+(1*4*4)]
    shufps xmm0,xmm0,93h
    movss xmm0,[rdx+(0*4*4)]

    ;M11
    movups xmm1,[rcx]
    mulps xmm1,xmm0
    haddps xmm1,xmm1
    haddps xmm1,xmm1
    movss [r8],xmm1

    movss xmm2,[rcx]
    movss [r8+8],xmm2

    movss xmm3,[rdx]
    movss [r8+12],xmm3

    rep ret

Original comment by Jorgy...@gmail.com on 13 Jul 2010 at 3:41