Autotester complete misbehavior

[0xd34df00d]

Autotester script always returns different results:

19:52:02 deadfood Ubuntu-1204-precise-64-minimal ~/barzer/src/autotester % ./autotest.py -r -in ~/queries.big                              
<autotest time="2012-08-12 19:52" level="0">
<test score="70" id="130" user="84">Durst</test>
<test score="70" id="229" user="87">susha in nw york</test>
<test score="70" id="681" user="84">point   shoot HD</test>
<test score="84" id="1154" user="87">jeanne</test>
<test score="70" id="2006" user="84">A&I</test>
<test score="70" id="2187" user="84">Canvas Scenic (Light Stand Mount)</test>
<test score="70" id="2303" user="84">Square or Rectangular Kit</test>
<test score="70" id="2404" user="84">Stainless Steel Manual Tanks</test>
<test score="70" id="2436" user="84">Sprint Systems of Photography</test>
<test score="34" id="2496" user="84">DC Couplers</test>
<test score="70" id="2681" user="84">Light & Motion</test>
<test score="70" id="2814" user="84">Spring & A Style Clamps</test>
<test score="70" id="2822" user="84">S Series</test>
<test score="34" id="2836" user="84">La Sardina</test>
<test score="70" id="2845" user="84">Framing Accessories</test>
<test score="84" id="3292" user="84">Battery Packs</test>
<test score="70" id="3433" user="84">Sterling Publishing</test>
<test score="84" id="3936" user="84">Satellite</test>
<test score="70" id="3976" user="84">Slide Viewers Slide Sorters</test>
<test score="70" id="4180" user="84">Flatbed and Document Fed Scanners</test>
<test score="70" id="4384" user="84">Monster Cable</test>
<stat over_level="21"  not_passed="21"  passed="4518"  ></stat>
</autotest>
19:52:06 deadfood Ubuntu-1204-precise-64-minimal ~/barzer/src/autotester % ./autotest.py -r -in ~/queries.big
<autotest time="2012-08-12 19:52" level="0">
<test score="70" id="284" user="84">canon lens un</test>
<test score="70" id="1527" user="54">board approval required</test>
<test score="70" id="2104" user="84">E Films</test>
<test score="84" id="2620" user="84">Battery Testers</test>
<test score="70" id="2633" user="84">Wedding & Portraiture</test>
<test score="70" id="2682" user="84">Marshall Retouching</test>
<test score="70" id="2727" user="84">Side Splashes</test>
<test score="70" id="2802" user="59">red converse sneakers under $150</test>
<test score="70" id="2845" user="84">Framing Accessories</test>
<test score="70" id="3433" user="84">Sterling Publishing</test>
<test score="70" id="3657" user="84">Light Controls</test>
<test score="46" id="3738" user="84">California Sunbounce</test>
<test score="70" id="3864" user="84">K 5600 Lighting</test>
<test score="70" id="4076" user="82">largest pot</test>
<stat over_level="14"  not_passed="14"  passed="4524"  ></stat>
</autotest>
19:52:10 deadfood Ubuntu-1204-precise-64-minimal ~/barzer/src/autotester % ./autotest.py -r -in ~/queries.big
<autotest time="2012-08-12 19:54" level="0">
<test score="70" id="297" user="84">Paper Developers</test>
<test score="70" id="483" user="87">borough</test>
<test score="70" id="793" user="84">Retaining Rings</test>
<test score="70" id="1035" user="84">Lighting Filters</test>
<test score="70" id="1612" user="84">Lighting for Photography & Videography</test>
<test score="70" id="1664" user="84">Canvas (Light Stand Mount)</test>
<test score="84" id="1683" user="84">Crossbar</test>
<test score="70" id="2029" user="84">Y Cord</test>
<test score="70" id="2187" user="84">Canvas Scenic (Light Stand Mount)</test>
<test score="70" id="2629" user="59">red converse</test>
<test score="70" id="2645" user="54">green buiding</test>
<test score="70" id="2681" user="84">Light & Motion</test>
<test score="70" id="3161" user="84">Light White Balancing Accessories</test>
<test score="70" id="3610" user="84">Battery Powered Strobe Lighting</test>
<test score="84" id="3762" user="84">Sunpak</test>
<test score="70" id="3973" user="84">Weston</test>
<test score="70" id="4000" user="84">batteries for canon t2i</test>
<test score="92" id="4149" user="84">Cengage Course Tech.</test>
<stat over_level="18"  not_passed="18"  passed="4518"  ></stat>
</autotest>

Please note that even not_passed + passed is always different which is obviously not good.

The sample queries data is on eu.barzer.net:/home/d34df00d/queries.big , barzer is 2861fb4ad8cba9351d8abebc2c95d3ec8e5f9cf3

[barzerman]

please assign to nikita

Sent from my iPhone

On Aug 12, 2012, at 2:25 PM, Georg Rudoy notifications@github.com wrote:

Autotester script always returns different results:

19:52:02 deadfood Ubuntu-1204-precise-64-minimal ~/barzer/src/autotester % ./autotest.py -r -in ~/queries.big

Durst susha in nw york point shoot HD jeanne A&I Canvas Scenic (Light Stand Mount) Square or Rectangular Kit Stainless Steel Manual Tanks Sprint Systems of Photography DC Couplers Light & Motion Spring & A Style Clamps S Series La Sardina Framing Accessories Battery Packs Sterling Publishing Satellite Slide Viewers Slide Sorters Flatbed and Document Fed Scanners Monster Cable

19:52:06 deadfood Ubuntu-1204-precise-64-minimal ~/barzer/src/autotester % ./autotest.py -r -in ~/queries.big

canon lens un board approval required E Films Battery Testers Wedding & Portraiture Marshall Retouching Side Splashes red converse sneakers under $150 Framing Accessories Sterling Publishing Light Controls California Sunbounce K 5600 Lighting largest pot

19:52:10 deadfood Ubuntu-1204-precise-64-minimal ~/barzer/src/autotester % ./autotest.py -r -in ~/queries.big

Paper Developers borough Retaining Rings Lighting Filters Lighting for Photography & Videography Canvas (Light Stand Mount) Crossbar Y Cord Canvas Scenic (Light Stand Mount) red converse green buiding Light & Motion Light White Balancing Accessories Battery Powered Strobe Lighting Sunpak Weston batteries for canon t2i Cengage Course Tech.

Please note that even not_passed + passed is always different which is obviously not good.

The sample queries data is on eu.barzer.net:/home/d34df00d/queries.big , barzer is 2861fb4https://github.com/barzerman/barzer/commit/2861fb4ad8cba9351d8abebc2c95d3ec8e5f9cf3

— Reply to this email directly or view it on GitHubhttps://github.com/barzerman/barzer/issues/339.

[0xd34df00d]

Already done.

[nchepanov]

I can reproduce this =( trying to find out the reason (

[barzerman]

ok and how is this doing now?

[0xd34df00d]

Haven't tried the newer version yet. The dataset is in eu.barzer.net:/home/d34df00d/barzer/src/build/site, it's actually the archive made by @barzerman 's script three days ago at the moment of opening the issue.

[barzerman]

three days of investigation no results what the fuck

[barzerman]

Гоша, будь добр попробуй разберись что со скрипиом не так? Никита работает по полчаса в день толком без интернета - это хуйня какаято

[0xd34df00d]

В барзер какая-то херь приходит. Я вставил дебаговый вывод в матчер:

424         auto res = matches(patBarz, resBarz, settings);
425         if (res > 1)
426             std::cout << "!!!!!!!!!!!!!!!!!!! score " << res

<< std::endl << std::string (pattern, patSize) << std::endl << "----" << std::endl << std::string (res, resSize) << std::endl << "=============" << std::endl;

Получается такая муть: https://paste.lugons.org/show/pwCCUF3lvuKXzxzqZkNj/

На скрипт я уже посмотрел, нашел проблему, почему сумма была разной — там не лочился мьютексом доступ к глобальной таблице со статистикой. Почему в барзер приходит говно — я не понял.

[0xd34df00d]

Ok, compiling barzer with -fomit-frame-pointer -march=native "fixes" this, but this sucks as a fix. I'd try to move code around a bit with plain -O2/-O3.

[barzerman]

-fomit-frame-pointer -march=native - how do these affect performance?

[0xd34df00d]

They actually are good for performance. For example, these two are recommended for those Gentoo users who care about performance :)

[pltr]

isn't gcc always omitting frame pointer with -O? I thought it was

[0xd34df00d]

According to the manual:

-O also turns on -fomit-frame-pointer on machines where doing so does

not interfere with debugging.

Not sure whether x86/amd64 are among those platforms. I'd rather say they are not.

[0xd34df00d]

Also, I've tried rewriting the whole string parsing algorithm there or replacing C strings and strstr with std::string and find/substr, but the bug still reproduces.

Once again, I have a couple of ideas why that may happen, and I'd check them tomorrow.

[barzerman]

please document the exact steps to reproduce the problem in the ticket

• i do think this is related to some bug in OUR code and not boost asio (in fact im pretty sure) . Here's why
• it only happens in compare mode (never for actual queries)
• the optimization you turn off points to the fact that there is some sort of a dangling reference somewhere

i have huge doubts this is an ASIO bug as we're using it in a very vanilla way and O3 is a vanilla optimization, for which I am sure boost has been tested well enough

[0xd34df00d]

1. Take the dataset from /home/d34df00d/barzer/src/build/site
2. Take the queries from /home/d34df00d/queries.big
3. Compile barzer with gcc 4.6 on Ubuntu 12.04 in Release mode (I guess RelWithDebInfo will do as well).
4. Run it as server with that dataset.
5. Run autotester.

I don't think it's a dangling reference or anything. -O3 is surely a vanilla optimization but gcc is buggy after all, and when quite strange things happen in your code, it's time to start blaming the compiler for generating shit. After all, I already encountered quite a lot misbehaviors in gcc.

I spent quarter of the day moving code around trying to fix this by forcing the compiler to generate some other code and another quarter for staring at disasm. That's hardly a dangling anything.

After all, what the fuck is this?

        uint16_t matches(const char *pattern, size_t patSize, const char *result, size_t resSize,
                        const ParseContext& ctx, const CompareSettings& settings)
        {
                Barz patBarz, resBarz;
                EnbarzParser(patBarz, *ctx.m_gp.getUniverse(ctx.m_userId))(pattern, patSize);
                EnbarzParser(resBarz, *ctx.m_gp.getUniverse(ctx.m_userId))(result, resSize);
                auto res = matches(patBarz, resBarz, settings);
                std::string resStr (result, resSize);
                if (res > 1)
                        std::cout << "!!!!!!!!!!!!!!!!!!! score " << res << std::endl << std::string (pattern, patSize) << std::endl << "----" << std::endl << std::string (res, resSize) << std::endl << "++++++++" << std::endl << resStr << std::endl << "=============" << std::endl;
                return res;
        }

And then you get something like this:

!!!!!!!!!!!!!!!!!!! score 70
<barz u="84">
<bead n="1">
    <fluff>over</fluff>
    <srctok>OverBoard</srctok>
</bead>

<bead n="2">
    <token>broad</token>
    <srctok>OverBoard</srctok>
</bead>
<spell>
<correction before="OverBoard" after="over broad"/>
</spell>
<traceinfo>
<match gram="7" file="site/alpha/0/fluff.xml" stmt="143490" emit="344"/>
</traceinfo>
<query>OverBoard</query>
</barz>

----
pppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppp
++++++++
<barz u="84">
<bead n="1">
    <fluff>over</fluff>
    <srctok>OverBoard</srctok>
</bead>

<bead n="2">
    <token>broad</token>
    <srctok>OverBoard</srctok>
</bead>
<spell>
<correction before="OverBoard" after="over broad"/>
</spell>
<traceinfo>
<match gram="7" file="site/alpha/0/fluff.xml" stmt="143490" emit="344"/>
</traceinfo>
<query>OverBoard</query>
</barz>

=============```

[barzerman]

autotester.py -r hangs for me on my ubuntu 12.04 about once out of three times

[barzerman]

server stalls on executable compiled with clang as well

[0xd34df00d]

Could you please show where it stalls? Like running it from gdb and pressing Ctrl+C when it seems stalled, and printing the corresponding backtrace.

In fact, this issue is far more interesting and sane to be dealt with then the original one.

[barzerman]

i ran a server with 6 threads ran 6 autotester.py scripts sequentially - as one stalled i launched a new one without bouncing barzer.exe now im on the 8th task - this is NOT a deadlock - the client apparently waits for connection close

[barzerman]

good news is that this is likely a client problem (i am now talking about the stalls while GENERATING answers) the situation with comparisons (and supposed buffer inconsistencies is different)

[0xd34df00d]

This info is unfortunately not the one I can derive productive info from. Could you please still send me the stack traces of all the frames during such stalls, as I've asked initially?

[barzerman]

consolidating https://github.com/barzerman/barzer/issues/331 into this

[barzerman]

fixed by upgrading boost to 1.50

[0xd34df00d]

Are you sure the original issue is fixed, not the stall? As I said, these two are different.

Georg Rudoy http://leechcraft.org 25.08.2012 20:11 пользователь "barzerman" notifications@github.com написал:

fixed by upgrading boost to 1.50

— Reply to this email directly or view it on GitHubhttps://github.com/barzerman/barzer/issues/339#issuecomment-8024717.

[barzerman]

the original issue hasnt really bothered us yet :) if and when it does we'll reopen

On Sat, Aug 25, 2012 at 12:16 PM, Georg Rudoy notifications@github.comwrote:

Are you sure the original issue is fixed, not the stall? As I said, these two are different.

Georg Rudoy http://leechcraft.org 25.08.2012 20:11 ÐÏÌØÚÏ×ÁÔÅÌØ "barzerman" notifications@github.com ÎÁÐÉÓÁÌ:

fixed by upgrading boost to 1.50

Reply to this email directly or view it on GitHub< https://github.com/barzerman/barzer/issues/339#issuecomment-8024717>.

Reply to this email directly or view it on GitHubhttps://github.com/barzerman/barzer/issues/339#issuecomment-8024763.

www.barzer.net