Possibly fixed large threads issue

[Ajfaust]

Just wanted to let you know I made some changes to the code and I believe it is working now. Please run your own tests though and see if I missed anything. The code is in problemb_version2 branch. @ckloi @tihju @mwared

[ckloi]

sometime it would not work correctly , if you a have large file and run multiple times

   threadClass.resultList[-1] += self.localList.pop(0)

[Ajfaust]

It only happens for really large thread numbers (~100+). Anything small like 5 or 10 seems to work. Do you think he will test with really large thread numbers?

[ckloi]

there will be a chance not working

import ProbB

fd = open('b')

l = map(lambda x : x-1 ,map(len,fd.readlines()))

print l
for i in range(1,4):
    for k in range(10000):
        x = ProbB.linelengths('b', i)
        #print (len(x))
        for j in range(len(x)):
            if l[j] == x[j]:
                pass
            else:
                print 'Error :'
                print '# of thread ' +str(i)
                print '# of iteration ' + str(k)
                print x
                raise Exception("Not the same")

        ProbB.threadClass.resultList = []

print("Done")

[tihju]

I think the problem is due to race condition. there are some places that we need to add lock but we have not considered it. what do you think about? do your guys think this is the problem. I thought flag should be one that need to lock, but that did not solve the problem. what else that we miss?

[Ajfaust]

The race condition isn't the problem. I moved acquire() around and it didn't really change anything. I do think it should go right after the while statement though.

[Ajfaust]

I think the problem is that read() is called inside run(). Because of this, threads get assigned id's, but they may not read the appropriate chunk (e.g. thread with id 2 may be executed before thread with id 0), which screws up the order of the array. I made a push that moves read() to init, and ran the program a bunch of times without any errors. Please take a look. @ckloi @tihju @mwared

[tihju]

wow, great job!

[Ajfaust]

Thanks! I'm not completely sure if it totally fixed it, but I really hope it did haha.

Btw great job guys. You're all doing great work.

[ckloi]

if u put read outside of run , there is no point we do multi threading

[tihju]

It will be ok if you call a function that contain read though. On Sat, Feb 18, 2017 at 7:00 PM ckloi notifications@github.com wrote:

if u put read outside of run , there is no point we do multi threading

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ckloi/145/issues/31#issuecomment-280892071, or mute the thread https://github.com/notifications/unsubscribe-auth/AQGkEQfgC-IGOF1W89mAlKgScZwnYUMxks5rd7BEgaJpZM4MFPoK .

[Ajfaust]

Oh yea I guess that's true. Damn it haha.

One thing I noticed is that: a) the line numbers that don't match are almost always sequential, and b) the values of the lines that don't match add up to the same number

For example,

two list are not equal in line: 135 312 , 148 two list are not equal in line: 136 730 , 894

312 + 730 = 148 + 894 = 1,042

I think this means that either our way of checking if a thread has stopped in the middle of a line isn't working, or the threads are somehow not stopping where they need to stop. Just thought I'd bring it up in case any of you have an idea on what it is/how to fix it. @ckloi @tihju @mwared

[ckloi]

nvm

[Ajfaust]

Ah I see. I knew you had to seek somehow. I never tried locking it though haha. Nice work!