laytonjbgmail commented 3 years ago

Good morning,

I'm using pymp to call three functions, each in their own section. However, the code is hanging on a function return. The code is below.

import pymp import statistics

def func1(data):

total = 0.0
for item in data:
    total = total + item
# end for

return total

end def

def func2(data):

print("in func2")
avg = sum(data) / len(data)
print("   avg = ",avg)
return avg

end def

def func3(data):

median = statistics.median(data)

return median

end def

with pymp.Parallel(3) as p: for sec_idx in p.xrange(3): if (sec_idx == 0): data1 = [1, 2, 3, 4] total = func1(data1) p.print('Section 1') elif (sec_idx == 1): data2 = [5, 6, 7, 8] avg = func2(data2) p.print('Section 2') elif (sec_idx == 2): data3 = [9, 10, 11, 12, 12, 13, 14, 15] median = func3(data3) p.print('Section 3')

end if

# end for

end with

while ('total' not in globals() ): pass

end while

print("total = ",total)

while (median not in globals() ): pass

end if

print("median = ",median)

while ('avg' not in globals() ): pass

end while

print("avg = ",avg)

I can get output from the first section but the next one that uses func2, doesn't seem to return.

I'm sure there is a better way to wait until all three functions complete, but for now, I just keep checking if each function returns their value. But, in my code, the function returns seem to hang.

Thanks!

classner commented 3 years ago

Hi @laytonjbgmail ,

Could you add your code again, but place it in a Markdown code section (triple ` on one line separated by empty lines above and below, each above and below the code itself)? That would make it much easier to read and understand! You can use the 'Preview' tab to check how it will be rendered!

Christoph

laytonjbgmail commented 3 years ago

Apologies. That's shows my noobness. Let me try again:


import pymp
import statistics

def func1(data):

    print("   in func1")
    value1 = 0.0
    for item in data:
        value1 = value1 + item
    # end for

    return value1
# end def

def func2(data):

    print("   in func2")
    value2 = sum(data) / len(data)
    return value2
# end def

def func3(data):

    print("   in func3")
    value3 = statistics.median(data)

    return value3
# end def

with pymp.Parallel(3) as p:
    for sec_idx in p.xrange(3):
        if (sec_idx == 0):
            p.print('Section 1')
            data1 = [1, 2, 3, 4]
            total = func1(data1)
        elif (sec_idx == 1):
            p.print('Section 2')
            data2 = [5, 6, 7, 8]
            avg = func2(data2)
        elif (sec_idx == 2):
            p.print('Section 3')
            data3 = [9, 10, 11, 12, 12, 13, 14, 15]
            median = func3(data3)
        # end if
    # end for
# end with

print(" ")
print("total = ",total)
print("median = ",median)
print("avg = ",avg)

I hope this is better (it looked OK in preview).

Thanks!

classner commented 3 years ago

Hi @laytonjbgmail !

No worries and thanks for updating: works perfectly now!

There's a few changes I made to your code:

I 'import'ed pymp only after a __main__ guard for the actual code execution. This seems to be required for newer versions of Python to prevent an import problem with the multiprocessing module.
I synchronized the results through a pymp.shared.array. The variables that you define within the pymp.Parallel section are otherwise private and unknown to the process outside of that parallel loop. Because the access to each element of the array is not concurrent, no mutexes or other synchronization is required.

I end up with the following working code:

#!/usr/bin/env python3
import statistics

def func1(data):

    print("   in func1")
    value1 = 0.0
    for item in data:
        value1 = value1 + item
    # end for

    return value1
# end def

def func2(data):

    print("   in func2")
    value2 = sum(data) / len(data)
    return value2
# end def

def func3(data):

    print("   in func3")
    value3 = statistics.median(data)

    return value3
# end def

if __name__ == "__main__":
    import pymp

    results = pymp.shared.array(3)
    with pymp.Parallel(3) as p:
        for sec_idx in p.xrange(3):
            if (sec_idx == 0):
                p.print('Section 1')
                data1 = [1, 2, 3, 4]
                results[0] = func1(data1)
            elif (sec_idx == 1):
                p.print('Section 2')
                data2 = [5, 6, 7, 8]
                results[1] = func2(data2)
            elif (sec_idx == 2):
                p.print('Section 3')
                data3 = [9, 10, 11, 12, 12, 13, 14, 15]
                results[2] = func3(data3)
            # end if
        # end for
    # end with

    print(" ")
    print("total = ", results[0])
    print("median = ", results[1])
    print("avg = ", results[2])

Let me know if that works for you.

laytonjbgmail commented 3 years ago

That works great. Thanks

I have to admit that I slapped my forehead - I should have thought about a shared array.

At some point it would be nice to have each function return it's own result and not in a shared array but that's a "nice" to have :)

BTW - I didn't know about putting the import in the main to reduce problems

I'll have to google that one.

Thanks again! (apologies for the silly question)

Jeff

On Mon, Jun 28, 2021 at 1:10 AM Christoph Lassner @.***> wrote:

Hi @laytonjbgmail https://github.com/laytonjbgmail !

No worries and thanks for updating: works perfectly now!

There's a few changes I made to your code:

I 'import'ed pymp only after a main guard for the actual code execution. This seems to be required for newer versions of Python to prevent an import problem with the multiprocessing module.

I synchronized the results through a pymp.shared.array. The variables that you define within the pymp.Parallel section are otherwise private and unknown to the process outside of that parallel loop. Because the access to each element of the array is not concurrent, no mutexes or other synchronization is required.

I end up with the following working code:

!/usr/bin/env python3

import statistics

def func1(data):
print("   in func1")
value1 = 0.0
for item in data:
    value1 = value1 + item
# end for

return value1
end def

def func2(data):
print("   in func2")
value2 = sum(data) / len(data)
return value2
end def

def func3(data):
print("   in func3")
value3 = statistics.median(data)

return value3
end def

if name == "main": import pymp
results = pymp.shared.array(3)
with pymp.Parallel(3) as p:
    for sec_idx in p.xrange(3):
        if (sec_idx == 0):
            p.print('Section 1')
            data1 = [1, 2, 3, 4]
            results[0] = func1(data1)
        elif (sec_idx == 1):
            p.print('Section 2')
            data2 = [5, 6, 7, 8]
            results[1] = func2(data2)
        elif (sec_idx == 2):
            p.print('Section 3')
            data3 = [9, 10, 11, 12, 12, 13, 14, 15]
            results[2] = func3(data3)
        # end if
    # end for
# end with

print(" ")
print("total = ", results[0])
print("median = ", results[1])
print("avg = ", results[2])
Let me know if that works for you.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/classner/pymp/issues/22#issuecomment-869360119, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMYXLBIVQEZ3QBN5GJF7CQLTU773RANCNFSM47J7BYJQ .

laytonjbgmail commented 3 years ago

I was playing with code a bit and have a question. How can you

classner commented 3 years ago

Glad this worked! Good luck with your project!

classner / pymp

Problem with not returning from process #22

end def

end def

end def

end if

end with

end while

end if

end while

!/usr/bin/env python3

end def

end def

end def