IronLanguages / ironpython2

Implementation of the Python programming language for .NET Framework; built on top of the Dynamic Language Runtime (DLR).
http://ironpython.net
Apache License 2.0
1.07k stars 229 forks source link

ANSYS Workbench Ironpython threading problem #783

Open 1990chs opened 3 years ago

1990chs commented 3 years ago

Description

ANSYS Workbench use Ironpython language to develop extension and I try to use threading modulus to improve calculating speed(most are I/O tasks). But it seems that the multi threading don't work.

Steps to Reproduce

  1. First I build a classic as follow:
    
    import threading

class MyThread(threading.Thread): def init(self, func, args, name=''): threading.Thread.init(self) self.name = name self.func = func self.args = args

def getResult(self):
    return self.result

def run(self):
    self.result = self.func(*self.args)
2. Then I use the following way to use it:

from myThread import MyThread results = [] for i in range(3): t = MyThread(funcs[i], (q, nloops)_) t.start() t.join() results.append(t.getResult())


I can get the right result, but the calculate speed don't improve. It is almost same speed with no using thread.

### Versions

IronPython 2.7.0.40 on .NET 4.0.30319.42000
The version is very old and it is the ANSYS Workbench contain, so I ca't update it to the lasting version.

![屏幕截图 2021-06-06 111815](https://user-images.githubusercontent.com/27892759/120911306-fd3f5380-c6b8-11eb-8fb4-75368ca27134.png)
slozier commented 3 years ago

You are misusing threading. t.join will block until the thread terminates so it's as if you weren't using threading at all. You need to do something like:

threads = []
for i in range(3):
    t = MyThread(funcs[i], (q, nloops)_)
    t.start()
    thread.append(t)

for t in threads:
    t.join()
    results.append(t.getResult())
1990chs commented 3 years ago

You are misusing threading. t.join will block until the thread terminates so it's as if you weren't using threading at all. You need to do something like:

threads = []
for i in range(3):
    t = MyThread(funcs[i], (q, nloops)_)
    t.start()
    thread.append(t)

for t in threads:
    t.join()
    results.append(t.getResult())

MyThread(funcs[i], (q, nloops)_) for i in range(3)]

I put t.join() and t.start() in one loop. you mean they should be on different loop ?

slozier commented 3 years ago

I put t.join() and t.start() in one loop. you mean they should be on different loop ?

Yes. One loop starts all your threads and the other loop wait for them to end.

1990chs commented 3 years ago

I put t.join() and t.start() in one loop. you mean they should be on different loop ?

Yes. One loop starts all your threads and the other loop wait for them to end.

Thinks! I will try it

1990chs commented 3 years ago

I put t.join() and t.start() in one loop. you mean they should be on different loop ?

Yes. One loop starts all your threads and the other loop wait for them to end.

I have try it. It seems the speed is the same without using thread, I don't know why . This version ironPython don't have multiprocessing modulus, so sad. So, is there any way to improve calculate speed ?

Cyril-Pop commented 3 years ago

you can try to use Net framework with AsParallel() (extension System.Linq) https://docs.microsoft.com/fr-fr/dotnet/api/system.linq.parallelenumerable.asparallel?view=net-5.0

an example

import clr
import sys
clr.AddReference("System.Core")
import System
clr.ImportExtensions(System.Linq)
from System.IO import *
from System.Net import *
import time

request = WebRequest.Create("https://www.gutenberg.org/files/65568/65568-0.txt")
with request.GetResponse() as response:  
    with response.GetResponseStream() as stream:
        with StreamReader(stream) as reader:
            html = reader.ReadToEnd()
            html = html.encode("utf-8")

def counterWords(chrt, lstTxt):
    chrt = chrt.encode("utf-8")
    count_ = lstTxt.count(chrt)
    return [chrt, count_]

searchCharacs = ["the","on","a","in","day","this", "of", "his", "and", "he", "when", "it", "an", "she", "not", "you"]
# make a big list
searchCharacs = searchCharacs * 10
lstTxt = html.split()
# test without AsParallel
print("test without AsParallel")
start = time.time()
result =[]
for s in searchCharacs:
    result.append(counterWords(s, lstTxt))
elapse = ("%s s" % (time.time()-start))
print(elapse)

# test with AsParallel
print("test with AsParallel")
start2 = time.time()
threadResult = searchCharacs.AsParallel().Select(lambda searchC: counterWords(searchC, lstTxt))
elapse2 = ("%s s" % (time.time()-start2))
# print the result
# print(list(threadResult))
print(elapse2)

my result time image

1990chs commented 3 years ago

you can try to use Net framework with AsParallel() (extension System.Linq) https://docs.microsoft.com/fr-fr/dotnet/api/system.linq.parallelenumerable.asparallel?view=net-5.0

an example

import clr
import sys
clr.AddReference("System.Core")
import System
clr.ImportExtensions(System.Linq)
from System.IO import *
from System.Net import *
import time

request = WebRequest.Create("https://www.gutenberg.org/files/65568/65568-0.txt")
with request.GetResponse() as response:    
  with response.GetResponseStream() as stream:
      with StreamReader(stream) as reader:
          html = reader.ReadToEnd()
          html = html.encode("utf-8")

def counterWords(chrt, lstTxt):
  chrt = chrt.encode("utf-8")
  count_ = lstTxt.count(chrt)
  return [chrt, count_]

searchCharacs = ["the","on","a","in","day","this", "of", "his", "and", "he", "when", "it", "an", "she", "not", "you"]
# make a big list
searchCharacs = searchCharacs * 10
lstTxt = html.split()
# test without AsParallel
print("test without AsParallel")
start = time.time()
result =[]
for s in searchCharacs:
  result.append(counterWords(s, lstTxt))
elapse = ("%s s" % (time.time()-start))
print(elapse)

# test with AsParallel
print("test with AsParallel")
start2 = time.time()
threadResult = searchCharacs.AsParallel().Select(lambda searchC: counterWords(searchC, lstTxt))
elapse2 = ("%s s" % (time.time()-start2))
# print the result
# print(list(threadResult))
print(elapse2)

my result time image

Thinks. I will try.