FraBle / python-sutime

Python wrapper for Stanford CoreNLP's SUTime
GNU General Public License v3.0
153 stars 43 forks source link

sutime not working with ProcessPool #34

Open akashdave-v opened 4 years ago

akashdave-v commented 4 years ago

Why it is not working by just replacing ThreadPoolExecutor with ProcessPoolExecutor? @FraBle Can you please look at this and reply as soon as possible? Thank you.

Here is a code snippet using your code:

import os
import json
from sutime import SUTime
from concurrent.futures import ProcessPoolExecutor
from multiprocessing import cpu_count

def get_sutime():
    jar_files = os.path.join(os.path.dirname(__file__), os.pardir, "jars")
    sutime = SUTime(jars=jar_files, mark_time_ranges=True)
    return sutime

def struct_filing(sentence, sutime):
    return sentence, sutime.parse(sentence)

def get_time(text_list):
    sutime = get_sutime()
    with ProcessPoolExecutor(max_workers=cpu_count()) as executor:
        return dict(
            future.result()
            for future in [
                executor.submit(struct_filing, sentence, sutime)
                for sentence in text_list
            ]
        )

if __name__ == "__main__":
    data_set_1 = {
        "The Formula One race came to Singapore in 2008.",
        "It was a tremendous success and gained wide international coverage.",
        "In a first for the F1, the event was held at night.",
        "State-of-the-art technology was used to ensure that the race operated"
        " in near daylight conditions.",
        "Holding the race at night had a great advantage as European audiences"
        " could watch it at a convenient time.",
        "Over the years, Singapore’s tourism industry has gained significant"
        " benefits from the media attention that the race has generated",
    }
    print(json.dumps(get_time(data_set_1), sort_keys=True, indent=4))

Originally posted by @FraBle in https://github.com/FraBle/python-sutime/issues/9#issuecomment-433708712