uezo commented 4 years ago

Usage

Create instance of SimpleProgressIndicator and pass it as argument like below:

from janome.dic import UserDictionary, SimpleProgressIndicator
from janome import sysdic

# create progress indicator
progress_indicator = SimpleProgressIndicator(update_frequency=0.001)

# create user dictionay
user_dict = UserDictionary(
    user_dict="/path/to/userdict.csv",
    enc="utf8",
    type="ipadic",
    connections=sysdic.connections,
    progress_handler=SimpleProgressIndicator()  # pass progress indicator here
)

# save dictionary
user_dict.save("/path/to/save")

Then, progress will be indicated.

Reading user dictionary from CSV: 100.0% | 152869/152869
Running create_minimum_transducer: 19.0% | 29000/152869

Custom indicator

You can also use your own custom indicator. Here is the example using tqdm.

from janome.progress import ProgressHandler
from tqdm import tqdm

class TQDMProgressIndicator(ProgressHandler):
    def __init__(self, update_frequency=0.1, tqdm=None):
        self.update_frequency = update_frequency
        self.tqdm = tqdm
        self.total = None
        self.value = None
        self.desc = None

    def on_start(self, total, value=0, desc=None):
        self.total = total
        self.value = value
        self.desc = desc or "Processing"
        self.tqdm = None

    def on_progress(self, value=1):
        self.value += value

        if self.tqdm is None:
            self.tqdm = tqdm(total=self.total, desc=self.desc)

        if float.is_integer(self.value * self.update_frequency):
            self.tqdm.update(int(1 / self.update_frequency))

    def on_complete(self):
        self.tqdm.n = self.value
        self.tqdm.update(0)
        self.tqdm.close()
        self.total = self.value = self.desc = self.tqdm = None