ottokart / punctuator2

A bidirectional recurrent neural network model with attention mechanism for restoring missing punctuation in unsegmented text
http://bark.phon.ioc.ee/punctuator
MIT License
657 stars 195 forks source link

Unclear on which paths to use in the model #51

Open helga-lvl opened 4 years ago

helga-lvl commented 4 years ago

Hey! I'm trying to train Punctuator 2 in Icelandic. I preprocessed the raw text with the script:

with open("isl_out.txt", 'w', encoding='utf-8') as out_txt:encoding='utf-8') as text:

    for line in text:
        line = line.replace("\"", "").strip()
        line = line.replace("“", "").strip()

        line = multiple_punct.sub(r"\g<1>", line)

        if skip(line):
            skipped += 1
            continue

        line = process_line(line)
        alllines_en.append(line)

print("Skipped %d lines" % skipped)

and saved 80% to isl.train.txt, 10% to isl.dev.txt and 10% to isl.test.txt.

Then I ran: python data.py datadir

python main.py models.py 256 0.02

After this has finished running, I'm very unsure about what to do next, with:

cat data.dev.txt | python punctuator.py <model_path> <model_output_path>

I tried: cat datadir/isl.dev.txt | python punctuator.py Model_models.py_h256_lr0.02.pcl model_output.pcl

but get an error:

Loading model parameters...
Traceback (most recent call last):
  File "punctuator.py", line 149, in <module>
    net, _ = models.load(model_file, 1, x)
  File "/Users/starspace/Desktop/LVL/H12/punctuator2-master/models.py", line 54, in load
    from . import models
ValueError: Attempted relative import in non-package

So most probably I was running the wrong command, but otherwise I don't know what to do with this error. Explanations would be very much appreciated, thanks!

itzsimpl commented 4 years ago

Simply change line 54 in models.py from

from . import models

to

import models

and it should work.