search

koomri / text-segmentation

Implementation of the paper: Text Segmentation as a Supervised Learning Task
253 stars 57 forks source link

Pretrained Model #7

Closed ShirleyHan6 closed 5 years ago

ShirleyHan6 commented 5 years ago

Is there a pretrained model we could use directly?

koomri commented 5 years ago

yes. The link to wiki-727 includes also CPU and GPU trained models

ShirleyHan6 commented 5 years ago

Thanks!

johnyoonh commented 5 years ago

@ShirleyHan6 Did it work for you? Using python run_web_server.py --model model_cpu.t7, The cutoffs is always all 0's for me, and the segmentation is blank. I've used both torch v0.31 and 1.2 in linux. I even tried using model_gpu.t7 by using torch.load with param map_location=torch.device('cpu') but the result was the same. I also tried both python2.7 and python3.6. 

def treat_text(raw_text):
    sentences = split_sentences(raw_text, 0)
    print(sentences)

    cutoffs = evaluate.predict_cutoffs(sentences, model, word2vec)
    total = []
    segment = []
    for i, (sentence, cutoff) in enumerate(zip(sentences, cutoffs)):
        segment.append(sentence)
        if cutoff:
            total.append(segment)
            segment = []

    return total
koomri commented 5 years ago

@johnyoonh The webapp is deprecated.

johnyoonh commented 5 years ago

@koomri My concern is predict_cutoffs always returning 0 rather than the webapp itself. Any idea why that might be happening?