search

kylebgorman / pynini

Read-only mirror of Pynini
http://pynini.opengrm.org
Apache License 2.0
120 stars 26 forks source link

One-off bug? #18

Closed lxkain closed 4 years ago

lxkain commented 4 years ago

Using openfst 1.7.5, and pynini 2.0.9.post2, if I do:

words='you say potato'.split()

word_table = ni.SymbolTable()
word_table.add_symbol('<ε>')  # empty symbol at index 0
print(word_table.num_symbols())

for sym in words:
    word_table.add_symbol(sym)
    print(word_table.num_symbols())

I see output:

1
2
2
3
kylebgorman commented 4 years ago

Thanks for very clear report; looks suspicious. Will investigate: would probably be an issue with OpenFst itself. There has been some recent churn in the symbol table object.

On Tue, Dec 17, 2019 at 5:21 PM Alexander Kain notifications@github.com wrote:

Using openfst 1.7.5, and pynini 2.0.9.post2, if I do:

words='you say potato'.split()

word_table = ni.SymbolTable()

word_table.add_symbol('<ε>') # empty symbol at index 0

print(word_table.num_symbols())

for sym in words:

word_table.add_symbol(sym)

print(word_table.num_symbols())

I see output:

1

2

2

3

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/kylebgorman/pynini/issues/18?email_source=notifications&email_token=AABG4OIKOGFX6IT56DOJ53LQZFGGXA5CNFSM4J4CLG4KYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4IBFWTUQ, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABG4OLTNDSHW53J6BHZVVDQZFGGXANCNFSM4J4CLG4A .

lxkain commented 4 years ago

Should I try with openfst 1.7.4?

kylebgorman commented 4 years ago

You could but I’m not sure it’d help track this down.

Unrelatedly almost all of this stuff is on Conda-Forge as of yesterday.

On Tue, Dec 17, 2019 at 5:32 PM Alexander Kain notifications@github.com wrote:

Should I try with openfst 1.7.4?

— You are receiving this because you commented.

Reply to this email directly, view it on GitHub https://github.com/kylebgorman/pynini/issues/18?email_source=notifications&email_token=AABG4ONIESW5ELUBXEDI2G3QZFHODA5CNFSM4J4CLG4KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEHEFZOI#issuecomment-566779065, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABG4OOO424BTIJLKOPJQS3QZFHODANCNFSM4J4CLG4A .

lxkain commented 4 years ago

I tried installing from conda-forge on a brand-new anaconda python 3.7 install. After about one hour I hit CTRL-C :(

kylebgorman commented 4 years ago

Good news, this doesn't replicate at (internal) HEAD for me so we must have fixed it or, IDK, something else is going on. Will put out another version bump before too long.

lxkain commented 4 years ago

This bug persists in version 2.1.0 :(

Incidentally, the conda-forge install works now!

kylebgorman commented 4 years ago

Would you be willing to take a look? I absolutely cannot reproduce this as I do not own any Macs and it doesn't seem to occur anywhere else.

On Mon, Feb 24, 2020 at 10:16 PM Alexander Kain notifications@github.com wrote:

This bug persists in version 2.1.0 :(

Incidentally, the conda-forge install works now!

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/kylebgorman/pynini/issues/18?email_source=notifications&email_token=AABG4OMFI26ZK73FVWXAOJTRESEQ5A5CNFSM4J4CLG4KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEM2MQNY#issuecomment-590661687, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABG4OLEHNHVSTQGZX67P4DRESEQ5ANCNFSM4J4CLG4A .

lxkain commented 4 years ago

I ran this in an ubuntu VM and indeed it works fine! Very puzzling. I'm not sure if I will have the time to debug this, and I can use Linux as a work-around for the time being.

lxkain commented 4 years ago

CFLAGS="-stdlib=libc++ -mmacosx-version-min=10.7" python setup.py install

fixes this