Drop pyfst, migrate to openfst python bindings?

[lkraav]

Recent kaldi requires recent openfst-1.6.x, but pyfst has trouble building with it.

fst/_fst.cpp:30878:191: note: candidate is:
In file included from /usr/include/fst/script/draw.h:10:0,
                 from fst/_fst.cpp:293:
/usr/include/fst/script/draw-impl.h:29:3: note: fst::FstDrawer<Arc>::FstDrawer(const fst::Fst<Arc>&, const fst::SymbolTable*, const fst::SymbolTable*, const fst::SymbolTable*, bool, const string&, float, float, bo
ol, bool, float, float, int, int, const string&, bool) [with Arc = fst::ArcTpl<fst::LogWeightTpl<float> >; std::string = std::basic_string<char>]
   FstDrawer(const Fst<Arc> &fst, const SymbolTable *isyms,
   ^
/usr/include/fst/script/draw-impl.h:29:3: note:   candidate expects 16 arguments, 15 provided

openfst has built-in python bindings since 1.5. Is it within reach to drop pyfst and migrate over?

[alumae]

Thanks, I didn't know about the python bindings in OpenFst 1.5+. Will investigate.

[lkraav]

Thanks. In the meanwhile, thanks to Gentoo being able to keep "still-in-use" libraries around, I managed to install openfst-1.6.1, build kaldi-5.1.65 with it, then downgrade to openfst-1.3.4 and build pyfst against this.

make build/output/recording-myaudio.txt now successfully continues at the pyfst step.

I wonder if there's any weirdness that can occur with this mix. I have a 100 minute audio file I'm transcribing, and ivector-extract + fgmm-global-gselect-to-post processes have been maxing out all 8 cores of the i7 CPU for a couple of hours now at the sid/extract_ivectors.sh: extracting iVectors step. Is this normal to take a long time even on a modern machine?

[alumae]

No, this is weird. This step shouldn't take much time.

[alumae]

Do you need the speaker ID step at all? Speaker ID tries to add speaker names (Estonian public figures) to the speech segments. You can turn it off using

DO_SPEAKER_ID=no

in Makefile.options

[alumae]

I migrated compounder.py to use OpenFst's native extension. Please test.

[lkraav]

No, this is weird. This step shouldn't take much time.

You were correct. I deleted build/, and restarted the transcription process from scratch (also used DO_SPEAKER_ID=no). Transcription now finished successfully within reasonable time.

EDIT it would be cool if the build script output some timestamps in the beginning of each line.

I will soon test openfst native extension version and report back.

[lkraav]

I will soon test openfst native extension version and report back.

Btw everything seems to work with just openfst, we can say goodbye to pyfst.