miniconda is not in kaldi_root

[boeddeker]

Is there a reason, why miniconda must be installed in the home directory? (Many scripts have the path hard coded) https://github.com/kaldi-asr/kaldi/blob/cbdbedefcdd47e02a685c1dc2d1b128c30bdf6b2/tools/extras/install_miniconda.sh#L6

I would like to have miniconda installed either somewhere relative to the kaldi_root or have an environment variable to set the path to the miniconda installation.

My motivation to have it somewhere else:

• When miniconda is installed in the home directory, I cannot install miniconda in a shared folder for all users in our department (currently done for kaldi)
• We make a backup of the home directory, but the miniconda should not be in the backup (miniconda can be large, some packages need much space: e.g. numpy, mkl, pytorch, tensorflow, ...)

[boeddeker]

Another issue is, that install_miniconda.sh changes the user python environment. So you cannot have another python environment:

https://github.com/kaldi-asr/kaldi/blob/cbdbedefcdd47e02a685c1dc2d1b128c30bdf6b2/tools/extras/install_miniconda.sh#L16-L19

Without the --user flag it would install it to the miniconda directory.

[danpovey]

That script.was added for.a.very specific.purpose which I now forget. It is not at all a core part of Kaldi. It was for something nonstandard.

On Fri, Jan 10, 2020, 9:14 PM Christoph Boeddeker notifications@github.com wrote:

Another issue is, that install_miniconda.sh changes the user python environment. So you cannot have another python environment:

https://github.com/kaldi-asr/kaldi/blob/cbdbedefcdd47e02a685c1dc2d1b128c30bdf6b2/tools/extras/install_miniconda.sh#L16-L19

Without the --user flag it would install it to the miniconda directory.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/kaldi-asr/kaldi/issues/3835?email_source=notifications&email_token=AAZFLO3KY6CZLFIWLCSB5SDQ5BYFFA5CNFSM4KFHSQRKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEIT4BTY#issuecomment-573030607, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAZFLO4IFBM7PGOI72VLLSDQ5BYFFANCNFSM4KFHSQRA .

[boeddeker]

I think it was introduced in https://github.com/kaldi-asr/kaldi/pull/2142 to use https://github.com/fgnt/nn-gev . In CHiME6 it is also used for WPE based dereverberation.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[kkm000]

@boeddeker, so do you still believe this to be a problem? As Dan noted, miniconda is not an integral part of Kaldi, and the script exists for a particular recipe. If you use miniconda in in your own, it can be anywhere you like. Or am I missing the point?

[boeddeker]

@kkm000 yes, for me it is a problem, because I need to have kaldi independent of my user accont (i.e. sharing one kaldi root between different users) and interference with my own python environment is annoying.

• Some scripts use $HOME/miniconda3/bin/python, so in my home directory must be a miniconda3 folder. I would expect something like $KALDI_ROOT/tools/miniconda3/bin/python.
• Since the miniconda must be in the home directory, I cannot install two times kaldi without an interference between both installations (One kaldi may be much older and require different python package versions).
• I cannot install kaldi for all our users, because everyone has to run the installer for python
• The --user flag for pip is a bad idea, because it will influence also other python enviroments (All python environments share the user packages.)
• My home directory is very slow. When python is installed in the home directory, the startup time is very bad.
• My home directory is backuped. I do not need multiple GBs of backups from numpe etc.

Non of the point is very critical and I can work around many points (Symlinks, manual install, changing scripts, ...).

But since the way of installing this tool in kaldi seems to differ from other tools in kaldi and I like the idea how the other tools are installed, I opened this issue.

[kkm000]

Keep in mind that these scripts have been written with the inexperienced user in mind, with the chief goal to just get these users up and running. For one, we prefer to install everything in a location that is guaranteed to be accessible to the user without sudo rights. In other words, they exist to make the recipes "just work" for those who do not have any experience with the tools. We have never pursued a goal to make them universal to cover all sensible use cases, not even close.

Since you have an obviously advanced scenario (shared environments, both on a machine and across the team), and seem to be an advanced user who won't have a problem installing conda manually, I think it might be the best course of action for someone as experienced as you. Am I understanding your situation? If so, the scripts in tools/extras are just too simplistic for you--it's not their purpose. Do you think we;re on the same page now?

[boeddeker]

Yes, I know that it is sometimes good to accept "suboptimal" solutions to get any solution. So merging it for a for.a.very specific.purpose is completely fine.

My idea for this issue was to highlight that the current state is not ideal. Since other scripts started to use this python installation (e.g. CHiME 6), it may be worth to improve it.

I know that my points are special and I know that I cannot expect that every tool works out of the box in our environment. I thought, that all tools (except this) that I need in kaldi already support our workflow it would be nice, when this would also support our workflow. Especially, because it would bring the installation pattern closer to the other installation scripts.

At the time of the issue I thought about making a PR, once I know what the kaldi developer want to have. Currently, I have sadly not the time to do this. I could propose an alternative installation script, but I haven't the time to fix all egs that depend on the current installation script (I have no environment, where it is trivial to test them).