sklearn breaks installer

[aaronchantrill]

Description

When I run the installer, it ends with the following error:

Collecting sklearn
  Downloading sklearn-0.0.post1.tar.gz (3.6 kB)
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [18 lines of output]
      The 'sklearn' PyPI package is deprecated, use 'scikit-learn'
      rather than 'sklearn' for pip commands.

      Here is how to fix this error in the main use cases:
      - use 'pip install scikit-learn' rather than 'pip install sklearn'
      - replace 'sklearn' by 'scikit-learn' in your pip requirements files
        (requirements.txt, setup.py, setup.cfg, Pipfile, etc ...)
      - if the 'sklearn' package is used by one of your dependencies,
        it would be great if you take some time to track which package uses
        'sklearn' instead of 'scikit-learn' and report it to their issue tracker
      - as a last resort, set the environment variable
        SKLEARN_ALLOW_DEPRECATED_SKLEARN_PACKAGE_INSTALL=True to avoid this error

      More information is available at
      https://github.com/scikit-learn/sklearn-pypi-package

      If the previous advice does not cover your use case, feel free to report it at
      https://github.com/scikit-learn/sklearn-pypi-package/issues/new
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

[notice] A new release of pip is available: 23.0 -> 23.0.1
[notice] To update, run: pip install --upgrade pip
Notice: Error installing python requirements:

4308

To me, this is just saying that we should replace "sklearn" with "scikit-learn" in python-requirements.txt.

I tried git-cloning a copy of Naomi and updating that line in python-requirements.txt, but every time I try to install Naomi now, it gives me the following choice:

=========================================================================
Install ...
=========================================================================

It looks like you already have Naomi installed.
To start Naomi just type 'Naomi' in any terminal.

Running the install process again will create a backup of Naomi
in the '~/.config/naomi-backup' directory and then create a fresh install.
Is this what you want?

Choice [Y/N]:

If I select 'Y'es, it appears to delete the ~/Naomi directory and then errors out:

  1) Use the recommended ('Stable')
  2) Monthly releases sound good to me ('Milestone')
  3) I'm a developer or want the cutting edge, put me on 'Nightly'
Choice [1-3]: S - Skipping Section

find: ‘/home/achantrill/Naomi’: No such file or directory
find: ‘/home/achantrill/Naomi’: No such file or directory
find: ‘/home/achantrill/Naomi/installers’: No such file or directory
/dev/fd/63: line 255: cd: /home/achantrill/Naomi: No such file or directory
/dev/fd/63: line 257: cd: /home/achantrill/Naomi: No such file or directory
...
sudo: ./naomi_apt_requirements.sh: command not found
Notice: Error installing apt packages

At this point, it does not appear that I can use the installer to install a manually downloaded or modified version of Naomi.

I tried to get around this by setting the SKLEARN_ALLOW_DEPRECATED_SKLEARN_PACKAGE_INSTALL variable from the first error message, but then get an error about automake-1.15 not existing on my system (automake-1.16 does exist on my system) when attempting to build OpenFST. At this point I need to look at the whole install process and see how I can simplify things.

Expected Behavior

If I git-clone a copy of Naomi, then cd to the ~/Naomi directory and run ./naomi-setup.sh, it should allow me to bypass downloading a new copy of Naomi by pressing the "S"-Skip character at Version selection.

Actual Behavior

It appears that the installer deletes the ~/Naomi directory when you select "Install" from the first menu, so there is no directory structure if you then select 'S'kip at version selection.

Possible Fix

When installing, don't delete the ~/Naomi directory until after the user selects a version to install. Update the "sklearn" line in python_requirements.txt with "scikit-learn" Re-do the install process. See if it is still necessary to install OpenFST manually.

Steps to Reproduce

1. Using git, create a clone of the Naomi directory structure
2. Run naomi-setup.sh in the newly cloned directory
3. Select "Install"
4. Select "1" - proceed uninterrupted
5. Select "S"kip

Context

I'm trying to install Naomi. I am installing under Windows Subsystem for Linux (WSL) on Debian Bullseye. I don't know if these issues affect every version of Bullseye. I'm going to try again on a Bullseye virtual machine.

Your Environment

• Environment name and version (e.g. PHP 5.4 on nginx 1.9.1): Python 3.9.2
• Operating System and version: Debian Bullseye running on WSL on Windows 11

[Akul2010]

I think this is probably happening because Naomi detects a previously made profile in the same system.

Also, the initial error log says this:

      - if the 'sklearn' package is used by one of your dependencies,
        it would be great if you take some time to track which package uses

I think this means that one of the other packages you are using might have sklearn listed as a dependency.

[AustinCasteel]

The installer says exactly what it did. You forced an install ontop of one that already exists so it backed up what was there and then started fresh.

If you start entirely from scratch you can do an install with oneliner for example. Or if you have source then you can go through the process and skip the version select in this case to use the source you already have, put in for developers so that dev work is not overwritten. However, there is a missed case where if you try to install naomi while it is already installed and there is a profile & configs, it will instead back everything up, and then do the version select to get the version you want. But since you are testing the pkg issue with configs it is backing up your changes in an effort to clean slate and then when you skip the version select there is no longer source in the proper dir.

This is definitely a bug that honestly I am surprised hasn't popped up this whole time since the installer overhaul. Now how to fix this is a little complicated. If someone has naomi completely installed & working but then tries to reinstall without affecting the profile & configs, then the config will be out of sync, and depending on the naomi version the profile might be broken. So I am not sure how we should go about that.

[aaronchantrill]

@Akul2010 It is the NaomiSTTTrainer.py module that requires sklearn. That module is there to help people adapt their acoustic models using their own recordings. The goal is to eventually find the closest model to the user's voice to allow better adaptation to accents, speech impediments, etc. I just need to update the line in python-requirements.txt and test. I am surprised that pip actually returns an error condition because of this, though.

[aaronchantrill]

@AustinCasteel Thanks for thinking about this. It does provide me some strong encouragement to try to optimize the installation procedure, as far as minimizing the number of programs that have to be compiled, which is something I've been avoiding. I certainly appreciate all the work that's gone into getting Naomi to install automatically.

[aaronchantrill]

@AustinCasteel I see what's happening now. It is the existence of the ~/.config/naomi folder that is triggering the different behavior. I was confused because that used to not be created until running Naomi with --repopulate. If I delete the ~/.config/naomi folder before re-running the installer, I can skip downloading Naomi.

[aaronchantrill]

Fixed with "Fix record and playback during install" pull request #376