Simplify installing Caster.

[LexiconCode]

Caster needs to be easy to install. This is especially so for those that use this project out of necessity due to disabilities. Currently there are a lot of steps to install. This can be confusing as many of the installers/packages come from a variety of sources. These sources have their own instructions which may be out of date or difficult to find.

There are a number of approaches that would be functional.

• Updated video for install procedure.
• Text version of install instructions.
• Create a windows installer for Caster. Which contains/downloads the latest or required dependencies.

I'm of the mind to create an installer.

[LexiconCode]

Installation

Installing is now branch specific refer to the development or master branch instructions. (Recommended)

Troubleshooting FAQ

• When starting Dragon with NatLink Cannot load compatibility module support (GUID = {dd990001-bb89-1d2-b031-0060088dc929})).

  Install Microsoft Visual C++ 2010 Service Pack 1 x86, Reboot PC, and register regsvr32 manually on the natlink.pyd from an admin command prompt. Detailed information on Natlinks Troubleshooting Page.

• Running "Configure NatLink via GUI" does not bring up the settings window - try running the program as an administrator:

  1. Open an administrator command prompt by searching for "cmd" in start and either right click, run as administrator or ctrl-shift-enter.
  2. Change directory to the folder where start_configurenatlink.py was installed. This is likely to be C:\NatLink\NatLink\confignatlinkvocolaunimacro so the command would be cd C:\NatLink\NatLink\confignatlinkvocolaunimacro.
  3. Run python start_configurenatlink.py.

• To fix ImportError: No module named win32con Package win32con is out of date or not installed. Try pip install pywin32 Alternatively if the error persists use the Windows installer

• To fix lost sys.stderr For Caster must use pywin32 for system wide features, such as registering COM objects or implementing Windows Services, then you must run the following command from an elevated command prompt:

python C:\Python27\Scripts\pywin32_postinstall.py -install

• To fix ImportError: cannot import name RuleWrap You likely either have the wrong version of Dragonfly installed, or don't have it installed at all. RuleWrap is a Dragonfly import. Try pip uninstall dragonfly (it's okay if it doesn't find the package) then pip install dragonfly2.

• To fix ERROR:action.exec:Execution failed: Function(mouse_alternates): [Error 126] The specified module could not be found. Triggered by using the Legion command In order to use Legion, you may need to install Microsoft Visual C++ Redistributable Packages for Visual Studio 2013 (x86).

• To fix ImportError: DLL load failed: %1 is not a valid Win32 application. You're most likely using the 64-bit version of Python or 64-bit Python library.

  Uninstall Python 64-bit and delete C:\Python27 to remove any extraneous libraries. Then install or reinstall Python v2.7.X 32-bit Finally you will need to reinstall the caster dependencies as well.

• To fix UnicodeDecodeError: 'ascii' codec can't decode byte 0x83 in position 52: ordinal not in range(128)when trying to use words.txt.

  Dragonfly out of date or you're using the old dragonfly not dragonfly2 package. Run the following in command prompt one at a time.

  1. pip uninstall dragonfly to check and remove the old dragonfly package.
  2. pip install dragonfly2 or pip install --upgrade dragonfly2 to install or update Dragonfly

Extra information

Castor dependencies --installed by Dragonfly

• pywin32
• dragonfly2
• setuptools
• six
• pyperclip

--installed by Caster

• wxpython
• pillow
• toml

---

Alternative Speech Secognition Engines.

CMU Pocket Sphinx Install Instructions for dragonfly - Linux only - Currently not supported Caster but it is a work in progress.

[Versatilus]

I like this idea, but I do have a few comments for now.

• I don't think psutil is required anymore. At this point, I can't even remember what it was used for.
• I would like to target the fork of Dragonfly by @Danesprite, though I'm a little worried this will cause confusion until the Sphinx engine supports all of Caster's features. Please correct me if I'm wrong about its currents feature level.
• At this point, 2.7 is the oldest version of Python I'm interested in supporting.
• Similarly, I only have interest in supporting Unicode, so DNS versions older than 12 (or is that 11?) are out.

I'm sure I'll have more comments later.

[LexiconCode]

I've talked to Danesprite about the status of Sphinx and Caster

Well, you shouldn't have to worry about engine implementations in caster; all engines should implement a compiler class to convert dragonfly grammars, rules and elements into a format that the engine can use. Having said that, I have noticed WSR specific code in the Nexusclass and in other places. Maybe there would need to be Sphinx configuration options too.

With the Sphinx engine, the main issue is that Repetition and Dictation don't work together yet. This is mostly a problem with the pyjsgf expansion matching code, which needs a revamp. This would still be with utterance pauses, but the engine should properly segment rules from caster to be recognised in sequence. I'll be doing some testing with caster and Sphinx soon.

Sphinx status doesn't really matter with his fork of dragonfly when it comes to utilizing it for Caster. Sphinx back-end doesn't have to be promoted until it's ready. _sphinx_moduleloader.py has to be specifically executed in a location where there's grammars.

If installer was created the dragonfly fork would be completely hidden from the user unless they bothered to look at the code after it was installed. All that being said I think we could use it now without confusing the user.

As for your other points I'm in agreement.

[Versatilus]

I'm supportive of whatever steps need to be taken to take Sphinx support from theory into reality. Due to the nature of my disability, I am unable to test anything directly at this time, but I'll help in whatever other ways I can.

In terms of WSR support, I'd like to look for some sort of existing application to host Caster for use with WSR. From what I understand, the WSR main program is just a thin shell around the actual speech engine, which is exposed to .NET. It has some pretty bad bugs that make it unusable. Considering how long those bugs have been there, I'm not going to hold my breath waiting for them to be fixed.

[LexiconCode]

Excellent, I think I got the testing in hand. We each focus or skill set and energy as we have time.

I'd like to look for some sort of existing application to host Caster for use with WSR

Could you expand a little bit on this when you have time?

Microsoft really has not poured development time into upgrading WSR engine since Windows Vista.

[shippy]

I've had two things on my wishlist:

1. Get the whole Natlink / dragonfly / caster contraption working in a conda environment (and thus be installable with a conda environment.yml, which is a superset of pip). In principle, getting things to work with conda instead of the Natlink-distributed zip of Python installer + packages shouldn't be difficult, but I just haven't been able to make it work.
2. Create a chocolatey / OneGet installation recipe.

[LexiconCode]

@shippy Thanks for the feedback! Some notes

Myth 5. conda doesn't work with virtualenv, so it's useless for my workflow

Reality: You actually can install (some) conda packages within a virtualenv, but better is to use Conda's own environment manager: it is fully-compatible with pip and has several advantages over virtualenv.

virtualenv/venv are utilites that allow users to create isolated Python environments that work with pip. Conda has its own built-in environment manager that works seamlessly with both conda and pip, and in fact has several advantages over virtualenv/venv:

conda environments integrate management of different Python versions, including installation and updating of Python itself. Virtualenvs must be created upon an existing, externally managed Python executable. conda environments can track non-python dependencies; for example seamlessly managing dependencies and parallel versions of essential tools like LAPACK or OpenSSL Rather than environments built on symlinks – which break the isolation of the virtualenv and can be flimsy at times for non-Python dependencies – conda-envs are true isolated environments within a single executable path. While virtualenvs are not compatible with conda packages, conda environments are entirely compatible with pip packages. First conda install pip, and then you can pip install any available package within that environment. You can even explicitly list pip packages in conda environment files, meaning the full software stack is entirely reproducible from a single environment metadata file.

I'll have to look into a chocolatey / OneGet installation recipe.

What are some issues you've experienced trying to get dependencies to run within conda?

[LexiconCode]

I propose we should use pyinstaller Overall Pyinstaller focuses on cross-platform and Python deployment even without the Python installed. If the consensus is to move forward with Pyinstaller I will clean up the thread and update the opening post.

Documentation Tutorial Supplemental resource

Some significant features of pyinstaller.

Packaging of Python programs into standard executables, that work on computers without Python installed.
Multi-platform: works under
    Windows (32-bit and 64-bit),
    Linux (32-bit and 64-bit),
    Mac OS X (32-bit and 64-bit),
    contributed suppport for FreeBSD, Solaris, HPUX, and AIX.
Multi-version: supports Python 2.7 and Python 3.3—3.6.
Flexible packaging mode:
    Single directory: build a directory containing an executable plus all the external binary modules (.dll, .pyd, .so) used by the program.
    Single file: build a single executable file, totally self-contained, which runs without any external dependency.
    Custom: you can automate PyInstaller to do whatever packaging mode you want through a simple script file in Python.
Explicit intelligent support for many 3rd-packages (for hidden imports, external data files, etc.), to make them work with PyInstaller out-of-the-box (see Supported Packages).
Full single-file Egg-support: required .egg files are automatically inspected for dependencies and bundled, and all the egg-specific features are supported at runtime as well (entry points, etc.).
Partial directory Egg-support: required .egg directories are automatically inspected for dependencies and bundled, but egg-specific features will not work at runtime.
Automatic support for binary libraries used through ctypes (see the ctypes Dependencies Support for details).
Support for automatic binary packing through the well-known UPX compressor.
Optional console mode (see standard output and standard error at runtime).

And many other features

Side notes Can be integrated on a basic level to virtualenv pyinstaller can be used within conda pyinstaller

Evaluation of the following systems Chocolatey / OneGet

Chocolatey is a Windows only packaging and distribution system. For our purposes we need something that is cross-platform. We really need to maintain one installer and packaging system. Many of the Windows specific features that we need are found within pyinstaller.

OneGet is a Microsoft product that is very interesting but Windows only. OneGet still has to be installed before use in addition to completely different scripting system to implement packages. Looking over the documentation seems complicated and I think this sums up some misconceptions about what OneGet trying to achieve.

Both of the systems focus on Windows not built for Python deployment.

Thoughts?

[BazookaMusic]

since I have some free time and I need to format my laptop making this work would be great. before using pyinstaller we should also consider actually creating a setup.py for caster itself so that caster can have a normal directory structure. For example, caster/lib contains files which the user will not normally modify. Thus it could be installed as a python package like dragonfly, which would Fix all the annoying import errors.After that we can package the whole thing using pyinstaller.

about that, should the installer also deal with natlink installation?

[LexiconCode]

@gerrish I apologize I haven't had the time to pursue this like I wanted.

I agree creating a standard directory structure via setup.py. Ideally the installer pyinstaller would handle natlink. I like the idea of packaging caster/lib. The end product should be an installer that is simple, complete and extensible. Eventually we need to be able to add in alternate backends such as Google Voice or Pocket Sphinx. I think we should start off with our current dependencies for Castor.

Since were talking about directory structure caster\bin\data and any other config should be stored in the outside of the project directory. That makes the project directory disposable if there is no edits to the source code. It also would allow for multiple castor project directories using the same config. The default path could be set within settings.json.

Maybe C:\Users\%USERNAME%\AppData\Local\Caster\data and the equivalent for Linux ~/.Caster/data Plus we should move or merge filter rules to datawhich are stored in \caster\user. Perhaps we should use user instead of data thinking down the road to possibly having profiles? An example could beC:\Users\%USERNAME%\AppData\Local\Caster\user\"user name"

[LexiconCode]

Caster as a Python package #246

[synkarius]

Looking over settings.py, it seems to me that at least

• custom rules/filters
• the TOML files,
• Sikuli files,
• configdebug.txt

should be moved to the user space. Is there anything I'm missing?

[LexiconCode]

@synkarius I do not believe your missing anything.

[synkarius]

@LexiconCode @Versatilus That commit just now moves the config files, words.txt, configdebug.txt, and the location for Sikuli scripts outside of the installation directory. (They will now go into a ".caster" directory in the user's home directory, as discussed above. For me, that's C:\Users\synkarius\.caster.) This puts us on the path to configure Caster to be installable via pip like Dragonfly is.

Note: If you don't move your files to the new location in advance, it will generate new files. The old files will remain in the old location but will be ignored until you copy them over.

Next I'll get the user rules and filters moved. And Sikuli scripts.

Then, I'll see what the configuration looks like to get this installable via pip. Doing away with import errors and dependency warnings via pip is a great idea. Thank you @gerrish.

[LexiconCode]

Looks great! Thank you for your work on this.

[LexiconCode]

https://github.com/dictation-toolbox/caster/pull/374 fulfills the pip package and separates the user space.