BUG: tab command completion is failing

[prjemian]

With current environment, tab completion is failing for requests after the second .. How to fix?

[prjemian]

This came up before. One of the packages caused this. Which?

[prjemian]

• https://duckduckgo.com/?q=ipython+tab+completion+fails&t=brave&ia=web
• https://stackoverflow.com/questions/32183369/ipython-tab-completion-broken (>7y)
• https://stackoverflow.com/questions/2603798/ipython-tab-autocomplete-does-not-work-on-imported-module (>13y)

not sure any of these will help this problem

[prjemian]

• https://duckduckgo.com/?q=ipython+tab+completion+is+failing+after+second+.&t=brave&ia=web
• https://stackoverflow.com/questions/40536560/ipython-and-jupyter-autocomplete-not-working (>6y) looks familiar

Particularly: https://stackoverflow.com/a/64554305/1046449

A possible reason a user may believe that autocomplete is not working may be that autocomplete is just taking too long. Circa 2020-11-27 this is particularly true for Pandas when operating with jedi in a Jupyter notebook environment.

The issue can be solved by using the following magic which deactivates jedi

%config Completer.use_jedi = False

See the SO link for more details.

[prjemian]

Maybe also version specific with jedi: pip install --upgrade jedi==0.17.2 (earlier versions presented this failure)

Install today (total anaconda installation) has:

(base) [bcda-rhel9 /APSshare/anaconda3] conda list jedi
# packages in environment at /APSshare/anaconda3/x86_64:
#
# Name                    Version                   Build  Channel
jedi                      0.18.1          py310h06a4308_1  

[prjemian]

Also, in same SO page:

For anyone arriving here with an "older" version of ipython, there was a bug with the Jedi dependency, (0.18.0 doesn't work, 0.17.2 does).

[prjemian]

Investigate jedi

Note: Current tag of jedi is 0.18.2

[prjemian]

pin jedi <0.18 installed 0.17.2 -- Same problem still.

[prjemian]

pin jedi >=0.18.2 installed 0.18.2 -- same problem still

[prjemian]

conclusion: failing tab completion is not due to jedi version

[prjemian]

On suggestion, disable jedi in the IPython session altogether: %config Completer.use_jedi = False -- Same problem still

[prjemian]

Investigate readline

(bluesky_2023_2) prjemian@zap:~/training$ conda list readline
# packages in environment at /home/prjemian/.conda/envs/bluesky_2023_2:
#
# Name                    Version                   Build  Channel
readline                  8.2                  h8228510_1    conda-forge

[prjemian]

That's the most recent conda-forge package, last update a month ago. Reports (undocumented) of this problem started before that. Try pinning older versions from conda-forge channel.

pin	installed	result
readline ==8.1.2	8.1.2	ImportError: cannot import name 'file_log_handler' from 'apstools.utils'
readline ==8.1.0	8.1.0	Still same problem (no tab completion after second .)

[prjemian]

Conclusion: failing tab completion is not due to readline version

[prjemian]

This issue seems related, but to older IPython 7. This issue involves IPython 8.13.2.

[prjemian]

Test this out in a very clean environment, such as a fresh OS install inside a Docker container. No extra environment variables and such.

[prjemian]

Use a simple test code with no import requirements:

class Data:
    thing = [1, 2, 3]
    that = dict(a="a", b="b"*8)

class Thing1:
    ying = "one"
    yang = "two"
    data = Data()

class Thing2:
    appen = 3
    zell = 4
    seuss = Thing1()

ogden = Thing2()

Interaction

>>> ogden.<TAB>
ogden.appen  ogden.seuss  ogden.zell   
>>> ogden.seuss.<TAB>
ogden.seuss.data  ogden.seuss.yang  ogden.seuss.ying  
>>> ogden.seuss.data<TAB>
ogden.seuss.data
>>> ogden.seuss.data.th<TAB>
ogden.seuss.data.that   ogden.seuss.data.thing  
>>> ogden.seuss.data.that<CR>
{'a': 'a', 'b': 'bbbbbbbb'}
>>> ogden.seuss.data.thing<CR>
[1, 2, 3]

[prjemian]

First test environment: Python3 from Miniconda3 base environment in Docker container

```Dockerfile FROM continuumio/miniconda3 LABEL Name="sandbox" USER root CMD ["/bin/bash"] SHELL ["/bin/bash", "-c"] WORKDIR /home # ------------------------------ docker developer UI ENV PROMPT_DIRTRIM=3 ```

Response of this configuration is shown above. Result: tab completion working at all levels tested.

[prjemian]

Test 2: Test with IPython

Starting from Dockerfile above: conda install -y ipython

In [2]: ogden<TAB> ()
In [2]: ogden.<TAB> (appen, seuss, zell)
In [2]: ogden.seuss.<TAB> (data, yang, ying)
In [2]: ogden.seuss.data.<TAB> (that, thing)
Out[2]: [1, 2, 3]

Result: tab completion working at all levels tested.

[prjemian]

Test 3: add readline==8.1.2

same results

Result: tab completion working at all levels tested.

[prjemian]

Test 3: use bluesky_2023_2 environment and IPython

Dockerfile

```Dockerfile FROM continuumio/miniconda3 LABEL Name="sandbox" USER root CMD ["/bin/bash"] SHELL ["/bin/bash", "-c"] WORKDIR /home # ------------------------------ docker developer UI ENV PROMPT_DIRTRIM=3 RUN conda install -y conda-libmamba-solver ADD \ https://raw.githubusercontent.com/BCDA-APS/bluesky_training/main/bluesky/environments/environment_2023_2.yml \ ./env.yml RUN \ conda env create \ --force \ -n bluesky_2023_2 \ -f ./env.yml \ --solver=libmamba ```

Then, in the container:

1. conda activate bluesky_2023_2
2. ipython
3. paste the test code

same results

Result: tab completion working at all levels tested.

[prjemian]

Summary (so far) with testing in Docker container: No problem observed.

[prjemian]

Test 4: use bluesky_2023_2 environment and run blueskyStarter.sh

Dockerfile

```bash FROM continuumio/miniconda3 LABEL Name="sandbox" USER root CMD ["/bin/bash"] SHELL ["/bin/bash", "--login", "-c"] ENV HOME /root WORKDIR "${HOME}" # ------------------------------ docker developer UI ENV \ PROMPT_DIRTRIM=3 \ BLUESKY_CONDA_ENV=bluesky_2023_2 \ BLUESKY_DIR="${HOME}/bluesky" \ IPYTHON_DIR="${HOME}/.ipython" RUN conda install -y conda-libmamba-solver ADD \ https://raw.githubusercontent.com/BCDA-APS/bluesky_training/main/bluesky/environments/environment_2023_2.yml \ ./env.yml RUN \ conda env create \ --force \ -n "${BLUESKY_CONDA_ENV}" \ -f ./env.yml \ --solver=libmamba \ && conda init bash RUN conda env list RUN which python RUN echo "${BLUESKY_CONDA_ENV}" # ref: https://pythonspeed.com/articles/activate-conda-dockerfile/ # Too bad an env var (${BLUESKY_CONDA_ENV}) fails when used here SHELL ["conda", "run", "-n", "bluesky_2023_2", "/bin/bash", "-c"] RUN which python RUN ls -lAFgh "${HOME}/" ADD \ https://raw.githubusercontent.com/BCDA-APS/bluesky_training/main/new_bluesky_instrument.py \ "${HOME}/" RUN python "${HOME}/new_bluesky_instrument.py" "${BLUESKY_DIR}" RUN mkdir -p "${IPYTHON_DIR}" RUN ipython profile create bluesky --ipython-dir="${IPYTHON_DIR}" RUN /bin/cp \ "${BLUESKY_DIR}/console/__start_bluesky_instrument__.py" \ "${IPYTHON_DIR}/profile_bluesky/startup/" RUN sed -i s:ipython-bluesky:ipython:g "${BLUESKY_DIR}/blueskyStarter.sh" WORKDIR "${BLUESKY_DIR}" ```

same results

Result: tab completion working at all levels tested.

[prjemian]

Next steps

Look at user account's environment setup in problematic configurations.

Continue with modifications to Dockerfile.

[prjemian]

Test 5: same conda environment on new linuxmint user account with minimal environment configuration

same results

Result: tab completion working at all levels tested.

[prjemian]

Still not obvious what will resolve this.