`yt search` fails due to argcomplete version conflict

[BlackstoneEngineering]

Problem

I updated pip (8.1.0) then installed yotta (0.14.2) on a fresh Ubuntu 15 machine. When I run yt search it errors out due to version conflicts with argcomplete

yt search something
Fatal Exception, yotta=unknown
Traceback (most recent call last):
  File "/usr/local/bin/yt", line 4, in <module>
    yotta.main()
  File "/usr/local/lib/python2.7/dist-packages/yotta/main.py", line 201, in main
    status = args.command(args, following_args)
  File "/usr/local/lib/python2.7/dist-packages/yotta/search.py", line 125, in execCommand
    for result in itertools.islice(registry_access.search(query=args.query, keywords=args.kw, registry=args.registry), args.limit):
  File "/usr/local/lib/python2.7/dist-packages/yotta/lib/registry_access.py", line 683, in search
    headers = _headersForRegistry(registry)
  File "/usr/local/lib/python2.7/dist-packages/yotta/lib/registry_access.py", line 414, in _headersForRegistry
    'X-Yotta-Client-Version': _getYottaVersion(),
  File "/usr/local/lib/python2.7/dist-packages/yotta/lib/registry_access.py", line 397, in _getYottaVersion
    _yotta_version = pkg_resources.require("yotta")[0].version
  File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 943, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 835, in resolve
    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (argcomplete 1.1.0 (/usr/local/lib/python2.7/dist-packages), Requirement.parse('argcomplete<1.0,>=0.8.0'), set(['yotta']))

Solution

Need to update / fix yotta so it can use newer versions of the argcomplete package as that is the new default for newer systems.

Alternatively we could force all users to install yotta in a VE, but that seems impractical due to the scope of yotta.

[autopulated]

Thanks for reporting – do you know if things work OK with argcomplete 1.1.0 if you relax the restriction in setup.py?

[bogdanm]

I've also had quite a few issues with updating yotta, caused by problems similar to this one. Since pip seems to have problems with updating dependencies, I'm wondering if yotta shouldn't pin dependency number (argcomplete==1.1.0 and so on) instead of specifying a range of dependencies.

[autopulated]

We don't pin them, we specify compatible ranges. In this case the range is argcomplete<1.0,>=0.8.0

Please report any issues like this, so we can keep the ranges up to date!

[bogdanm]

We don't pin them, we specify compatible ranges

I know, and I'm saying that maybe we shouldn't. Not necessarily because of the way we specify dependency ranges, but because lots of times pip behaves unpredictably when I do pip install -U yotta (one or two times it behaved so unpredictably that I had to reinstall Python completely on my Windows box). Also, if you pin versions, it's much easier for people to reproduce the same yotta version you published. Updating the pinned numbers can happen when a new yotta version is released, for example. Just my £0.02.

[autopulated]

Ah double negative :)

Pinning exact versions will mean requiring installation in a virtual env, otherwise conflicts are inevitable. While reproduction of the exact version is a nice idea, I think it is less important than making sure people get the latest bugfixes (and potentially security updates) from dependencies when they install.

Basically the same reasoning that applies to dependency versions within yotta itself.

I can't recall an instance where an updated dependency broke yotta by breaking functionality* (only cases like this were we should have been more relaxed in the versions we allow updates to ;)

* well not quite true, there was one case where a dependency update broke a test case in the CI (the Travis CI will use the latest matching versions of dependencies on each build)

[bogdanm]

You're right, I should've mentioned venvs earlier. I think that's a good idea. I'm not talking about updates that broke yotta, I'm talking about updates that brought pip and hence Python. That actually happened :)

[BlackstoneEngineering]

@autopulated I have not tested further

[autopulated]

should be fixed by #743