Stop forcing us to update

[andersonwinkler]

The title says it all. It seems the tool doesn't run if it detects a newer one has been released, forcing us to update. However, updating brings updates to other dependencies which in turn breaks other packages that are more important than nda-tools.

Please, don't force this anymore.

Running NDATools Version 0.2.16
Your version of nda-tools is out of date. Please upgrade to the latest version (0.2.21) from PyPi or GitHub and try again. 
    To upgrade using pip, run: 
pip install nda-tools==0.2.21

Btw, I can't stop getting amazed about how difficult it is to download ABCD and other NDA data. It has been 4 years since I started working with this data and it continues to be clunky, difficult, buggy, with outdated documentation in the NDA website, video tutorials that are no longer valid, and all sorts of issues.

Life is short, we have a lot of things to do, can't waste weeks trying to do something as mundane as downloading data.

Isn't the full time job of somebody to make this data accessible? Why does it have to be so lousy?

[effigies]

In the absence of a change in policy from NDAR, you could consider running with pipx, which installs nda-tools in a single-purpose virtual environment so that upgrading nda-tools does not infect your base (or other virtual) environment.

e.g.

$ pip install pipx
$ pipx install nda-tools

[jacob-lee]

Generally, creating a special purpose environment for this is the way to go, whether you use conda, venv, virtualenv, poetry or pipx; that makes updating relatively painless, since it doesn't effect anything else.

That said, it is fairly unusual for a command line tool to become un-operational because it's slightly out of date.

[andersonwinkler]

Thanks Chris and Jacob for the comments.

We use virtual environments for all projects. Each project has its own. For Python tools, recent ones use conda environments; older projects use virtualenv. Non-Python tools have their versions defined with lmod. Each project has its own environment variables, and everything is set up in a text file that is sourced before we start working on that project.

Surely we could have multiple Python environments for a given project. It'd be wasteful -- hundreds, thousands of files consuming a limited amount of inodes for a tool that is relatively minor and that does a task that shouldn't be complex. We've in fact once run out of inodes on Biowulf working on this very project because of the large number of files in the partition.

That said, it is fairly unusual for a command line tool to become un-operational because it's slightly out of date.

Absolutely, hence the frustration. It didn't have to be like this. I suppose hacking is the way to go, but did it have to be?

[ericearl]

@jacob-lee I think @andersonwinkler was concerned with downloading more-so than uploading. But that made me want to add the comment that we developed a tool when I was in the DCAN Labs for uploading to the NDA a little more smoothly (especially for BIDS data) and it is documented here:

https://ndabids.readthedocs.io/

It's certainly not bulletproof, but I've still been using it to upload BIDS data to the NDA outside of the DCAN Labs at the NIMH Data Science & Sharing Team.

[jrussell9000]

Bumping this issue based on the recent update. It's really unusual to force users to upgrade with each minor update to the code base - off the top of my head, I can't think of another python package does this. I've submitted a pull request to remove the sys.exit in the checkversion function.... #96

[lauragails]

@jrussell9000 How did you fix this? I tried downloading from the github, the pipx suggestion above, made new environments and have had no luck.

(I also emailed our hpc admins in case the issue is on my end)

[jrussell9000]

@lauragails (also.... @andersonwinkler) Sorry I missed this! The fix is to remove the 'sys.exit(1)' call from line 54 of NDATools/init.py - either in the source files before you build it, or on your system after you've installed nda-tools.