pip installable

[NickSwainston]

Is it possible to make this repo pip installable since it already has a setup.py?

I would make it myself but I don't think it makes sense for me to use my pypi account

[johnsmorgan]

I think that's a good thing to aspire to but I think a lot of work needs to be done to reach that stage. In particular, the installed scripts need to be checked to ensure that everything is installed that needs to be installed. It seems strange to me that install.py installs make_beam_test.py but not make_beam.py. It would also be good if the required .hdf5 file were also automatically installed as part of the process.

[NickSwainston]

This is perhaps a separate issue but it may be worth making make_beam_test.py into a standard test script so you can be used when running python setup.py test or rename it as setup.py will automatically grab any script that starts or begins with "test".

[NickSwainston]

I think I've found a solution for installing the hdf5. https://git-lfs.github.com/ we can use this tool to make a pointer to where the file is stored and then will be able to be installed with pip

[andreww5au]

Hi Nick, I've just discovered that this change might work for when you install it with pip (I haven't tried), but it silently breaks the install if you 'git clone' the repo.

The .h5 file exists (and it installs without an error), but when you try and use the package, it throws an exception because the .h5 file is 134 bytes long, and contains this:

---

version https://git-lfs.github.com/spec/v1 oid sha256:a7649c6e03b8128a1de4614c2f363af5fa44f3890ae27bf893d56ca337bc48ee size 133209592

Even better, the real .h5 file that I had in the checked-out repo was overwritten by this stub file.

Andrew

On 2020-07-06 11:34 AM, Nick Swainston wrote:

I think I've found a solution for installing the hdf5. https://git-lfs.github.com/ we can use this tool to make a pointer to where the file is stored and then will be able to be installed with pip

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/MWATelescope/mwa_pb/issues/6#issuecomment-653997296, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABPFUBQ5JCID34QC4ALUXVDR2FA3ZANCNFSM4JCC7WDA.

[NickSwainston]

How are you installing mwa_pb? If you git clone the repo then use pip install . It installs the .h5 file correctly (it's 128 Mb in the install directory)

[andreww5au]

On 2020-07-18 8:36 AM, Nick Swainston wrote:

How are you installing mwa_pb? If you git clone the repo then use |pip install .| It installs the .h5 file correctly (it's 128 Mb in the install directory)

Hi Nick, that still doesn't work. I've done some digging - it's not pip that does the LFS file download, it's the git LFS package that does it, and only if the user has installed and initialised git-lfs in their personal account.

So the sequence would have to be:

$ apt-get install git-lfs

or the equivalent on a Mac (no idea about Pawsey machines)

Then (for each user account that needs to clone/pull mwa_pb):

$ git lfs install

Then:

$ git clone git@github.com:MWATelescope/mwa_pb.git $ cd mwa_pb

Then either 'pip install .' or 'python setup.py install'

If you haven't both installed the git-lfs package, and done the 'git lfs install' to set up the right git 'hooks' for that local user, the .h5 file stub will download (and silently overwrite any existing actual data file) and be installed as a 134 byte text file.

Wouldn't it be better to just add a couple of lines in setup.py to 'wget' a fresh copy of the .h5 file on every install, instead of requiring git-lfs to be installed and configured for every user?

Andrew

[NickSwainston]

I have also accounted for this issue when trying to install on Galaxy.

I agree that git-lfs seems to be inconvenient so we should use a wget command. I think the best place to do that is in config.py and download it the h5 files does not exist.

I can do this once Greg has put the file on ws.mwatelescope.org

[PaulHancock]

This package is now available on pypi.org, but it doesn't install for me. Pulling from github and installing does work (and I have git-lfs already enabled).

[NickSwainston]

I've had the same problem and I'm pretty sure it's due to version automation I put in the setup.py. It tries to get a version number from the tags but it fails because I don't think pypi keeps the git info. So either we have to ditch the automation or try and work out if we can fix the pypi issue.

[NickSwainston]

Pretty sure I fixed it #17

[PaulHancock]

Can confirm that it will pip install on my machine.

[NickSwainston]

Excellent