search

jcbird / ppv

Tools for dealing with SDSS-V plate files and plate runs.
BSD 3-Clause "New" or "Revised" License
1 stars 0 forks source link

Issue with installation of new version #17

Closed eleonorazari closed 3 years ago

eleonorazari commented 3 years ago

Hi John, me again.

I have an issue (or maybe 2) with the new installation of ppv.

The first one is after I give the command:

pip install -e .

It seems everything is working but I get the following message:

Obtaining file:///home/zari/ppv Installing collected packages: ppv Attempting uninstall: ppv Found existing installation: ppv 0.36 Uninstalling ppv-0.36: Successfully uninstalled ppv-0.36 Running setup.py develop for ppv Successfully installed ppv Traceback (most recent call last): File "/home/zari/anaconda3/envs/ppv/bin/pip", line 10, in sys.exit(main()) File "/home/zari/anaconda3/envs/ppv/lib/python3.8/site-packages/pip/_internal/cli/main.py", line 75, in main return command.main(cmd_args) File "/home/zari/anaconda3/envs/ppv/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 121, in main return self._main(args) File "/home/zari/anaconda3/envs/ppv/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 265, in _main self.handle_pip_version_check(options) File "/home/zari/anaconda3/envs/ppv/lib/python3.8/site-packages/pip/_internal/cli/req_command.py", line 149, in handle_pip_version_check session = self._build_session( File "/home/zari/anaconda3/envs/ppv/lib/python3.8/site-packages/pip/_internal/cli/req_command.py", line 90, in _build_session session = PipSession( File "/home/zari/anaconda3/envs/ppv/lib/python3.8/site-packages/pip/_internal/network/session.py", line 249, in init self.headers["User-Agent"] = user_agent() File "/home/zari/anaconda3/envs/ppv/lib/python3.8/site-packages/pip/_internal/network/session.py", line 159, in user_agent setuptools_version = get_installed_version("setuptools") File "/home/zari/anaconda3/envs/ppv/lib/python3.8/site-packages/pip/_internal/utils/misc.py", line 665, in get_installed_version working_set = pkg_resources.WorkingSet() File "/home/zari/anaconda3/envs/ppv/lib/python3.8/site-packages/pip/_vendor/pkg_resources/init.py", line 567, in init self.add_entry(entry) File "/home/zari/anaconda3/envs/ppv/lib/python3.8/site-packages/pip/_vendor/pkg_resources/init.py", line 623, in add_entry for dist in find_distributions(entry, True): File "/home/zari/anaconda3/envs/ppv/lib/python3.8/site-packages/pip/_vendor/pkg_resources/init.py", line 1983, in find_eggs_in_zip if metadata.has_metadata('PKG-INFO'): File "/home/zari/anaconda3/envs/ppv/lib/python3.8/site-packages/pip/_vendor/pkg_resources/init.py", line 1414, in has_metadata return self._has(path) File "/home/zari/anaconda3/envs/ppv/lib/python3.8/site-packages/pip/_vendor/pkg_resources/init.py", line 1854, in _has return zip_path in self.zipinfo or zip_path in self._index() File "/home/zari/anaconda3/envs/ppv/lib/python3.8/site-packages/pip/_vendor/pkg_resources/init.py", line 1731, in zipinfo return self._zip_manifests.load(self.loader.archive) File "/home/zari/anaconda3/envs/ppv/lib/python3.8/site-packages/pip/_vendor/pkg_resources/init.py", line 1688, in load mtime = os.stat(path).st_mtime FileNotFoundError: [Errno 2] No such file or directory: '/home/zari/anaconda3/envs/ppv/lib/python3.8/site-packages/ppv-0.36-py3.8.egg'

I am not sure this is actually an error but then, when I try to do the usual vetting:

from ppv.targets import Targets

I get the following error:

StopIteration Traceback (most recent call last)

in ----> 1 from ppv.targets import Targets ~/ppv/src/ppv/targets.py in 1 from . import groups ----> 2 from . import fiveplates 3 import numpy as np 4 from astropy.table import Table 5 from astropy.coordinates import SkyCoord ~/ppv/src/ppv/fiveplates.py in 15 from . import config 16 from .process import simulate_platedesign ---> 17 from .parse_platedata import main_platedata 18 import numpy as np 19 from astropy import units as u ~/ppv/src/ppv/parse_platedata.py in 6 plateruns = list(_fp_available) 7 ----> 8 platedata_tables = [io.load_fp_platedata(prun) for prun 9 in plateruns] 10 ~/ppv/src/ppv/parse_platedata.py in (.0) 6 plateruns = list(_fp_available) 7 ----> 8 platedata_tables = [io.load_fp_platedata(prun) for prun 9 in plateruns] 10 ~/ppv/src/ppv/data/io.py in load_fp_platedata(platerun, **table_kwds) 118 if five_plates makes further changes. 119 """ --> 120 pd_table = _load_commented_header(paths.fp_platedata(platerun)) 121 122 if platerun == '2020.08.x.bhm-mwm': # different table colnames ~/ppv/src/ppv/util/paths.py in fp_platedata(platerun) 136 fp_files(platerun)) 137 # Assuming only one file is retuned --> 138 pd_file = next(pd_file_s) 139 return fiveplates_platerun(platerun) / pd_file 140 StopIteration: I already removed the environment and installed everything again, but I still get the errors. Thanks, Eleonora
jcbird commented 3 years ago

This is likely an a versioning issue after moving to pip. I updated the setup.py to bump the version number and everything seems to be good to go. Steps to get it working:

1) Do a fresh pull of ppv 2) pip uninstall ppv ( just to make sure every last trace of old ppv is gone) 3) cd to directory of your local copy of ppv repo 4) pip install -e .

Let me know how that goes!

eleonorazari commented 3 years ago

I deleted the entire directory and cloned it again (and earlier in the day I also deleted the environment and created it again)

git clone https://github.com/jcbird/ppv.git

then cd ppv and

pip install -e .

Obtaining file:///home/zari/ppv Installing collected packages: ppv Running setup.py develop for ppv Successfully installed ppv

So it seems it's working fine, but:

import ppv -> works from ppv.targets import Targets -> same error as before.

jcbird commented 3 years ago

What version of python are you using?

On Mon, Nov 23, 2020 at 10:24 AM Eleonora notifications@github.com wrote:

I deleted the entire directory and cloned it again (and earlier in the day I also deleted the environment and created it again)

git clone https://github.com/jcbird/ppv.git

then cd ppv and

pip install -e .

Obtaining file:///home/zari/ppv Installing collected packages: ppv Running setup.py develop for ppv Successfully installed ppv

So it seems it's working fine, but:

import ppv -> works from ppv.targets import Targets -> same error as before.

— You are receiving this because you commented.

Reply to this email directly, view it on GitHub https://github.com/jcbird/ppv/issues/17#issuecomment-732269374, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABRADEYPROGXABU264TKLCLSRKECNANCNFSM4T7NDKCQ .

--

Jonathan Bird Stevenson Postdoctoral Fellow Department of Physics and Astronomy, Vanderbilt University

eleonorazari commented 3 years ago

3.8.6

jcbird commented 3 years ago

Just checking. Have you done a fresh pull of five-plates too?

On Mon, Nov 23, 2020 at 10:28 AM Eleonora notifications@github.com wrote:

3.8.6

— You are receiving this because you commented.

Reply to this email directly, view it on GitHub https://github.com/jcbird/ppv/issues/17#issuecomment-732272110, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABRADE7GUGDPOSAA3OV53B3SRKESLANCNFSM4T7NDKCQ .

--

Jonathan Bird Stevenson Postdoctoral Fellow Department of Physics and Astronomy, Vanderbilt University

eleonorazari commented 3 years ago

Yes, I did that. I am trying to install ppv environment and everything from scratch again.

eleonorazari commented 3 years ago

Nope, nada.

The only thing that comes to my mind is using my partner laptop, but it's not the most practical thing in the world.

jcbird commented 3 years ago

FOUND the bug! a critical file in five_plates was modified sometime in the last 24 hours in a unforseen way. Give me 30 minutes or less!

eleonorazari commented 3 years ago

Fingers crossed!

jcbird commented 3 years ago

Latest commit eebac58 should do the trick! Thanks for pushing on this and for the messages. Let me know.

Notes to myself and those who are interested: ppv gets the available five_plates plateruns by parsing https://github.com/sdss/five_plates/blob/master/plateruns/plateruns_description.txt

Sometime in the last 24 hours, that table was updated for the new BHM_03 run, but no BHM_03 plate_data exists yet, hence the failures. ppv can handle that situation gracefully now.

eleonorazari commented 3 years ago

Okay, it is almost working! Now I get the error below.

However if I use platerun_10y = ppv.fiveplates.Platerun('2020.10.y.mwm-bhm') everything works fine.

maybe_observed = OB_targets.available_in(platerun_bhm01) Please be patient. Initial target loading for Platerun can take up to 1 second per field. Loading target data from 18 Fields...

AttributeError Traceback (most recent call last) ~/ppv/src/ppv/fiveplates.py in fields(self) 420 try: --> 421 return self._fields 422 except AttributeError:

AttributeError: 'Platerun' object has no attribute '_fields'

During handling of the above exception, another exception occurred:

TypeError Traceback (most recent call last)

in ----> 1 maybe_observed = OB_targets.available_in(platerun_bhm1) ~/ppv/src/ppv/targets.py in available_in(self, pl_field_plrun) 140 if (isinstance(pl_field_plrun, groups.Platerun) | 141 isinstance(pl_field_plrun, fiveplates.Platerun)): --> 142 return self._available_in_platerun(pl_field_plrun) 143 return self._available_in_field(pl_field_plrun) # SAME logice for plate/field 144 ~/ppv/src/ppv/targets.py in _available_in_platerun(self, platerun_) 130 Initial target loading for Platerun can take up to 1 second per field. 131 Loading target data from {len(platerun_.fieldnames)} Fields...""", flush=True) --> 132 indx_all = [self.available_in(field) for field in platerun_.fields] 133 return np.bitwise_or.reduce(indx_all) 134 ~/ppv/src/ppv/fiveplates.py in fields(self) 421 return self._fields 422 except AttributeError: --> 423 self._fields = self.load_fields() 424 return self._fields 425 ~/ppv/src/ppv/fiveplates.py in load_fields(self) 403 404 def load_fields(self): --> 405 return [Field(fieldname, design_id=designID) for fieldname, designID in 406 zip(self.fieldnames, self.designIDs)] 407 ~/ppv/src/ppv/fiveplates.py in (.0) 403 404 def load_fields(self): --> 405 return [Field(fieldname, design_id=designID) for fieldname, designID in 406 zip(self.fieldnames, self.designIDs)] 407 ~/ppv/src/ppv/fiveplates.py in __init__(self, fieldname, design_id) 174 obstime=Time(self.epoch, format='decimalyear')) 175 self.platerun = self._get_platerun() --> 176 self._platedef_params = io.fp_platedef_params(self.platerun, 177 self.name, 178 self.designID) ~/ppv/src/ppv/data/io.py in fp_platedef_params(platerun, field, designID) 224 targetlists_zip = paths.fiveplates_targetlists(platerun) 225 params = {} --> 226 with ZipFile(os.fspath(targetlists_zip)) as tl_zip: 227 with tl_zip.open(paths.fiveplates_platedef(field, designID), 'r') as pldef: 228 rows = pldef.readlines() TypeError: expected str, bytes or os.PathLike object, not Column
jcbird commented 3 years ago

Ok, could you send the line where you define platerun_bhm01? I want to see what string is passed. Thanks.

On Mon, Nov 23, 2020 at 1:03 PM Eleonora notifications@github.com wrote:

Okay, it is almost working! Now I get the error below.

However if I use platerun_10y = ppv.fiveplates.Platerun('2020.10.y.mwm-bhm') everything works fine.

maybe_observed = OB_targets.available_in(platerun_bhm01) Please be patient. Initial target loading for Platerun can take up to 1 second per field. Loading target data from 18 Fields...

AttributeError Traceback (most recent call last) ~/ppv/src/ppv/fiveplates.py in fields(self) 420 try: --> 421 return self._fields 422 except AttributeError:

AttributeError: 'Platerun' object has no attribute '_fields'

During handling of the above exception, another exception occurred:

TypeError Traceback (most recent call last) in ----> 1 maybe_observed = OB_targets.available_in(platerun_bhm1)

~/ppv/src/ppv/targets.py in available_in(self, pl_field_plrun) 140 if (isinstance(pl_field_plrun, groups.Platerun) | 141 isinstance(pl_field_plrun, fiveplates.Platerun)): --> 142 return self._available_in_platerun(pl_field_plrun) 143 return self._available_in_field(pl_field_plrun) # SAME logice for plate/field 144

~/ppv/src/ppv/targets.py in available_in_platerun(self, platerun) 130 Initial target loading for Platerun can take up to 1 second per field. 131 Loading target data from {len(platerun_.fieldnames)} Fields...""", flush=True) --> 132 indx_all = [self.availablein(field) for field in platerun.fields] 133 return np.bitwise_or.reduce(indx_all) 134

~/ppv/src/ppv/fiveplates.py in fields(self) 421 return self._fields 422 except AttributeError: --> 423 self._fields = self.load_fields() 424 return self._fields 425

~/ppv/src/ppv/fiveplates.py in load_fields(self) 403 404 def load_fields(self): --> 405 return [Field(fieldname, design_id=designID) for fieldname, designID in 406 zip(self.fieldnames, self.designIDs)] 407

~/ppv/src/ppv/fiveplates.py in (.0) 403 404 def load_fields(self): --> 405 return [Field(fieldname, design_id=designID) for fieldname, designID in 406 zip(self.fieldnames, self.designIDs)] 407

~/ppv/src/ppv/fiveplates.py in init(self, fieldname, design_id) 174 obstime=Time(self.epoch, format='decimalyear')) 175 self.platerun = self._get_platerun() --> 176 self._platedef_params = io.fp_platedef_params(self.platerun, 177 self.name, 178 self.designID)

~/ppv/src/ppv/data/io.py in fp_platedef_params(platerun, field, designID) 224 targetlists_zip = paths.fiveplates_targetlists(platerun) 225 params = {} --> 226 with ZipFile(os.fspath(targetlists_zip)) as tl_zip: 227 with tl_zip.open(paths.fiveplates_platedef(field, designID), 'r') as pldef: 228 rows = pldef.readlines()

TypeError: expected str, bytes or os.PathLike object, not Column

— You are receiving this because you commented.

Reply to this email directly, view it on GitHub https://github.com/jcbird/ppv/issues/17#issuecomment-732362570, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABRADEZT6TBX2X5MIEKNOLDSRKWXXANCNFSM4T7NDKCQ .

--

Jonathan Bird Stevenson Postdoctoral Fellow Department of Physics and Astronomy, Vanderbilt University

eleonorazari commented 3 years ago

Here it is: platerun_bhm1 = ppv.fiveplates.Platerun('BHM_01')

jcbird commented 3 years ago

One more time! hahaha :)

Latest commit should fix this (> 18f516a) . Fresh pull and should be good to go.

Notes:

eleonorazari commented 3 years ago

okay give me 5 mins

eleonorazari commented 3 years ago

Okay! Everything is working :) Closing the issue!

jcbird commented 3 years ago

Woohoo! Thanks for your efforts !

On Mon, Nov 23, 2020 at 1:51 PM Eleonora notifications@github.com wrote:

Okay! Everything is working :) Closing the issue!

— You are receiving this because you commented.

Reply to this email directly, view it on GitHub https://github.com/jcbird/ppv/issues/17#issuecomment-732388831, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABRADE44QYDV3CTKSECSHKTSRK4LPANCNFSM4T7NDKCQ .

--

Jonathan Bird Stevenson Postdoctoral Fellow Department of Physics and Astronomy, Vanderbilt University