Adding both components of a close visual binary in the Object database

[aleciane]

Hi !

I am trying to add the two components of VV CrA that I have observed recently. The objects have been observed independently with SPIRou and I gave them the names VV CrA NE and VV CrA SW. However those names are not known into SIMBAD, and both objects are not independently known in SIMBAD. Both are linked to the same 2MASS number, because they are only separated by 1.4 arcsec. How can add them independently with apero_astrometrics.py ?

Thanks ! Evelyne

[njcuk9999]

Hi. Sorry about the lag in answering this message. Im in and out of work at the moment.

Hmm, this is an interesting one. We probably have to add one of the targets (the "main" one) via the simbad name and the other manually (withiut any of the aliases). We can take any properties from the "main" one and use those for the other or update any values accordingly. How does Gaia deal with these targets? Are they a single object? Worst case, you don't enter them on apero, and they will use all the header values, or we enter both targets completely manually (i.e. you supply the ra,dec,pm,teff etc)

[aleciane]

Hi !

So I tried to add the main one via the simbad name, but once I have answered to all the questions, I get the following error message (I use APERO installed on the LAM computer, the same as Andres):

09:38:21.799-|ASTROM| * 09:38:21.812-|ASTROM| Updating pending-list online database 09:38:21.825-*|ASTROM| 09:38:21.956-!!|ASTROM| E[01-010-00001]: Unhandled error has occurred: Error <class 'AttributeError'> 09:38:21.957-!!|ASTROM| 09:38:21.957-!!|ASTROM| Traceback (most recent call last): 09:38:21.957-!!|ASTROM| File "/net/GSP/nas12c/big_spirou/APERO_v07288/apero-drs/apero/core/utils/drs_startup.py", line 433, in run 09:38:21.958-!!|ASTROM| llmain = func(recipe, params) 09:38:21.958-!!|ASTROM| File "/net/GSP/nas12c/big_spirou/APERO_v07288/apero-drs/tools/bin/apero_astrometrics.py", line 195, in main 09:38:21.959-!!|ASTROM| drs_astrometrics.add_obj_to_sheet(params, add_objs) 09:38:21.959-!!|ASTROM| File "/net/GSP/nas12c/big_spirou/APERO_v07288/apero-drs/apero/tools/module/database/drs_astrometrics.py", line 1145, in add_obj_to_sheet 09:38:21.960-!!|ASTROM| google_sheet = gspd.spread.Spread(sheet_id) 09:38:21.960-!!|ASTROM| File "/home/external/ealecian/.conda/envs/apero-env-v07288/lib/python3.9/site-packages/gspread_pandas/spread.py", line 126, in init 09:38:21.960-!!|ASTROM| self.client = Client(user, config, scope, creds) 09:38:21.961-!!|ASTROM| File "/home/external/ealecian/.conda/envs/apero-env-v07288/lib/python3.9/site-packages/gspread_pandas/client.py", line 96, in init 09:38:21.961-!!|ASTROM| self._root = self._drive_request(file_id="root", params={"fields": "name,id"}) 09:38:21.962-!!|ASTROM| File "/home/external/ealecian/.conda/envs/apero-env-v07288/lib/python3.9/site-packages/gspread_pandas/client.py", line 197, in _drive_request 09:38:21.962-!!|ASTROM| res = self.request(method, url, params=params, json=data) 09:38:21.962-!!|ASTROM| File "/home/external/ealecian/.conda/envs/apero-env-v07288/lib/python3.9/site-packages/gspread_pandas/util.py", line 292, in request 09:38:21.963-!!|ASTROM| return ClientV4.request(client, *args, kwargs) 09:38:21.963-!!|ASTROM| AttributeError: type object 'Client' has no attribute 'request' 09:38:21.964-!!|ASTROM| 09:38:22.060-|ASTROM| * 09:38:22.074-@!|ASTROM| W[40-003-00005]: Recipe apero_astrometrics has NOT been successfully completed 09:38:22.087-*|ASTROM| full_v07288 Fri Jun 14 11:38:22 ealecian@node11: /net/GSP/nas12c/big_spirou/batch_jobs

[njcuk9999]

I've not seen that error before, were you definitely in the correct conda environment?

Can you check the version of modules compared to this list: https://github.com/njcuk9999/apero-drs/blob/v0.7.289-live/requirements_developer.txt

[aleciane]

Hi Neil

Le 18 juin 2024 à 21:19, Neil Cook @.***> a écrit :

I've not seen that error before, were you definitely in the correct conda environment?

Normaly yes Can you check the version of modules compared to this list: https://github.com/njcuk9999/apero-drs/blob/v0.7.289-live/requirements_developer.txt https://github.com/njcuk9999/apero-drs/blob/v0.7.289-live/requirements_developer.txt

Here is what I get :

pip show astropy | grep Version Version: 5.3 pip show astroquery | grep Version Version: 0.4.6 pip show barycorrpy | grep Version Version: 0.4.4 pip show Bottleneck | grep Version Version: 1.3.7 pip show duckdb | grep Version Version: 0.8.0 pip show matplotlib | grep Version Version: 3.7.1 pip show GitPython | grep Version Version: 3.1.31 pip show numba | grep Version Version: 0.57.0 pip show numexpr | grep Version Version: 2.8.4 pip show numpy | grep Version Version: 1.24.3 pip show pandas | grep Version Version: 2.0.2 pip show pandastable | grep Version Version: 0.13.1 pip show pandasql | grep Version Version: 0.7.3 pip show Pillow | grep Version Version: 9.5.0 pip show psutil | grep Version Version: 5.9.4 pip show PyYAML | grep Version Version: 6.0 pip show scikit-image | grep Version Version: 0.19.3 pip show scipy | grep Version Version: 1.10.0 pip show tqdm | grep Version Version: 4.65.0 pip show yagmail | grep Version Version: 0.15.293

Evelyne

Evelyne

— Reply to this email directly, view it on GitHub https://github.com/njcuk9999/apero-drs/issues/765#issuecomment-2176796484, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJQPZTU46JQZJN65RO54IBTZICB43AVCNFSM6AAAAABI22O42KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNZWG44TMNBYGQ. You are receiving this because you authored the thread.

[njcuk9999]

How about gspread-pandas, the one in the requirement file is:

gspread-pandas==3.2.2

I tried downgrading this module and I got a similar error as you!

Depending on which apero version you are running it might be worth trying a pip install -r requirements_developer.txt from inside the apero conda environment.

[aleciane]

Le 25 juin 2024 à 00:33, Neil Cook @.***> a écrit :

How about gspread-pandas, the one in the requirement file is:

gspread-pandas==3.2.2

I tried downgrading this module and I got a similar error as you!

I also have the 3.2.2 version

Depending on which apero version you are running it might be worth trying a pip install -r requirements_developer.txt from inside the apero conda environment.

I did that already.

Evelyne

— Reply to this email directly, view it on GitHub https://github.com/njcuk9999/apero-drs/issues/765#issuecomment-2187533947, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJQPZTSW45HKIU5MHZED4FDZJCNFFAVCNFSM6AAAAABI22O42KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOBXGUZTGOJUG4. You are receiving this because you authored the thread.

[njcuk9999]

Okay weird I'm not getting this error so either there is a problem with your server connecting to still a difference in versions

It is probably worth doing a pip freeze and then doing a diff against this list (the full list of files)

alabaster==0.7.16
astropy==5.3
astroquery==0.4.6
asttokens==2.4.1
Babel==2.14.0
backcall==0.2.0
backports.tarfile==1.1.1
barycorrpy==0.4.4
beautifulsoup4==4.12.3
blosc2==2.0.0
bokeh==3.1.1
Bottleneck==1.3.7
cachetools==5.3.3
certifi==2024.2.2
cffi==1.16.0
charset-normalizer==3.3.2
contourpy==1.2.1
cryptography==42.0.5
cssselect==1.2.0
cssutils==2.10.2
cycler==0.12.1
Cython==3.0.10
decorator==5.1.1
docutils==0.20.1
duckdb==0.8.0
executing==2.0.1
fonttools==4.51.0
future==1.0.0
gitchangelog==3.0.4
gitdb==4.0.11
GitPython==3.1.41
google-auth==2.29.0
google-auth-oauthlib==1.2.0
greenlet==3.0.3
gspread==5.12.4
gspread-pandas==3.3.0
html5lib==1.1
idna==3.7
imageio==2.34.1
imagesize==1.4.1
importlib_metadata==7.1.0
importlib_resources==6.4.0
ipdb==0.13.13
ipython==8.13.2
jaraco.classes==3.4.0
jaraco.context==5.3.0
jaraco.functools==4.0.1
jedi==0.19.1
jeepney==0.8.0
Jinja2==3.1.3
jplephem==2.22
keyring==25.2.0
kiwisolver==1.4.5
lbl @ git+https://github.com/njcuk9999/lbl.git@ea8e82b1836e24052a42fda4aeaa7b087b241644
llvmlite==0.40.1
lxml==5.2.1
MarkupSafe==2.1.5
matplotlib==3.7.1
matplotlib-inline==0.1.7
more-itertools==10.2.0
msgpack==1.0.8
mysql-connector-python==8.3.0
networkx==3.2.1
numba==0.57.0
numexpr==2.8.4
numpy==1.24.3
oauthlib==3.2.2
packaging==24.0
pandas==2.0.2
pandasql==0.7.3
pandastable==0.13.1
parso==0.8.4
pexpect==4.9.0
pickleshare==0.7.5
pillow==10.2.0
premailer==3.10.0
prompt-toolkit==3.0.43
psutil==5.9.4
ptyprocess==0.7.0
pure-eval==0.2.2
py-cpuinfo==9.0.0
pyasn1==0.6.0
pyasn1_modules==0.4.0
pycparser==2.22
pyerfa==2.0.1.4
Pygments==2.17.2
pyparsing==3.1.2
python-dateutil==2.9.0.post0
pytz==2024.1
pyvo==1.5.1
PyWavelets==1.6.0
PyYAML==6.0
requests==2.28.2
requests-oauthlib==2.0.0
rsa==4.9
scikit-image==0.19.3
scipy==1.10.0
SecretStorage==3.3.3
six==1.16.0
smmap==5.0.1
snowballstemmer==2.2.0
soupsieve==2.5
Sphinx==7.0.1
sphinxcontrib-applehelp==1.0.8
sphinxcontrib-devhelp==1.0.6
sphinxcontrib-htmlhelp==2.0.5
sphinxcontrib-jsmath==1.0.1
sphinxcontrib-qthelp==1.0.7
sphinxcontrib-serializinghtml==1.1.10
SQLAlchemy==2.0.15
stack-data==0.6.3
StrEnum==0.4.15
tables==3.8.0
tifffile==2024.4.24
tomli==2.0.1
tornado==6.4
tqdm==4.65.0
traitlets==5.14.3
ttkthemes==3.2.2
typing_extensions==4.11.0
tzdata==2024.1
urllib3==1.26.18
wcwidth==0.2.13
webencodings==0.5.1
wget==3.2
xlrd==2.0.1
xyzservices==2024.4.0
yagmail==0.15.293
zipp==3.18.1

[aleciane]

Hi Neil

I don’t see any difference in the requirements.

The problem appears when the code tries to write to the pending list (see below). I am using the same cluster as Andres at LAM. When Andres tries, it works. When I try I get the error. Could it be possible that I don’t have the rights to write to the database ?

Cheers Evelyne

The error occurs in the spread_pandas/util.py file line 292, i.e. in this routine :

def monkey_patch_request(client, retry_delay=10): """Monkey patch gspread's Client.request to auto-retry with a delay when you get a 100 seconds RESOURCE_EXCHAUSTED error."""

def request(*args, **kwargs):
    try:
        return ClientV4.request(client, *args, **kwargs)
    except APIError as e:
        error = str(e)
        # Only retry on 100 seconds quota breaches
        if "RESOURCE_EXHAUSTED" in error and (
            "100" in error or "Read requests" in error
        ):
            sleep(retry_delay)
            return request(*args, **kwargs)
        else:
            error = e

    if "error" in locals():
        raise error

client.request = request

... Add Teff for object [K]? Options are: [Y]es or [N]o

N 15:02:32.660-|ASTROM| * 15:02:32.668-|ASTROM| Updating pending-list online database 15:02:32.675-*|ASTROM| 15:02:32.835-!!|ASTROM| E[01-010-00001]: Unhandled error has occurred: Error <class 'AttributeError'> 15:02:32.835-!!|ASTROM| 15:02:32.835-!!|ASTROM| Traceback (most recent call last): 15:02:32.836-!!|ASTROM| File "/net/GSP/nas12c/big_spirou/APERO_v07288/apero-drs/apero/core/utils/drs_startup.py", line 433, in run 15:02:32.836-!!|ASTROM| llmain = func(recipe, params) 15:02:32.836-!!|ASTROM| File "/net/GSP/nas12c/big_spirou/APERO_v07288/apero-drs/tools/bin/apero_astrometrics.py", line 195, in main 15:02:32.836-!!|ASTROM| drs_astrometrics.add_obj_to_sheet(params, add_objs) 15:02:32.836-!!|ASTROM| File "/net/GSP/nas12c/big_spirou/APERO_v07288/apero-drs/apero/tools/module/database/drs_astrometrics.py", line 1145, in add_obj_to_sheet 15:02:32.836-!!|ASTROM| google_sheet = gspd.spread.Spread(sheet_id) 15:02:32.837-!!|ASTROM| File "/home/external/ealecian/.conda/envs/apero-env-v07288/lib/python3.9/site-packages/gspread_pandas/spread.py", line 126, in init 15:02:32.837-!!|ASTROM| self.client = Client(user, config, scope, creds) 15:02:32.837-!!|ASTROM| File "/home/external/ealecian/.conda/envs/apero-env-v07288/lib/python3.9/site-packages/gspread_pandas/client.py", line 96, in init 15:02:32.837-!!|ASTROM| self._root = self._drive_request(file_id="root", params={"fields": "name,id"}) 15:02:32.837-!!|ASTROM| File "/home/external/ealecian/.conda/envs/apero-env-v07288/lib/python3.9/site-packages/gspread_pandas/client.py", line 197, in _drive_request 15:02:32.838-!!|ASTROM| res = self.request(method, url, params=params, json=data) 15:02:32.838-!!|ASTROM| File "/home/external/ealecian/.conda/envs/apero-env-v07288/lib/python3.9/site-packages/gspread_pandas/util.py", line 292, in request 15:02:32.838-!!|ASTROM| return ClientV4.request(client, *args, kwargs) 15:02:32.838-!!|ASTROM| AttributeError: type object 'Client' has no attribute 'request' 15:02:32.838-!!|ASTROM| 15:02:32.900-|ASTROM| * 15:02:32.908-@!|ASTROM| W[40-003-00005]: Recipe apero_astrometrics has NOT been successfully completed 15:02:32.915-*|ASTROM|

Le 25 juin 2024 à 14:27, Neil Cook @.***> a écrit :

Okay weird I'm not getting this error so either there is a problem with your server connecting to still a difference in versions

It is probably worth doing a pip freeze and then doing a diff against this list (the full list of files)

— Reply to this email directly, view it on GitHub https://github.com/njcuk9999/apero-drs/issues/765#issuecomment-2188802065, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJQPZTWQVTR2UAYF6WD2ZZLZJFO3NAVCNFSM6AAAAABI22O42KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOBYHAYDEMBWGU. You are receiving this because you authored the thread.

[njcuk9999]

Maybe, the pending list is an online google sheet so maybe there is no internet connection? If it works for Andres and not for you then it probably isn't an apero problem.

The spread_pandas/util.py is not an apero function but a function in the gspread pandas module. Andres may have enabled something or got an admin at LAM to do something to allow this action. Maybe he can remember what was done originally when setting up apero?

I can't reproduce this and we have installed apero on multiple machines, multiple accounts.

One option to test the python/apero install would be for Andres to share his directory (make it read/writeable) and then try with that version of apero, it doesn't sound like this is the issue though and is more a problem for the cluster administrators.

[njcuk9999]

So I've been looking into this.... it may be somehow you have installed a module that has updated gspread (we require a module called gspread_pandas to be 3.2.2 but it seems most my systems are on gspread=5.X.X and gspread-pandas==3.2.2

It seems in the newest version of gspread this "Client" class has no method "request" - that was there in the older versions.

Then I found this: https://github.com/aiguofer/gspread-pandas/pull/101 which specifically states that gspread-pandas must be : | gspread>=5.0.0, <6

So I think the solution is to do:

pip install --force-reinstall "gspread==5.12.4"

and try again.

I'll add a line to the requirements to check gspread as well as gspread_pandas