Permission denied: '/home/.astropy' when importing astropy in AWS EMR

monocongo commented 2 years ago

Description

When I run a Jupyter notebook in AWS EMR using PySpark kernel I get a permission denied error when I run the first cell which contains an import of sbpy, it looks like astropy is trying to access '/home/.astropy', which probably just doesn't exist on the EC2 where the EMR notebook is running. How do we configure things so this won't happen and astropy can initialize without trying to access this location?

Expected behavior

I expected for sbpy to be imported without an error.

Actual behavior

An error was encountered:
[Errno 13] Permission denied: '/home/.astropy'
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/sbpy/data/__init__.py", line 792, in <module>
    from .ephem import Ephem  # noqa: E402
  File "/usr/local/lib/python3.7/site-packages/sbpy/data/ephem.py", line 20, in <module>
    from astroquery.jplhorizons import Horizons
  File "/usr/local/lib/python3.7/site-packages/astroquery/jplhorizons/__init__.py", line 218, in <module>
    from .core import Horizons, HorizonsClass
  File "/usr/local/lib/python3.7/site-packages/astroquery/jplhorizons/core.py", line 1308, in <module>
    Horizons = HorizonsClass()
  File "/usr/local/lib/python3.7/site-packages/astroquery/jplhorizons/core.py", line 92, in __init__
    super(HorizonsClass, self).__init__()
  File "/usr/local/lib/python3.7/site-packages/astroquery/query.py", line 179, in __init__
    paths.get_cache_dir(), 'astroquery',
  File "/usr/local/lib64/python3.7/site-packages/astropy/config/paths.py", line 170, in get_cache_dir
    return os.path.abspath(_find_or_create_root_dir('cache', linkto, rootname))
  File "/usr/local/lib64/python3.7/site-packages/astropy/config/paths.py", line 297, in _find_or_create_root_dir
    os.mkdir(innerdir)
PermissionError: [Errno 13] Permission denied: '/home/.astropy'

Steps to Reproduce

Open EMR notebook in AWS AMR
In the first cell run import sbpy.data.Names
```
from sbpy.data import Names
```

System Details

Results of the above from EMR notebook cell: Linux-4.14.281-212.502.amzn2.x86_64-x86_64-with-glibc2.2.5 Python 3.7.10 (default, Jun 3 2021, 00:02:01) [GCC 7.3.1 20180712 (Red Hat 7.3.1-13)] Numpy 1.20.0 pyerfa 2.0.0.1 astropy 4.3.1 Scipy 1.7.3

github-actions[bot] commented 2 years ago

Welcome to Astropy 👋 and thank you for your first issue!

A project member will respond to you as soon as possible; in the meantime, please double-check the guidelines for submitting issues and make sure you've provided the requested details.

GitHub issues in the Astropy repository are used to track bug reports and feature requests; If your issue poses a question about how to use Astropy, please instead raise your question in the Astropy Discourse user forum and close this issue.

If you feel that this issue has not been responded to in a timely manner, please leave a comment mentioning our software support engineer @embray, or send a message directly to the development mailing list. If the issue is urgent or sensitive in nature (e.g., a security vulnerability) please send an e-mail directly to the private e-mail feedback@astropy.org.

pllim commented 2 years ago

I don't see why you cannot change it in your instance to point to a place you can write to. Other HPC clusters have successfully done it and perhaps you could adopt their strategy. Have you read through https://docs.astropy.org/en/stable/config/index.html yet? Thanks.

If you have no control over this instance, please contact the service provider. I am not sure what we can do here.

github-actions[bot] commented 2 years ago

Hi humans :wave: - this issue was labeled as Close? approximately 12 hours ago. If you think this issue should not be closed, a maintainer should remove the Close? label - otherwise, I will close this issue in 7 days.

If you believe I commented on this issue incorrectly, please report this here

monocongo commented 2 years ago

Yes agreed, it looks like this can be taken care of with some configuration for astropy on the EMR cluster. I don't use astropy directly so this is my first exposure, I didn't know there was this local .astropy directory being created, etc., I'm just using sbpy for converting asteroid designations, and I can do that outside of an EMR job so I've just done the work needing sbpy locally and it works fine. In any event I really appreciate the fast attention to this issue and for the guidance to the configuration options, @pllim. Thank you!

pllim commented 2 years ago

Yes, indeed astropy creates a ~/.astropy for caching and config storing by default. I hope you could find a config that works for you. Good luck!

monocongo commented 1 year ago

Bmmk

On Sat, Jul 30, 2022, 3:42 PM P. L. Lim @.***> wrote:

Yes, indeed astropy creates a ~/.astropy for caching and config storing by default. I hope you could find a config that works for you. Good luck!

— Reply to this email directly, view it on GitHub https://github.com/astropy/astropy/issues/13512#issuecomment-1200298783, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAKEIHQN737TT2KLTDMROKDVWWOUNANCNFSM547JR6JQ . You are receiving this because you modified the open/close state.Message ID: @.***>

astropy / astropy