search

radio-astro-tools / spectral-cube

Library for reading and analyzing astrophysical spectral data cubes
http://spectral-cube.rtfd.org
BSD 3-Clause "New" or "Revised" License
98 stars 65 forks source link

trying to use yt from CASA 6 via spectral_cube #571

Open trhunter opened 5 years ago

trhunter commented 5 years ago

Hi, I think this is a question for John Zuhone. I have just started to try to use spectral_cube as an interface to yt. I have encountered the following issue when from trying to run spectral_cube in python3.5, and from an ipython/python3.6 conda environment with casatools imported. Is it a problem initializing the yt object? Thanks, -Todd

In [18]: import spectral_cube                                                                                                                                   

In [19]: cube = spectral_cube.SpectralCube.read('/lustre/naasc/users/cbrogan/2018.A.00031.T/FIGURES/Figure2/ALMA_199_nearest_highres/G358_Band5midres.contsub.pc
    ...: al_CH3OH_199574.851MHz_nearest_0.12kmsR0.0.image')                                                                                                     
WARNING: StokesWarning: Cube is a Stokes cube, returning spectral cube for I component [spectral_cube.spectral_cube]

In [20]: ytcube = cube.to_yt(spectral_factor=0.75)                                                                                                              
yt : [WARNING  ] 2019-08-21 17:43:45,798 Cannot find time
yt : [INFO     ] 2019-08-21 17:43:45,799 Detected these axes: RA---SIN DEC--SIN FREQ 
yt : [WARNING  ] 2019-08-21 17:43:45,804 No length conversion provided. Assuming 1 = 1 cm.
yt : [INFO     ] 2019-08-21 17:43:45,817 Parameters: current_time              = 0.0
yt : [INFO     ] 2019-08-21 17:43:45,817 Parameters: domain_dimensions         = [400 400  84]
yt : [INFO     ] 2019-08-21 17:43:45,817 Parameters: domain_left_edge          = [0.5 0.5 0.5]
yt : [INFO     ] 2019-08-21 17:43:45,817 Parameters: domain_right_edge         = [400.5 400.5  63.5]
yt : [INFO     ] 2019-08-21 17:43:45,817 Parameters: cosmological_simulation   = 0.0

In [23]: ytcube.quick_render_movie('movie')                                                                                                                     
WARNING: PossiblySlowWarning: This function (<function BaseSpectralCube.max at 0x7f5a28af8b70>) requires loading the entire cube into memory and may therefore be slow. [spectral_cube.utils]
WARNING: PossiblySlowWarning: This function (<function BaseSpectralCube.std at 0x7f5a28af8840>) requires loading the entire cube into memory and may therefore be slow. [spectral_cube.utils]
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-23-2b230f50e4fd> in <module>
----> 1 ytcube.quick_render_movie('movie')

~/.local/lib/python3.6/site-packages/spectral_cube-0.4.5.dev2227-py3.6.egg/spectral_cube/ytcube.py in quick_render_movie(self, outdir, size, nframes, camera_angle, north_vector, rot_vector, colormap, cmap_range, transfer_function, start_index, image_prefix, output_filename, log_scale, rescale)
    140 
    141         center = self.dataset.domain_center
--> 142         cam = self.dataset.h.camera(center, camera_angle, scale, size, tf,
    143                                     north_vector=north_vector, fields='flux')
    144 

~/.local/lib/python3.6/site-packages/yt-3.6.dev0-py3.6-linux-x86_64.egg/yt/data_objects/static_output.py in __getattr__(self, name)
    121         elif name in self.ds.index._index_properties:
    122             return getattr(self.ds.index, name)
--> 123         raise AttributeError
    124 
    125 class MutableAttribute(object):

AttributeError: 

In [24]:
keflavich commented 5 years ago

@trhunter use triple-backticks to quote code blocks

@jzuhone any ideas? This looks like we've initialized the yt cube incorrectly or incompletely within spectral-cube.

trhunter commented 5 years ago

FYI, this is on my NRAO workstation running RHEL 7.6, In [26]: yt.version
Out[26]: '3.6.dev0'

In [27]: spectral_cube.version
Out[27]: '0.4.5.dev2227'

jzuhone commented 5 years ago

@trhunter It looks like you’re using a dev version of yt. How did you get it?

trhunter commented 5 years ago

Initially, I tried to install with pip inside CASA 5.7 but that failed (see below). So I think I then did a git clone, which got the development version. However, my conda environment build for CASA 6 did build a release version (3.4.1). I have just put that at the top of my sys.path so that it gets loaded instead, but I still get the same error:

In [7]: yt.__version__                                                                                                                                          
Out[7]: '3.4.1'

In [8]: import spectral_cube                                                                                                                                    
WARNING: AstropyDeprecationWarning: astropy.extern.six will be removed in 4.0, use the six module directly if it is still needed [astropy.extern.six]

In [9]: cube = spectral_cube.SpectralCube.read('/lustre/naasc/users/cbrogan/2018.A.00031.T/FIGURES/Figure2/ALMA_199_nearest_highres/G358_Band5midres.contsub.pca
   ...: l_CH3OH_199574.851MHz_nearest_0.12kmsR0.0.image')                                                                                                       
registry available at 0.0.0.0:38436
WARNING: StokesWarning: Cube is a Stokes cube, returning spectral cube for I component [spectral_cube.spectral_cube]

In [10]: ytcube = cube.to_yt(spectral_factor=0.75)                                                                                                              
yt : [WARNING  ] 2019-08-21 18:53:19,155 Cannot find time
yt : [INFO     ] 2019-08-21 18:53:19,156 Detected these axes: RA---SIN DEC--SIN FREQ 
yt : [WARNING  ] 2019-08-21 18:53:19,164 No length conversion provided. Assuming 1 = 1 cm.
yt : [INFO     ] 2019-08-21 18:53:19,184 Parameters: current_time              = 0.0
yt : [INFO     ] 2019-08-21 18:53:19,184 Parameters: domain_dimensions         = [400 400  84]
yt : [INFO     ] 2019-08-21 18:53:19,184 Parameters: domain_left_edge          = [0.5 0.5 0.5]
yt : [INFO     ] 2019-08-21 18:53:19,185 Parameters: domain_right_edge         = [400.5 400.5  63.5]
yt : [INFO     ] 2019-08-21 18:53:19,186 Parameters: cosmological_simulation   = 0.0

In [11]: ytcube.quick_render_movie('movie')                                                                                                                     
WARNING: PossiblySlowWarning: This function (<function BaseSpectralCube.max at 0x7f7d482c8510>) requires loading the entire cube into memory and may therefore be slow. [spectral_cube.utils]
WARNING: PossiblySlowWarning: This function (<function BaseSpectralCube.std at 0x7f7d482c81e0>) requires loading the entire cube into memory and may therefore be slow. [spectral_cube.utils]
yt : [INFO     ] 2019-08-21 18:53:35,963 Adding field flux to the list of fields.
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-11-2b230f50e4fd> in <module>
----> 1 ytcube.quick_render_movie('movie')

~/.local/lib/python3.6/site-packages/spectral_cube-0.4.5.dev2227-py3.6.egg/spectral_cube/ytcube.py in quick_render_movie(self, outdir, size, nframes, camera_angle, north_vector, rot_vector, colormap, cmap_range, transfer_function, start_index, image_prefix, output_filename, log_scale, rescale)
    140 
    141         center = self.dataset.domain_center
--> 142         cam = self.dataset.h.camera(center, camera_angle, scale, size, tf,
    143                                     north_vector=north_vector, fields='flux')
    144 

/lustre/cv/users/thunter/miniconda3/envs/pycasa/lib/python3.6/site-packages/yt/data_objects/static_output.py in __getattr__(self, name)
    188         elif name in self.ds.index._index_properties:
    189             return getattr(self.ds.index, name)
--> 190         raise AttributeError
    191 
    192 class MutableAttribute(object):

AttributeError:

Here was the original install failure:

CASA <11>: pip.main(['install','yt','--user'])
Collecting yt                                 
  Downloading https://casa-pip.nrao.edu:443/repository/pypi-group/packages/18/25/a578748b2664b8fb52eec8f5d3d423c8a34fa7f19bfcda73486d11c593ee/yt-3.5.1-cp27-cp27
mu-manylinux1_x86_64.whl (11.7MB)                                                                                                         
    100% |\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588| 11.7MB 84kB/s                                                                                            
Collecting sympy>=1.0 (from yt)
  Downloading https://casa-pip.nrao.edu:443/repository/pypi-group/packages/21/21/f4105795ca7f35c541d82c5b06be684dd2f5cb4f508fb487cd7aea4de776/sympy-1.4-py2.py3-
none-any.whl (5.3MB)
    100% |\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588| 5.3MB 194kB/s
Requirement already satisfied: IPython>=1.0 in /home/casa/packages/RHEL6/prerelease/casa-prerelease-5.7.0-9/lib/python2.7/site-packages (from yt)
Requirement already satisfied: setuptools>=19.6 in /home/casa/packages/RHEL6/prerelease/casa-prerelease-5.7.0-9/lib/python2.7/site-packages (from yt)
Requirement already satisfied: numpy>=1.10.4 in /home/casa/packages/RHEL6/prerelease/casa-prerelease-5.7.0-9/lib/python2.7/site-packages (from yt)
Collecting matplotlib>=1.5.3 (from yt)
  Downloading https://casa-pip.nrao.edu:443/repository/pypi-group/packages/12/d1/7b12cd79c791348cb0c78ce6e7d16bd72992f13c9f1e8e43d2725a6d8adf/matplotlib-3.1.1.t
ar.gz (37.8MB)
    100% |\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588| 37.8MB 29kB/s
    Complete output from command python setup.py egg_info:

    Beginning with Matplotlib 3.1, Python 3.6 or above is required.

    This may be due to an out of date pip.

    Make sure you have pip >= 9.0.1.

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-HOslpw/matplotlib/
You are using pip version 9.0.0, however version 19.2.2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
trhunter commented 5 years ago

I had a fresh go at this on my home machine (Ubuntu 16.04.6 LTS). In CASA 5.5.0-149, I was able to:

from setuptools.command import easy_install
easy_install.main(['--user','pip'])
import subprocess,sys
subprocess.check_call([sys.executable,'-m','pip','install','--user','spectral_cube'])
subprocess.check_call([sys.executable,'-m','pip','install','--user','yt'])
import yt
import spectral_cube
yt.__version__
Out[27]: '3.5.1'
spectral_cube.__version__
Out[28]: u'0.4.4'

But I still get the Attribute error from quick_render_movie:

  24678     cube = spectral_cube.SpectralCube.read(img, format=myformat)
  24679     ytcube = cube.to_yt(spectral_factor=spectral_factor)
> 24680     ytcube.quick_render_movie(moviedir)

/home/thunter/.local/lib/python2.7/site-packages/spectral_cube/ytcube.pyc in quick_render_movie(self, outdir, size, nframes, camera_angle, north_vector, rot_vector, colormap, cmap_range, transfer_function, start_index, image_prefix, output_filename, log_scale, rescale)
    140 
    141         center = self.dataset.domain_center
--> 142         cam = self.dataset.h.camera(center, camera_angle, scale, size, tf,
    143                                     north_vector=north_vector, fields='flux')
    144 

/home/thunter/.local/lib/python2.7/site-packages/yt/data_objects/static_output.pyc in __getattr__(self, name)
    191         elif name in self.ds.index._index_properties:
    192             return getattr(self.ds.index, name)
--> 193         raise AttributeError
    194 
    195 class MutableAttribute(object):

AttributeError:
trhunter commented 5 years ago

I made another try today using a fresh anaconda with py36, and conda install yt, but I still get an attribute error when I try to visualize an ALMA cube using the spectral_cube tool. Is there any likelihood that this issue can be investigated? Thanks.

(py36) nepheline:ALMA2019 thunter$ python
Python 3.6.9 |Anaconda, Inc.| (default, Jul 30 2019, 13:42:17) 
[GCC 4.2.1 Compatible Clang 4.0.1 (tags/RELEASE_401/final)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import yt
>>> import spectral_cube
WARNING: AstropyDeprecationWarning: astropy.extern.six will be removed in 4.0, use the six module directly if it is still needed [astropy.extern.six]
>>> cube = spectral_cube.SpectralCube.read('G358_B6_altmaser.contsub_CH3OH_217299.205MHz_nearest_0.12kms.image.pbcor.fits',format='fits')
WARNING: FITSFixedWarning: 'obsfix' made the change 'Set OBSGEO-L to   -67.754929 from OBSGEO-[XYZ].
Set OBSGEO-B to   -23.022886 from OBSGEO-[XYZ].
Set OBSGEO-H to     5053.796 from OBSGEO-[XYZ]'. [astropy.wcs.wcs]
/Users/thunter/anaconda3/envs/py36/lib/python3.6/site-packages/spectral_cube/cube_utils.py:110: FutureWarning: Using a non-tuple sequence for multidimensional indexing is deprecated; use `arr[tuple(seq)]` instead of `arr[seq]`. In the future this will be interpreted as an array index, `arr[np.array(seq)]`, which will result either in an error or a different result.
  stokes_arrays[stokes_names[i_stokes]] = array[array_slice]
/Users/thunter/anaconda3/envs/py36/lib/python3.6/site-packages/spectral_cube/spectral_cube.py:2031: UserWarning: Cube is a Stokes cube, returning spectral cube for I component
  warnings.warn("Cube is a Stokes cube, returning spectral cube for I component")
>>> ytcube = cube.to_yt()
yt : [WARNING  ] 2019-10-14 10:11:19,669 Cannot find time
yt : [INFO     ] 2019-10-14 10:11:19,670 Detected these axes: RA---SIN DEC--SIN FREQ 
yt : [WARNING  ] 2019-10-14 10:11:19,675 No length conversion provided. Assuming 1 = 1 cm.
yt : [INFO     ] 2019-10-14 10:11:19,689 Parameters: current_time              = 0.0
yt : [INFO     ] 2019-10-14 10:11:19,689 Parameters: domain_dimensions         = [640 640  84]
yt : [INFO     ] 2019-10-14 10:11:19,690 Parameters: domain_left_edge          = [0.5 0.5 0.5]
yt : [INFO     ] 2019-10-14 10:11:19,690 Parameters: domain_right_edge         = [640.5 640.5  84.5]
yt : [INFO     ] 2019-10-14 10:11:19,690 Parameters: cosmological_simulation   = 0.0
>>> images = ytcube.quick_render_movie('217ghz_movie')
yt : [INFO     ] 2019-10-14 10:12:13,026 Adding field flux to the list of fields.
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/thunter/anaconda3/envs/py36/lib/python3.6/site-packages/spectral_cube/ytcube.py", line 147, in quick_render_movie
    cam = self.dataset.h.camera(center, camera_angle, scale, size, tf,
  File "/Users/thunter/anaconda3/envs/py36/lib/python3.6/site-packages/yt/data_objects/static_output.py", line 190, in __getattr__
    raise AttributeError
AttributeError
jzuhone commented 5 years ago

I'll try to get to this by tomorrow, sorry for the delay.

jzuhone commented 5 years ago

So this is more than a quick fix. I can address this during the week, but it may take a day or two.

trhunter commented 5 years ago

OK, thanks for the update.