fcoords and tcoords not the same shape for rays from particle datasets

yt-fido commented 8 years ago

Originally reported by: Britton Smith (Bitbucket: brittonsmith, GitHub: brittonsmith)

This script: http://paste.yt-project.org/show/6268/

gives the following error:

#!python

  File "test2.py", line 12, in <module>
    print ray["x"]
  File "/Users/britton/Documents/work/yt/yt-hg/yt/data_objects/data_containers.py", line 268, in __getitem__
    self.get_data(f)
  File "/Users/britton/Documents/work/yt/yt-hg/yt/data_objects/data_containers.py", line 1181, in get_data
    self._generate_fields(fields_to_generate)
  File "/Users/britton/Documents/work/yt/yt-hg/yt/data_objects/data_containers.py", line 1201, in _generate_fields
    fd = self._generate_field(field)
  File "/Users/britton/Documents/work/yt/yt-hg/yt/data_objects/data_containers.py", line 305, in _generate_field
    tr = self._generate_fluid_field(field)
  File "/Users/britton/Documents/work/yt/yt-hg/yt/data_objects/data_containers.py", line 325, in _generate_fluid_field
    rv = finfo(gen_obj)
  File "/Users/britton/Documents/work/yt/yt-hg/yt/fields/derived_field.py", line 184, in __call__
    dd = self._function(self, data)
  File "/Users/britton/Documents/work/yt/yt-hg/yt/geometry/coordinates/coordinate_handler.py", line 40, in _coords
    rv = data.ds.arr(data.fcoords[...,axi].copy(), units)
  File "/Users/britton/Documents/work/yt/yt-hg/yt/data_objects/data_containers.py", line 1282, in fcoords
    return self._current_chunk.fcoords
  File "/Users/britton/Documents/work/yt/yt-hg/yt/geometry/geometry_handler.py", line 271, in cached_func
    tr = func(self)
  File "/Users/britton/Documents/work/yt/yt-hg/yt/geometry/geometry_handler.py", line 322, in fcoords
    ci[ind:ind+c.shape[0], :] = c
ValueError: could not broadcast input array from shape (81,3) into shape (40,3)

This seems to be because the fcoords and tcoords for the ray are coming back with different shapes.

Bitbucket: https://bitbucket.org/yt_analysis/yt/issue/1175

yt-fido commented 8 years ago

Original comment by Erwin Tin-Hay Lau (Bitbucket: ethlau, GitHub: ethlau):

Problem solved. Thanks!

yt-fido commented 8 years ago

Original comment by Nathan Goldbaum (Bitbucket: ngoldbaum, GitHub: ngoldbaum):

@ethlau can you try again using the latest development version of yt?

yt-fido commented 8 years ago

Original comment by Nathan Goldbaum (Bitbucket: ngoldbaum, GitHub: ngoldbaum):

Merged in MatthewTurk/yt (pull request #2116)

Ensuring mask for dt in octree is in correct order. Closes #1175

yt-fido commented 8 years ago

Original comment by Nathan Goldbaum (Bitbucket: ngoldbaum, GitHub: ngoldbaum):

Merged in MatthewTurk/yt (pull request #2116)

Ensuring mask for dt in octree is in correct order. Closes #1175

yt-fido commented 8 years ago

Original comment by Nathan Goldbaum (Bitbucket: ngoldbaum, GitHub: ngoldbaum):

Merged in MatthewTurk/yt (pull request #2116)

Ensuring mask for dt in octree is in correct order. Closes #1175

yt-fido commented 8 years ago

Original comment by Nathan Goldbaum (Bitbucket: ngoldbaum, GitHub: ngoldbaum):

Merged in MatthewTurk/yt (pull request #2116)

Ensuring mask for dt in octree is in correct order. Closes #1175

yt-fido commented 8 years ago

Original comment by Erwin Tin-Hay Lau (Bitbucket: ethlau, GitHub: ethlau):

I am having the same issue when generating a ray object passing through non-root-grid cells (in ARTIO output).

#!python
Traceback (most recent call last):
  File "test.py", line 57, in <module>
    fields=['density', 'temperature','metallicity'])
  File "/lustre/home/client/fas/nagai/etl28/program/trident/trident/ray_generator.py", line 96, in make_simple_ray
    redshift=redshift)
  File "/lustre/home/client/fas/nagai/etl28/program/yt/yt-x86_64/src/yt-hg/yt/analysis_modules/cosmological_observation/light_ray/light_ray.py", line 451, in make_light_ray
    sub_data[field].extend(sub_ray[field][asort])
  File "/lustre/home/client/fas/nagai/etl28/program/yt/yt-x86_64/src/yt-hg/yt/data_objects/data_containers.py", line 270, in __getitem__
    self.get_data(f)
  File "/lustre/home/client/fas/nagai/etl28/program/yt/yt-x86_64/src/yt-hg/yt/data_objects/data_containers.py", line 1184, in get_data
    self._generate_fields(fields_to_generate)
  File "/lustre/home/client/fas/nagai/etl28/program/yt/yt-x86_64/src/yt-hg/yt/data_objects/data_containers.py", line 1204, in _generate_fields
    fd = self._generate_field(field)
  File "/lustre/home/client/fas/nagai/etl28/program/yt/yt-x86_64/src/yt-hg/yt/data_objects/data_containers.py", line 307, in _generate_field
    tr = self._generate_fluid_field(field)
  File "/lustre/home/client/fas/nagai/etl28/program/yt/yt-x86_64/src/yt-hg/yt/data_objects/data_containers.py", line 327, in _generate_fluid_field
    rv = finfo(gen_obj)
  File "/lustre/home/client/fas/nagai/etl28/program/yt/yt-x86_64/src/yt-hg/yt/fields/derived_field.py", line 197, in __call__
    dd = self._function(self, data)
  File "/lustre/home/client/fas/nagai/etl28/program/yt/yt-x86_64/src/yt-hg/yt/geometry/coordinates/coordinate_handler.py", line 40, in _coords
    rv = data.ds.arr(data.fcoords[...,axi].copy(), units)
  File "/lustre/home/client/fas/nagai/etl28/program/yt/yt-x86_64/src/yt-hg/yt/data_objects/data_containers.py", line 1291, in fcoords
    return self._current_chunk.fcoords
  File "/lustre/home/client/fas/nagai/etl28/program/yt/yt-x86_64/src/yt-hg/yt/geometry/geometry_handler.py", line 271, in cached_func
    tr = func(self)
  File "/lustre/home/client/fas/nagai/etl28/program/yt/yt-x86_64/src/yt-hg/yt/geometry/geometry_handler.py", line 322, in fcoords
    ci[ind:ind+c.shape[0], :] = c
ValueError: could not broadcast input array from shape (140,3) into shape (24,3)

yt-fido commented 8 years ago

Original comment by Matt Turk (Bitbucket: MatthewTurk, GitHub: MatthewTurk):

I have spent a fair amount of time digging in to this today, and what I have determined is that these are two possibilities for the error:

Something amiss with F/C ordering of the zones in the octree. I do not believe this extends to a global F/C ordering of the octree (Noct, 2,2,2) itself
Something amiss with "partial" selection of zones in the octree.

I'm leaning toward the second, but @xarthisius's bisection isn't necessarily supportive of that conclusion. This possibility arose when I was looking at what happened if I set n_ref=8 and over_refine_factor=0 in the load command. Doing that resulted in correct results.

Another item to note: the sum of dt should be 1.0 precisely. It's nearly exactly 0.5 for the standard case and it's precisely 1.0 when I set those two options.

yt-fido commented 8 years ago

Original comment by Kacper Kowalik (Bitbucket: xarthisius, GitHub: xarthisius):

The first bad revision is:
changeset:   19784:b9768c9cce41
branch:      yt
user:        Matthew Turk <matthewturk@gmail.com>
date:        Mon Jan 25 10:04:02 2016 -0600
summary:     Switching to F order for the octs

yt-fido commented 8 years ago

Original comment by Matt Turk (Bitbucket: MatthewTurk, GitHub: MatthewTurk):

In fact, my guess is that it's related to some memoryview work that I did.

yt-fido commented 8 years ago

Original comment by Matt Turk (Bitbucket: MatthewTurk, GitHub: MatthewTurk):

Hi Bili, this is fascinating, and I can certainly debug this. Thank you!

yt-fido commented 8 years ago

Original comment by Bili Dong (Bitbucket: qobilidop, GitHub: qobilidop):

I have an interesting finding that if I use yt (3.2.3) installed through conda, there will be no error. So maybe something has changed between that version and the development tip, I guess.

yt-project / yt

fcoords and tcoords not the same shape for rays from particle datasets #1175