search

pydata / xarray

N-D labeled arrays and datasets in Python
https://xarray.dev
Apache License 2.0
3.64k stars 1.09k forks source link

Confusing error message when attribute not equal during concat #2060

Closed djhoese closed 4 years ago

djhoese commented 6 years ago

Code Sample, a copy-pastable example if possible

In [1]: import dask.array as da; import xarray as xr; import numpy as np

In [2]: a = xr.DataArray(da.random.random((4, 6), chunks=2), attrs={'test': ['x1', 'y1']}, dims=('y', 'x'))

In [3]: b = xr.DataArray(da.random.random((4, 6), chunks=2), attrs={'test': ['x2', 'y2']}, dims=('y', 'x'))

In [4]: xr.concat([a, b], 'y')
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-4-c8b32db4cfb7> in <module>()
----> 1 xr.concat([a, b], 'y')

~/anaconda/envs/polar2grid_py36/lib/python3.6/site-packages/xarray/core/combine.py in concat(objs, dim, data_vars, coords, compat, positions, indexers, mode, concat_over)
    119         raise TypeError('can only concatenate xarray Dataset and DataArray '
    120                         'objects, got %s' % type(first_obj))
--> 121     return f(objs, dim, data_vars, coords, compat, positions)
    122 
    123 

~/anaconda/envs/polar2grid_py36/lib/python3.6/site-packages/xarray/core/combine.py in _dataarray_concat(arrays, dim, data_vars, coords, compat, positions)
    337 
    338     ds = _dataset_concat(datasets, dim, data_vars, coords, compat,
--> 339                          positions)
    340     return arrays[0]._from_temp_dataset(ds, name)
    341 

~/anaconda/envs/polar2grid_py36/lib/python3.6/site-packages/xarray/core/combine.py in _dataset_concat(datasets, dim, data_vars, coords, compat, positions)
    303         if k in concat_over:
    304             vars = ensure_common_dims([ds.variables[k] for ds in datasets])
--> 305             combined = concat_vars(vars, dim, positions)
    306             insert_result_variable(k, combined)
    307 

~/anaconda/envs/polar2grid_py36/lib/python3.6/site-packages/xarray/core/variable.py in concat(variables, dim, positions, shortcut)
   1772         return IndexVariable.concat(variables, dim, positions, shortcut)
   1773     else:
-> 1774         return Variable.concat(variables, dim, positions, shortcut)
   1775 
   1776 

~/anaconda/envs/polar2grid_py36/lib/python3.6/site-packages/xarray/core/variable.py in concat(cls, variables, dim, positions, shortcut)
   1299                 if var.dims != first_var.dims:
   1300                     raise ValueError('inconsistent dimensions')
-> 1301                 utils.remove_incompatible_items(attrs, var.attrs)
   1302 
   1303         return cls(dims, data, attrs, encoding)

~/anaconda/envs/polar2grid_py36/lib/python3.6/site-packages/xarray/core/utils.py in remove_incompatible_items(first_dict, second_dict, compat)
    157         if (k not in second_dict or
    158             (k in second_dict and
--> 159                 not compat(first_dict[k], second_dict[k]))):
    160             del first_dict[k]
    161 

~/anaconda/envs/polar2grid_py36/lib/python3.6/site-packages/xarray/core/utils.py in equivalent(first, second)
    106         return ((first is second) or
    107                 (first == second) or
--> 108                 (pd.isnull(first) and pd.isnull(second)))
    109 
    110 

ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

Problem description

If two or more DataArrays are concatentated and they have list attributes that are not equal an exception is raised about arrays not being truth values.

Expected Output

I guess the expected result would be that the list attribute is not included in the resulting DataArray's attributes.

Output of xr.show_versions()

``` DEBUG:matplotlib:$HOME=/Users/davidh DEBUG:matplotlib:matplotlib data path /Users/davidh/anaconda/envs/polar2grid_py36/lib/python3.6/site-packages/matplotlib/mpl-data DEBUG:matplotlib:loaded rc file /Users/davidh/anaconda/envs/polar2grid_py36/lib/python3.6/site-packages/matplotlib/mpl-data/matplotlibrc DEBUG:matplotlib:matplotlib version 2.2.0 DEBUG:matplotlib:interactive is False DEBUG:matplotlib:platform is darwin DEBUG:matplotlib:loaded modules: ['builtins', 'sys', '_frozen_importlib', '_imp', '_warnings', '_thread', '_weakref', '_frozen_importlib_external', '_io', 'marshal', 'posix', 'zipimport', 'encodings', 'codecs', '_codecs', 'encodings.aliases', 'encodings.utf_8', '_signal', '__main__', 'encodings.latin_1', 'io', 'abc', '_weakrefset', 'site', 'os', 'errno', 'stat', '_stat', 'posixpath', 'genericpath', 'os.path', '_collections_abc', '_sitebuiltins', 'sysconfig', '_sysconfigdata_m_darwin_darwin', '_osx_support', 're', 'enum', 'types', 'functools', '_functools', 'collections', 'operator', '_operator', 'keyword', 'heapq', '_heapq', 'itertools', 'reprlib', '_collections', 'weakref', 'collections.abc', 'sre_compile', '_sre', 'sre_parse', 'sre_constants', '_locale', 'copyreg', '_bootlocale', 'importlib', 'importlib._bootstrap', 'importlib._bootstrap_external', 'warnings', 'importlib.util', 'importlib.abc', 'importlib.machinery', 'contextlib', 'mpl_toolkits', 'sphinxcontrib', 'encodings.cp437', 'IPython', 'IPython.core', 'IPython.core.getipython', 'IPython.core.release', 'IPython.core.application', 'atexit', 'copy', 'glob', 'fnmatch', 'logging', 'time', 'traceback', 'linecache', 'tokenize', 'token', 'string', '_string', 'threading', 'shutil', 'zlib', 'bz2', '_compression', '_bz2', 'lzma', '_lzma', 'pwd', 'grp', 'traitlets', 'traitlets.traitlets', 'inspect', 'ast', '_ast', 'dis', 'opcode', '_opcode', 'six', '__future__', 'struct', '_struct', 'traitlets.utils', 'traitlets.utils.getargspec', 'traitlets.utils.importstring', 'ipython_genutils', 'ipython_genutils._version', 'ipython_genutils.py3compat', 'ipython_genutils.encoding', 'locale', 'platform', 'subprocess', 'signal', '_posixsubprocess', 'select', 'selectors', 'math', 'traitlets.utils.sentinel', 'traitlets.utils.bunch', 'traitlets._version', 'traitlets.config', 'traitlets.config.application', 'json', 'json.decoder', 'json.scanner', '_json', 'json.encoder', 'decorator', 'traitlets.config.configurable', 'traitlets.config.loader', 'argparse', 'textwrap', 'gettext', 'ipython_genutils.path', 'random', 'hashlib', '_hashlib', '_blake2', '_sha3', 'bisect', '_bisect', '_random', 'ipython_genutils.text', 'ipython_genutils.importstring', 'IPython.core.crashhandler', 'pprint', 'IPython.core.ultratb', 'pydoc', 'pkgutil', 'urllib', 'urllib.parse', 'IPython.core.debugger', 'bdb', 'IPython.utils', 'IPython.utils.PyColorize', 'IPython.utils.coloransi', 'IPython.utils.ipstruct', 'IPython.utils.colorable', 'pygments', 'pygments.util', 'IPython.utils.py3compat', 'IPython.utils.encoding', 'IPython.core.excolors', 'IPython.testing', 'IPython.testing.skipdoctest', 'pdb', 'cmd', 'code', 'codeop', 'IPython.core.display_trap', 'IPython.utils.openpy', 'IPython.utils.path', 'IPython.utils.process', 'IPython.utils._process_posix', 'pexpect', 'pexpect.exceptions', 'pexpect.utils', 'pexpect.expect', 'pexpect.pty_spawn', 'pty', 'tty', 'termios', 'ptyprocess', 'ptyprocess.ptyprocess', 'fcntl', 'resource', 'ptyprocess.util', 'pexpect.spawnbase', 'pexpect.run', 'IPython.utils._process_common', 'shlex', 'IPython.utils.decorators', 'IPython.utils.data', 'IPython.utils.terminal', 'IPython.utils.sysinfo', 'IPython.utils._sysinfo', 'IPython.core.profiledir', 'IPython.paths', 'tempfile', 'IPython.utils.importstring', 'IPython.terminal', 'IPython.terminal.embed', 'IPython.core.compilerop', 'IPython.core.magic_arguments', 'IPython.core.error', 'IPython.utils.text', 'pathlib', 'ntpath', 'IPython.core.magic', 'getopt', 'IPython.core.oinspect', 'IPython.core.page', 'IPython.core.display', 'base64', 'binascii', 'mimetypes', 'IPython.lib', 'IPython.lib.security', 'getpass', 'IPython.lib.pretty', 'datetime', '_datetime', 'IPython.utils.dir2', 'IPython.utils.wildcard', 'pygments.lexers', 'pygments.lexers._mapping', 'pygments.modeline', 'pygments.plugin', 'pygments.lexers.python', 'pygments.lexer', 'pygments.filter', 'pygments.filters', 'pygments.token', 'pygments.regexopt', 'pygments.unistring', 'pygments.formatters', 'pygments.formatters._mapping', 'pygments.formatters.html', 'pygments.formatter', 'pygments.styles', 'IPython.core.inputsplitter', 'IPython.core.inputtransformer', 'IPython.core.splitinput', 'IPython.utils.tokenize2', 'IPython.core.interactiveshell', 'runpy', 'pickleshare', 'pickle', '_compat_pickle', '_pickle', 'IPython.core.prefilter', 'IPython.core.autocall', 'IPython.core.macro', 'IPython.core.alias', 'IPython.core.builtin_trap', 'IPython.core.events', 'IPython.core.displayhook', 'IPython.core.displaypub', 'IPython.core.extensions', 'IPython.core.formatters', 'IPython.utils.sentinel', 'IPython.core.history', 'sqlite3', 'sqlite3.dbapi2', '_sqlite3', 'IPython.core.logger', 'IPython.core.payload', 'IPython.core.usage', 'IPython.display', 'IPython.lib.display', 'IPython.utils.io', 'IPython.utils.capture', 'IPython.utils.strdispatch', 'IPython.core.hooks', 'IPython.utils.syspathcontext', 'IPython.utils.tempdir', 'IPython.utils.contexts', 'IPython.terminal.interactiveshell', 'prompt_toolkit', 'prompt_toolkit.interface', 'prompt_toolkit.application', 'prompt_toolkit.buffer', 'prompt_toolkit.auto_suggest', 'prompt_toolkit.filters', 'prompt_toolkit.filters.base', 'prompt_toolkit.utils', 'wcwidth', 'wcwidth.wcwidth', 'wcwidth.table_wide', 'wcwidth.table_zero', 'six.moves', 'prompt_toolkit.filters.cli', 'prompt_toolkit.enums', 'prompt_toolkit.key_binding', 'prompt_toolkit.key_binding.vi_state', 'prompt_toolkit.cache', 'prompt_toolkit.filters.types', 'prompt_toolkit.filters.utils', 'prompt_toolkit.clipboard', 'prompt_toolkit.clipboard.base', 'prompt_toolkit.selection', 'prompt_toolkit.clipboard.in_memory', 'prompt_toolkit.completion', 'prompt_toolkit.document', 'prompt_toolkit.history', 'prompt_toolkit.search_state', 'prompt_toolkit.validation', 'prompt_toolkit.buffer_mapping', 'prompt_toolkit.key_binding.bindings', 'prompt_toolkit.key_binding.bindings.basic', 'prompt_toolkit.keys', 'prompt_toolkit.layout', 'prompt_toolkit.layout.containers', 'prompt_toolkit.layout.controls', 'prompt_toolkit.mouse_events', 'prompt_toolkit.token', 'prompt_toolkit.layout.lexers', 'prompt_toolkit.layout.utils', 'prompt_toolkit.layout.processors', 'prompt_toolkit.reactive', 'prompt_toolkit.layout.screen', 'prompt_toolkit.layout.dimension', 'prompt_toolkit.layout.margins', 'prompt_toolkit.renderer', 'prompt_toolkit.layout.mouse_handlers', 'prompt_toolkit.output', 'prompt_toolkit.styles', 'prompt_toolkit.styles.base', 'prompt_toolkit.styles.defaults', 'prompt_toolkit.styles.from_dict', 'prompt_toolkit.styles.utils', 'prompt_toolkit.styles.from_pygments', 'pygments.style', 'pygments.styles.default', 'prompt_toolkit.key_binding.bindings.named_commands', 'prompt_toolkit.key_binding.bindings.completion', 'prompt_toolkit.key_binding.registry', 'prompt_toolkit.key_binding.input_processor', 'prompt_toolkit.key_binding.bindings.emacs', 'prompt_toolkit.key_binding.bindings.scroll', 'prompt_toolkit.key_binding.bindings.vi', 'prompt_toolkit.key_binding.digraphs', 'prompt_toolkit.key_binding.defaults', 'prompt_toolkit.eventloop', 'prompt_toolkit.eventloop.base', 'prompt_toolkit.eventloop.callbacks', 'prompt_toolkit.input', 'prompt_toolkit.terminal', 'prompt_toolkit.terminal.vt100_input', 'prompt_toolkit.shortcuts', 'prompt_toolkit.layout.menus', 'prompt_toolkit.layout.prompt', 'prompt_toolkit.layout.toolbars', 'prompt_toolkit.terminal.vt100_output', 'array', 'prompt_toolkit.key_binding.manager', 'IPython.terminal.debugger', 'IPython.core.completer', 'unicodedata', 'typing', 'typing.io', 'typing.re', 'IPython.core.latex_symbols', 'IPython.utils.generics', 'simplegeneric', 'jedi', 'jedi.api', 'jedi.parser', 'jedi.parser.parser', 'jedi.parser.tree', 'jedi._compatibility', 'imp', 'jedi.parser.pgen2', 'jedi.parser.pgen2.parse', 'jedi.parser.tokenize', 'jedi.parser.token', 'jedi.common', 'jedi.settings', 'jedi.parser.pgen2.pgen', 'jedi.parser.pgen2.grammar', 'jedi.parser.python', 'jedi.parser.python.parser', 'jedi.parser.python.tree', 'jedi.parser.python.diff', 'difflib', 'jedi.debug', 'jedi.parser.cache', 'gc', 'jedi.cache', 'jedi.api.classes', 'jedi.evaluate', 'jedi.evaluate.representation', 'jedi.evaluate.cache', 'jedi.evaluate.compiled', 'jedi.evaluate.helpers', 'jedi.evaluate.filters', 'jedi.evaluate.flow_analysis', 'jedi.evaluate.context', 'jedi.evaluate.compiled.fake', 'jedi.evaluate.recursion', 'jedi.evaluate.iterable', 'jedi.evaluate.analysis', 'jedi.evaluate.pep0484', 'jedi.evaluate.precedence', 'jedi.evaluate.docstrings', 'jedi.evaluate.param', 'jedi.evaluate.imports', 'jedi.evaluate.sys_path', 'jedi.evaluate.site', 'jedi.evaluate.dynamic', 'jedi.evaluate.stdlib', 'jedi.evaluate.instance', 'jedi.evaluate.finder', 'jedi.api.keywords', 'pydoc_data', 'pydoc_data.topics', 'jedi.api.interpreter', 'jedi.evaluate.compiled.mixed', 'jedi.api.usages', 'jedi.api.helpers', 'jedi.api.completion', 'IPython.terminal.ptutils', 'IPython.terminal.shortcuts', 'IPython.terminal.magics', 'IPython.lib.clipboard', 'IPython.terminal.pt_inputhooks', 'IPython.terminal.prompts', 'pkg_resources', 'zipfile', 'plistlib', 'xml', 'xml.parsers', 'xml.parsers.expat', 'pyexpat.errors', 'pyexpat.model', 'pyexpat', 'xml.parsers.expat.model', 'xml.parsers.expat.errors', 'email', 'email.parser', 'email.feedparser', 'email.errors', 'email._policybase', 'email.header', 'email.quoprimime', 'email.base64mime', 'email.charset', 'email.encoders', 'quopri', 'email.utils', 'socket', '_socket', 'email._parseaddr', 'calendar', 'pkg_resources.extern', 'pkg_resources._vendor', 'pkg_resources.extern.six', 'pkg_resources._vendor.six', 'pkg_resources.extern.six.moves', 'pkg_resources._vendor.six.moves', 'pkg_resources.py31compat', 'pkg_resources.extern.appdirs', 'pkg_resources._vendor.packaging.__about__', 'pkg_resources.extern.packaging', 'pkg_resources.extern.packaging.version', 'pkg_resources.extern.packaging._structures', 'pkg_resources.extern.packaging.specifiers', 'pkg_resources.extern.packaging._compat', 'pkg_resources.extern.packaging.requirements', 'pkg_resources.extern.pyparsing', 'pkg_resources.extern.six.moves.urllib', 'pkg_resources.extern.packaging.markers', 'IPython.terminal.ipapp', 'IPython.core.magics', 'IPython.core.magics.auto', 'IPython.core.magics.basic', 'IPython.core.magics.code', 'IPython.core.magics.config', 'IPython.core.magics.display', 'IPython.core.magics.execution', 'timeit', 'cProfile', '_lsprof', 'profile', 'optparse', 'pstats', 'IPython.utils.module_paths', 'IPython.utils.timing', 'IPython.core.magics.extension', 'IPython.core.magics.history', 'IPython.core.magics.logging', 'IPython.core.magics.namespace', 'IPython.core.magics.osm', 'IPython.core.magics.pylab', 'IPython.core.pylabtools', 'IPython.core.magics.script', 'IPython.lib.backgroundjobs', 'IPython.core.shellapp', 'IPython.extensions', 'IPython.extensions.storemagic', 'IPython.utils.frame', 'IPython.core.completerlib', 'pygments.lexers.shell', 'pygments.lexers.html', 'pygments.lexers.javascript', 'pygments.lexers.jvm', 'pygments.lexers.css', 'pygments.lexers.ruby', 'pygments.lexers.perl', 'pygments.lexers.markup', 'prompt_toolkit.eventloop.posix', 'prompt_toolkit.eventloop.inputhook', 'prompt_toolkit.eventloop.select', 'prompt_toolkit.eventloop.posix_utils', 'prompt_toolkit.eventloop.utils', 'storemagic', 'dask', 'dask.core', 'dask.utils_test', 'dask.context', 'dask.local', 'dask.compatibility', 'queue', 'gzip', 'urllib.request', 'http', 'http.client', 'email.message', 'uu', 'email._encoded_words', 'email.iterators', 'ssl', 'ipaddress', '_ssl', 'urllib.error', 'urllib.response', '_scproxy', 'dask.order', 'dask.callbacks', 'dask.optimization', 'dask.delayed', 'uuid', 'ctypes', '_ctypes', 'ctypes._endian', 'ctypes.util', 'ctypes.macholib', 'ctypes.macholib.dyld', 'ctypes.macholib.framework', 'ctypes.macholib.dylib', 'toolz', 'toolz.itertoolz', 'toolz.compatibility', 'toolz.utils', 'toolz.functoolz', 'toolz._signatures', 'toolz.dicttoolz', 'toolz.recipes', 'toolz.sandbox', 'toolz.sandbox.core', 'toolz.sandbox.parallel', 'dask.threaded', 'multiprocessing', 'multiprocessing.context', 'multiprocessing.process', 'multiprocessing.reduction', '__mp_main__', 'multiprocessing.pool', 'multiprocessing.util', 'dask.base', 'dask.hashing', 'dask.utils', 'numbers', 'dask.optimize', 'dask.sharedict', 'cloudpickle', 'cloudpickle.cloudpickle', 'encodings.raw_unicode_escape', 'dask._version', 'dask.array', 'dask.array.core', 'toolz.curried', 'toolz.curried.operator', 'toolz.curried.exceptions', 'numpy', 'numpy._globals', 'numpy.__config__', 'numpy.version', 'numpy._import_tools', 'numpy.add_newdocs', 'numpy.lib', 'numpy.lib.info', 'numpy.lib.type_check', 'numpy.core', 'numpy.core.info', 'numpy.core.multiarray', 'numpy.core.umath', 'numpy.core._internal', 'numpy.compat', 'numpy.compat._inspect', 'numpy.compat.py3k', 'numpy.core.numerictypes', 'numpy.core.numeric', 'numpy.core.arrayprint', 'numpy.core.fromnumeric', 'numpy.core._methods', 'numpy.core.defchararray', 'numpy.core.records', 'numpy.core.memmap', 'numpy.core.function_base', 'numpy.core.machar', 'numpy.core.getlimits', 'numpy.core.shape_base', 'numpy.core.einsumfunc', 'numpy.testing', 'unittest', 'unittest.result', 'unittest.util', 'unittest.case', 'unittest.suite', 'unittest.loader', 'unittest.main', 'unittest.runner', 'unittest.signals', 'numpy.testing.decorators', 'numpy.testing.utils', 'numpy.lib.utils', 'numpy.testing.nosetester', 'numpy.lib.ufunclike', 'numpy.lib.index_tricks', 'numpy.lib.function_base', 'numpy.lib.twodim_base', 'numpy.matrixlib', 'numpy.matrixlib.defmatrix', 'numpy.lib.stride_tricks', 'numpy.lib.mixins', 'numpy.lib.nanfunctions', 'numpy.lib.shape_base', 'numpy.lib.scimath', 'numpy.lib.polynomial', 'numpy.linalg', 'numpy.linalg.info', 'numpy.linalg.linalg', 'numpy.linalg.lapack_lite', 'numpy.linalg._umath_linalg', 'numpy.lib.arraysetops', 'numpy.lib.npyio', 'numpy.lib.format', 'numpy.lib._datasource', 'numpy.lib._iotools', 'numpy.lib.financial', 'numpy.lib.arrayterator', 'numpy.lib.arraypad', 'numpy.lib._version', 'numpy._distributor_init', 'numpy.fft', 'numpy.fft.info', 'numpy.fft.fftpack', 'numpy.fft.fftpack_lite', 'numpy.fft.helper', 'numpy.polynomial', 'numpy.polynomial.polynomial', 'numpy.polynomial.polyutils', 'numpy.polynomial._polybase', 'numpy.polynomial.chebyshev', 'numpy.polynomial.legendre', 'numpy.polynomial.hermite', 'numpy.polynomial.hermite_e', 'numpy.polynomial.laguerre', 'numpy.random', 'numpy.random.info', 'cython_runtime', 'mtrand', 'numpy.random.mtrand', 'numpy.ctypeslib', 'numpy.ma', 'numpy.ma.core', 'numpy.ma.extras', 'dask.array.chunk', 'dask.array.numpy_compat', 'distutils', 'distutils.version', 'dask.array.slicing', 'dask.array.optimization', 'dask.array.routines', 'dask.array.creation', 'dask.array.wrap', 'dask.array.reshape', 'dask.array.ufunc', 'dask.array.reductions', 'dask.array.percentile', 'dask.array.ma', 'dask.array.random', 'dask.array.linalg', 'dask.array.ghost', 'dask.array.learn', 'dask.array.fft', 'scipy', 'scipy._distributor_init', 'scipy.__config__', 'scipy.version', 'scipy._lib', 'scipy._lib._testutils', 'scipy._lib._version', 'scipy._lib.six', 'scipy._lib._ccallback', 'scipy._lib._ccallback_c', 'scipy.fftpack', 'scipy.fftpack.basic', 'scipy.fftpack._fftpack', 'scipy.fftpack.pseudo_diffs', 'scipy.fftpack.convolve', 'scipy.fftpack.helper', 'numpy.dual', 'scipy.fftpack.realtransforms', 'dask.array.rechunk', 'xarray', 'xarray.core', 'xarray.core.alignment', 'xarray.core.utils', 'pandas', 'pytz', 'pytz.exceptions', 'pytz.lazy', 'pytz.tzinfo', 'pytz.tzfile', 'dateutil', 'dateutil._version', 'pandas.compat', 'pandas.compat.chainmap', 'dateutil.parser', 'dateutil.relativedelta', 'dateutil._common', 'dateutil.tz', 'dateutil.tz.tz', 'dateutil.tz._common', 'pandas.compat.numpy', 'pandas._libs', '_cython_0_27_2', 'pandas._libs.tslib', 'pandas._libs.tslibs', 'pandas._libs.tslibs.timedeltas', 'pandas._libs.tslibs.timezones', 'pandas._libs.tslibs.parsing', 'pandas._libs.tslibs.fields', 'pandas._libs.hashtable', 'pandas._libs.lib', 'pandas._libs.interval', 'decimal', '_decimal', 'pandas.core', 'pandas.core.config_init', 'pandas.core.config', 'pandas.io', 'pandas.io.formats', 'pandas.io.formats.printing', 'pandas.core.dtypes', 'pandas.core.dtypes.inference', 'pandas.io.formats.console', 'pandas.io.formats.terminal', 'pandas.core.api', 'pandas.core.algorithms', 'pandas.core.dtypes.cast', 'pandas.core.dtypes.common', 'pandas._libs.algos', 'pandas.core.dtypes.dtypes', 'pandas.core.dtypes.generic', 'pandas.core.dtypes.missing', 'pandas.core.common', 'pandas.api', 'pandas.api.types', 'pandas.core.dtypes.api', 'pandas.core.dtypes.concat', 'pandas.errors', 'pandas.core.categorical', 'pandas.core.accessor', 'pandas.core.base', 'pandas.util', 'pandas.util._decorators', 'pandas._libs.properties', 'pandas.core.util', 'pandas.core.util.hashing', 'pandas._libs.hashing', 'pandas.util._validators', 'pandas.core.nanops', 'bottleneck', 'bottleneck.reduce', 'bottleneck.nonreduce', 'bottleneck.nonreduce_axis', 'bottleneck.move', 'bottleneck.slow', 'bottleneck.slow.reduce', 'bottleneck.slow.nonreduce', 'bottleneck.slow.nonreduce_axis', 'bottleneck.slow.move', 'bottleneck.version', 'bottleneck.benchmark', 'bottleneck.benchmark.bench', 'bottleneck.benchmark.autotimeit', 'bottleneck.benchmark.bench_detailed', 'bottleneck.tests', 'bottleneck.tests.util', 'pandas.compat.numpy.function', 'pandas.core.missing', 'pandas.core.groupby', 'pandas.core.index', 'pandas.core.indexes', 'pandas.core.indexes.api', 'pandas.core.indexes.base', 'pandas._libs.index', 'pandas._libs.join', 'pandas.core.indexes.frozen', 'pandas.core.sorting', 'pandas.core.ops', 'pandas.core.strings', 'pandas.core.indexes.category', 'pandas.core.indexes.multi', 'pandas.core.indexes.interval', 'pandas.core.indexes.datetimes', 'pandas.core.indexes.numeric', 'pandas.tseries', 'pandas.tseries.frequencies', 'pandas.tseries.offsets', 'pandas.core.tools', 'pandas.core.tools.datetimes', 'pandas._libs.tslibs.strptime', 'dateutil.easter', 'pandas._libs.tslibs.frequencies', 'pandas.core.indexes.datetimelike', 'pandas.core.tools.timedeltas', 'pandas._libs.period', 'pandas.core.indexes.timedeltas', 'pandas.core.indexes.range', 'pandas.core.indexes.period', 'pandas.core.frame', 'pandas.core.generic', 'pandas.core.indexing', 'pandas.core.internals', 'pandas.core.sparse', 'pandas.core.sparse.array', 'pandas._libs.sparse', 'pandas.io.formats.format', 'pandas.io.common', 'csv', '_csv', 'mmap', 'pandas.io.formats.common', 'pandas.core.series', 'pandas.core.indexes.accessors', 'pandas.plotting', 'pandas.plotting._misc', 'pandas.plotting._style', 'pandas.plotting._compat', 'pandas.plotting._tools', 'pandas.plotting._core', 'pandas.core.window', 'pandas._libs.window', 'pandas.core.panel', 'pandas.core.reshape', 'pandas.core.reshape.util', 'pandas._libs.groupby', 'pandas.core.panel4d', 'pandas.core.panelnd', 'pandas.core.reshape.reshape', 'pandas.core.sparse.api', 'pandas.core.sparse.list', 'pandas.core.sparse.series', 'pandas.core.sparse.scipy_sparse', 'pandas.core.sparse.frame', 'pandas._libs.reshape', 'pandas.core.tools.numeric', 'pandas.util._depr_module', 'pandas.stats', 'pandas.stats.api', 'pandas.stats.moments', 'pandas.tseries.api', 'pandas.core.computation', 'pandas.core.computation.api', 'pandas.core.computation.eval', 'pandas.core.computation.scope', 'pandas.core.computation.engines', 'pandas.core.computation.align', 'pandas.core.computation.common', 'pandas.core.computation.ops', 'pandas.core.reshape.api', 'pandas.core.reshape.concat', 'pandas.core.reshape.merge', 'pandas.core.reshape.pivot', 'pandas.core.reshape.tile', 'pandas.tools', 'pandas.tools.plotting', 'pandas.util._print_versions', 'pandas.io.api', 'pandas.io.parsers', 'pandas.io.date_converters', 'pandas._libs.parsers', 'pandas.io.clipboards', 'pandas.io.excel', 'pandas._libs.json', 'pandas.compat.openpyxl_compat', 'pandas.io.pytables', 'pandas.core.computation.pytables', 'pandas.core.computation.expr', 'pandas.io.json', 'pandas.io.json.json', 'pandas.io.json.normalize', 'pandas.io.json.table_schema', 'pandas.io.html', 'pandas.io.sql', 'pandas.io.sas', 'pandas.io.sas.sasreader', 'pandas.io.feather_format', 'pandas.io.parquet', 'pandas.io.stata', 'pandas.io.pickle', 'pandas.compat.pickle_compat', 'pandas.io.packers', 'pandas.io.msgpack', 'pandas.io.msgpack.exceptions', 'pandas.io.msgpack._version', 'pandas.io.msgpack._packer', 'pandas.io.msgpack._unpacker', 'pandas.util._move', 'pandas.io.gbq', 'pandas.util._tester', 'pandas.testing', 'pandas.util.testing', 'pandas._libs.testing', 'pandas._version', 'xarray.core.pycompat', 'xarray.core.indexing', 'xarray.core.nputils', 'xarray.core.duck_array_ops', 'xarray.core.npcompat', 'xarray.core.dtypes', 'xarray.core.variable', 'xarray.core.common', 'xarray.core.formatting', 'xarray.core.options', 'xarray.core.ops', 'xarray.core.combine', 'xarray.core.merge', 'xarray.core.computation', 'xarray.core.extensions', 'xarray.core.dataarray', 'xarray.plot', 'xarray.plot.plot', 'xarray.plot.utils', 'xarray.plot.facetgrid', 'xarray.core.groupby', 'xarray.core.resample', 'xarray.core.rolling', 'xarray.core.dask_array_ops', 'xarray.core.accessors', 'xarray.core.coordinates', 'xarray.core.dataset', 'xarray.conventions', 'xarray.coding', 'xarray.coding.times', 'xarray.coding.variables', 'xarray.backends', 'xarray.backends.common', 'xarray.backends.memory', 'xarray.backends.netCDF4_', 'xarray.backends.netcdf3', 'xarray.backends.pydap_', 'xarray.backends.pynio_', 'xarray.backends.scipy_', 'xarray.backends.h5netcdf_', 'xarray.backends.zarr', 'xarray.backends.api', 'xarray.backends.rasterio_', 'xarray.version', 'xarray.util', 'xarray.util.print_versions', 'xarray.tutorial', 'xarray.ufuncs', 'xarray.testing', 'netCDF4', '_cython_0_27_3', 'netCDF4._netCDF4', 'netCDF4.utils', 'netcdftime', 'netcdftime._netcdftime', 'h5netcdf', 'h5netcdf.core', 'h5py', 'h5py._errors', 'h5py._conv', 'h5py.h5r', 'h5py._objects', 'h5py.defs', 'h5py.h5t', 'h5py.utils', 'h5py.h5', 'h5py.h5z', 'h5py.h5a', 'h5py.h5s', 'h5py.h5p', 'h5py.h5ac', 'h5py._proxy', 'h5py.h5d', 'h5py.h5ds', 'h5py.h5f', 'h5py.h5g', 'h5py.h5i', 'h5py.h5fd', 'h5py._hl', 'h5py._hl.filters', 'h5py._hl.base', 'h5py._hl.compat', 'h5py._hl.files', 'h5py._hl.group', 'h5py.h5o', 'h5py.h5l', 'h5py._hl.dataset', 'h5py._hl.selections', 'h5py._hl.selections2', 'h5py._hl.datatype', 'h5py.version', 'h5py._hl.attrs', 'h5py.tests', 'h5py.tests.common', 'h5py.tests.old', 'h5py.tests.old.test_attrs', 'h5py.highlevel', 'h5py.tests.old.test_attrs_data', 'h5py.tests.old.test_base', 'h5py.tests.old.test_dataset', 'h5py.tests.old.test_datatype', 'h5py.tests.old.test_dimension_scales', 'h5py.tests.old.test_file', 'h5py.tests.old.test_file_image', 'h5py.tests.old.test_group', 'h5py.tests.old.test_h5', 'h5py.tests.old.test_h5f', 'h5py.tests.old.test_h5p', 'h5py.tests.old.test_h5t', 'h5py.tests.old.test_objects', 'h5py.tests.old.test_selections', 'h5py.tests.old.test_slicing', 'h5py.tests.hl', 'h5py.tests.hl.test_dataset_getitem', 'h5py.tests.hl.test_dataset_swmr', 'h5py.tests.hl.test_dims_dimensionproxy', 'h5py.tests.hl.test_file', 'h5py.tests.hl.test_attribute_create', 'h5py.tests.hl.test_threads', 'h5py.tests.hl.test_datatype', 'h5netcdf.compat', 'h5netcdf.attrs', 'h5netcdf.dimensions', 'h5netcdf.utils', 'distributed', 'distributed.config', 'logging.config', 'logging.handlers', 'socketserver', 'distributed.compatibility', 'asyncio', 'asyncio.base_events', 'concurrent', 'concurrent.futures', 'concurrent.futures._base', 'concurrent.futures.process', 'multiprocessing.connection', '_multiprocessing', 'concurrent.futures.thread', 'asyncio.compat', 'asyncio.coroutines', 'asyncio.constants', 'asyncio.events', 'asyncio.base_futures', 'asyncio.log', 'asyncio.futures', 'asyncio.base_tasks', '_asyncio', 'asyncio.tasks', 'asyncio.locks', 'asyncio.protocols', 'asyncio.queues', 'asyncio.streams', 'asyncio.subprocess', 'asyncio.transports', 'asyncio.unix_events', 'asyncio.base_subprocess', 'asyncio.selector_events', 'asyncio.sslproto', 'html', 'html.entities', 'yaml', 'yaml.error', 'yaml.tokens', 'yaml.events', 'yaml.nodes', 'yaml.loader', 'yaml.reader', 'yaml.scanner', 'yaml.parser', 'yaml.composer', 'yaml.constructor', 'yaml.resolver', 'yaml.dumper', 'yaml.emitter', 'yaml.serializer', 'yaml.representer', 'yaml.cyaml', '_yaml', 'distributed.core', 'tornado', 'tornado.gen', 'tornado.concurrent', 'tornado.log', 'tornado.escape', 'tornado.util', 'tornado.speedups', 'curses', '_curses', 'tornado.stack_context', 'tornado.ioloop', 'tornado.platform', 'tornado.platform.auto', 'tornado.platform.posix', 'tornado.platform.common', 'tornado.platform.interface', 'tornado.platform.asyncio', 'tornado.locks', 'distributed.comm', 'distributed.comm.addressing', 'distributed.comm.registry', 'distributed.comm.core', 'distributed.metrics', 'psutil', 'psutil._common', 'psutil._compat', 'psutil._exceptions', 'psutil._psosx', 'psutil._psposix', 'psutil._psutil_osx', 'psutil._psutil_posix', 'distributed.utils', 'tblib', 'tblib.cpython', 'tblib.pickling_support', 'multiprocessing.forkserver', 'multiprocessing.semaphore_tracker', 'multiprocessing.spawn', 'distributed.comm.inproc', 'distributed.protocol', 'distributed.protocol.compression', 'distributed.protocol.core', 'msgpack', 'msgpack._version', 'msgpack.exceptions', 'msgpack._packer', 'msgpack._unpacker', 'distributed.protocol.serialize', 'distributed.protocol.pickle', 'distributed.protocol.utils', 'distributed.comm.tcp', 'tornado.netutil', 'certifi', 'certifi.core', 'encodings.idna', 'stringprep', 'tornado.iostream', 'tornado.tcpclient', 'tornado.tcpserver', 'tornado.process', 'distributed.comm.utils', 'distributed.sizeof', 'distributed.system_monitor', 'distributed.deploy', 'distributed.deploy.local', 'distributed.nanny', 'multiprocessing.queues', 'distributed.node', 'distributed.versions', 'distributed.process', 'distributed.proctitle', 'distributed.security', 'distributed.worker', 'distributed.profile', 'bokeh', 'bokeh.util', 'bokeh.util.version', 'bokeh._version', 'bokeh.util.logconfig', 'bokeh.settings', 'bokeh.util.paths', 'bokeh.util.warnings', 'bokeh.sampledata', 'six.moves.urllib', 'six.moves.urllib.request', 'bokeh.palettes', 'distributed.batched', 'distributed.diskutils', 'distributed.locket', 'distributed.preloading', 'filecmp', 'click', 'click.core', 'click.types', 'click._compat', 'click.exceptions', 'click.utils', 'click.globals', 'click.termui', 'click.formatting', 'click.parser', 'click._unicodefun', 'click.decorators', 'distributed.threadpoolexecutor', 'distributed._concurrent_futures_thread', 'distributed.utils_comm', 'distributed.utils_perf', 'distributed.scheduler', 'sortedcontainers', 'sortedcontainers.sortedlist', 'sortedcontainers.sortedset', 'sortedcontainers.sorteddict', 'distributed.publish', 'distributed.queues', 'tornado.queues', 'distributed.client', 'distributed.cfexecutor', 'distributed.recreate_exceptions', 'distributed.lock', 'distributed.stealing', 'distributed.diagnostics', 'distributed.diagnostics.graph_layout', 'distributed.diagnostics.plugin', 'distributed.diagnostics.progressbar', 'distributed.diagnostics.progress', 'distributed.variable', 'distributed.deploy.adaptive', 'distributed.deploy.ssh', 'distributed.worker_client', 'distributed._version', 'matplotlib', 'distutils.sysconfig', 'distutils.errors', 'matplotlib.cbook', 'matplotlib.cbook.deprecation', 'matplotlib.cbook._backports', 'matplotlib.compat', 'matplotlib.compat.subprocess', 'matplotlib.rcsetup', 'matplotlib.testing', 'matplotlib.fontconfig_pattern', 'pyparsing', 'matplotlib.colors', 'matplotlib._color_data', 'cycler', 'matplotlib._version'] DEBUG:shapely.geos:Trying `CDLL(/Users/davidh/anaconda/envs/polar2grid_py36/bin/../lib/libgeos_c.dylib)` DEBUG:shapely.geos:Library path: '/Users/davidh/anaconda/envs/polar2grid_py36/bin/../lib/libgeos_c.dylib' DEBUG:shapely.geos:DLL: DEBUG:shapely.geos:Trying `CDLL(/usr/lib/libc.dylib)` DEBUG:shapely.geos:Library path: '/usr/lib/libc.dylib' DEBUG:shapely.geos:DLL: DEBUG:pip.vcs:Registered VCS backend: git DEBUG:pip.vcs:Registered VCS backend: hg DEBUG:pip.vcs:Registered VCS backend: svn DEBUG:pip.vcs:Registered VCS backend: bzr INSTALLED VERSIONS ------------------ commit: None python: 3.6.4.final.0 python-bits: 64 OS: Darwin OS-release: 17.5.0 machine: x86_64 processor: i386 byteorder: little LC_ALL: None LANG: en_US.UTF-8 LOCALE: en_US.UTF-8 xarray: 0.10.1 pandas: 0.21.0 numpy: 1.13.3 scipy: 1.0.0 netCDF4: 1.3.1 h5netcdf: 0.5.0 h5py: 2.7.1 Nio: None zarr: None bottleneck: 1.2.1 cyordereddict: None dask: 0.17.1 distributed: 1.21.2 matplotlib: 2.2.0 cartopy: 0.16.0 seaborn: None setuptools: 39.0.1 pip: 9.0.1 conda: None pytest: 3.4.0 IPython: 6.1.0 sphinx: 1.6.6 ```
shoyer commented 6 years ago

I think we should probably call this a bug and just drop the call to utils.remove_incompatible_items() -- see https://github.com/pydata/xarray/issues/1614.

concat() is the only operation for which we check variables for equality.

djhoese commented 6 years ago

So this would mean concat would not retain any .attrs, right?

shoyer commented 6 years ago

So this would mean concat would not retain any .attrs, right?

concat would only retain .attrs from the first variable, ignoring all subsequent variables.