search

data-apis / python-record-api

Inferring Python API signatures from tracing usage.
MIT License
75 stars 6 forks source link

Record return types and determine exposed modules for objects #15

Open saulshanabrook opened 4 years ago

saulshanabrook commented 4 years ago

One thing we should add is the ability to record the return type from a function.

This would help us understand the signatures better.

Also, it could help with another issue, which is that currently we output functions/classes from the modules they are defined in, not the module they are imported from.

We should instead try to output them where they are imported from. However, if multiple libraries import from different locations, we should choose one and have the others be aliases.

One way to do that would be to record the return types from the getattr calls, so if you did something like import numpy; numpy.arange we would know the return type of the getattr is the arange function.

We currently record those getattr, but don't have the return type.

If we did, we could infer that any getattr from a module, which returns a class/fn from a different module represents an import alias.

saulshanabrook commented 4 years ago

We are currently looking at the Python bytecode stack to understand what operations are called, see:

I think for getting the return values of functions, we might have to basically wait till the next bytecode is executed and then look at the top value of the stack to see what was left there by the execution.

saulshanabrook commented 4 years ago

This was merged in #78 but I had to revert that in https://github.com/data-apis/python-record-api/pull/94 because it was causing segfaults.

Of koalas:

Ivy Default Cache set to: /root/.ivy2/cache
The jars for the packages stored in: /root/.ivy2/jars
:: loading settings :: url = jar:file:/usr/local/lib/python3.8/site-packages/pyspark/jars/ivy-2.4.0.jar!/org/apache/ivy/core/settings/ivysettings.xml
io.delta#delta-core_2.12 added as a dependency
:: resolving dependencies :: org.apache.spark#spark-submit-parent-e4659a54-a70a-4cb1-9422-368fc8c06f48;1.0
    confs: [default]
    found io.delta#delta-core_2.12;0.7.0 in central
    found org.antlr#antlr4;4.7 in central
    found org.antlr#antlr4-runtime;4.7 in central
    found org.antlr#antlr-runtime;3.5.2 in central
    found org.antlr#ST4;4.0.8 in central
    found org.abego.treelayout#org.abego.treelayout.core;1.0.3 in central
    found org.glassfish#javax.json;1.0.4 in central
    found com.ibm.icu#icu4j;58.2 in central
downloading https://repo1.maven.org/maven2/io/delta/delta-core_2.12/0.7.0/delta-core_2.12-0.7.0.jar ...
    [SUCCESSFUL ] io.delta#delta-core_2.12;0.7.0!delta-core_2.12.jar (91ms)
downloading https://repo1.maven.org/maven2/org/antlr/antlr4/4.7/antlr4-4.7.jar ...
    [SUCCESSFUL ] org.antlr#antlr4;4.7!antlr4.jar (58ms)
downloading https://repo1.maven.org/maven2/org/antlr/antlr4-runtime/4.7/antlr4-runtime-4.7.jar ...
    [SUCCESSFUL ] org.antlr#antlr4-runtime;4.7!antlr4-runtime.jar (11ms)
downloading https://repo1.maven.org/maven2/org/antlr/antlr-runtime/3.5.2/antlr-runtime-3.5.2.jar ...
    [SUCCESSFUL ] org.antlr#antlr-runtime;3.5.2!antlr-runtime.jar (7ms)
downloading https://repo1.maven.org/maven2/org/antlr/ST4/4.0.8/ST4-4.0.8.jar ...
    [SUCCESSFUL ] org.antlr#ST4;4.0.8!ST4.jar (9ms)
downloading https://repo1.maven.org/maven2/org/abego/treelayout/org.abego.treelayout.core/1.0.3/org.abego.treelayout.core-1.0.3.jar ...
    [SUCCESSFUL ] org.abego.treelayout#org.abego.treelayout.core;1.0.3!org.abego.treelayout.core.jar(bundle) (3ms)
downloading https://repo1.maven.org/maven2/org/glassfish/javax.json/1.0.4/javax.json-1.0.4.jar ...
    [SUCCESSFUL ] org.glassfish#javax.json;1.0.4!javax.json.jar(bundle) (6ms)
downloading https://repo1.maven.org/maven2/com/ibm/icu/icu4j/58.2/icu4j-58.2.jar ...
    [SUCCESSFUL ] com.ibm.icu#icu4j;58.2!icu4j.jar (269ms)
:: resolution report :: resolve 2981ms :: artifacts dl 464ms
    :: modules in use:
    com.ibm.icu#icu4j;58.2 from central in [default]
    io.delta#delta-core_2.12;0.7.0 from central in [default]
    org.abego.treelayout#org.abego.treelayout.core;1.0.3 from central in [default]
    org.antlr#ST4;4.0.8 from central in [default]
    org.antlr#antlr-runtime;3.5.2 from central in [default]
    org.antlr#antlr4;4.7 from central in [default]
    org.antlr#antlr4-runtime;4.7 from central in [default]
    org.glassfish#javax.json;1.0.4 from central in [default]
    ---------------------------------------------------------------------
    |                  |            modules            ||   artifacts   |
    |       conf       | number| search|dwnlded|evicted|| number|dwnlded|
    ---------------------------------------------------------------------
    |      default     |   8   |   8   |   8   |   0   ||   8   |   8   |
    ---------------------------------------------------------------------
:: retrieving :: org.apache.spark#spark-submit-parent-e4659a54-a70a-4cb1-9422-368fc8c06f48
    confs: [default]
    8 artifacts copied, 0 already retrieved (15071kB/51ms)
20/10/28 15:44:37 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).

[Stage 0:>                                                          (0 + 1) / 1]

20/10/28 15:44:50 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.

[Stage 2:>                                                          (0 + 2) / 2]

20/10/28 15:44:54 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.

[Stage 4:=============================>                             (1 + 1) / 2]

20/10/28 15:44:57 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.

[Stage 10:=============================>                            (1 + 1) / 2]

20/10/28 15:45:03 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
20/10/28 15:45:07 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
20/10/28 15:45:11 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
20/10/28 15:45:12 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
20/10/28 15:45:40 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
20/10/28 15:45:41 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
20/10/28 15:45:45 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
20/10/28 15:45:46 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
20/10/28 15:45:47 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
20/10/28 15:45:48 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
20/10/28 15:45:49 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
20/10/28 15:45:50 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
20/10/28 15:45:51 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
20/10/28 15:45:51 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
20/10/28 15:45:52 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
20/10/28 15:45:54 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
20/10/28 15:45:55 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
20/10/28 15:45:55 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
20/10/28 15:45:56 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
20/10/28 15:45:57 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
20/10/28 15:45:58 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
20/10/28 15:45:59 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
20/10/28 15:45:59 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
20/10/28 15:46:00 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
============================= test session starts ==============================
platform linux -- Python 3.8.4, pytest-6.1.1, py-1.9.0, pluggy-0.13.1
rootdir: /usr/src/app
plugins: cov-2.10.1, record_api-1.3.0
collected 1201 items / 6 deselected / 1195 selected

databricks/koalas/accessors.py ....
20/10/28 15:46:01 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
20/10/28 15:46:02 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
20/10/28 15:46:09 WARN package: Truncated the string representation of a plan since it was too large. This behavior can be adjusted by setting 'spark.sql.debug.maxToStringFields'.
20/10/28 15:46:12 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
20/10/28 15:46:13 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
20/10/28 15:46:13 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
databricks/koalas/base.py ................
databricks/koalas/config.py ...
databricks/koalas/datetimes.py ................
databricks/koalas/extensions.py ....
20/10/28 15:47:02 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
20/10/28 15:47:23 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
20/10/28 15:47:25 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
20/10/28 15:47:31 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
20/10/28 15:47:34 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.

[Stage 438:>                                                        (0 + 2) / 2]

20/10/28 15:47:37 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
20/10/28 15:47:42 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
20/10/28 15:47:44 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
20/10/28 15:47:47 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
20/10/28 15:47:48 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
20/10/28 15:47:51 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
20/10/28 15:48:11 WARN BLAS: Failed to load implementation from: com.github.fommil.netlib.NativeSystemBLAS
20/10/28 15:48:11 WARN BLAS: Failed to load implementation from: com.github.fommil.netlib.NativeRefBLAS
20/10/28 15:48:26 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
20/10/28 15:48:28 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
20/10/28 15:48:28 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
20/10/28 15:49:07 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
20/10/28 15:49:08 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
20/10/28 15:49:18 WARN WindowExec: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
Fatal Python error: Segmentation fault

Thread 0x00007ff09248a700 (most recent call first):
  File "/usr/local/lib/python3.8/site-packages/pyspark/accumulators.py", line 239 in poll
  File "/usr/local/lib/python3.8/site-packages/pyspark/accumulators.py", line 268 in handle
  File "/usr/local/lib/python3.8/socketserver.py", line 720 in __init__
  File "/usr/local/lib/python3.8/socketserver.py", line 360 in finish_request
  File "/usr/local/lib/python3.8/socketserver.py", line 347 in process_request
  File "/usr/local/lib/python3.8/socketserver.py", line 316 in _handle_request_noblock
  File "/usr/local/lib/python3.8/socketserver.py", line 237 in serve_forever
  File "/usr/local/lib/python3.8/threading.py", line 870 in run
  File "/usr/local/lib/python3.8/threading.py", line 932 in _bootstrap_inner
  File "/usr/local/lib/python3.8/threading.py", line 890 in _bootstrap

Current thread 0x00007ff0965a8740 (most recent call first):
  File "/usr/local/lib/python3.8/site-packages/record_api/get_stack.py", line 70 in __getitem__
  File "/usr/local/lib/python3.8/site-packages/record_api/core.py", line 349 in TOS
  File "/usr/local/lib/python3.8/site-packages/record_api/core.py", line 423 in log_called_method
  File "/usr/local/lib/python3.8/site-packages/record_api/core.py", line 414 in __call__
  File "/usr/local/lib/python3.8/site-packages/record_api/core.py", line 634 in __call__
  File "<doctest databricks.koalas.frame.DataFrame.isnull[0]>", line 1 in <module>
  File "/usr/local/lib/python3.8/doctest.py", line 1336 in __run
  File "/usr/local/lib/python3.8/doctest.py", line 1483 in run
  File "/usr/local/lib/python3.8/doctest.py", line 1844 in run
  File "/usr/local/lib/python3.8/site-packages/_pytest/doctest.py", line 287 in runtest
  File "/usr/local/lib/python3.8/site-packages/_pytest/runner.py", line 163 in pytest_runtest_call
  File "/usr/local/lib/python3.8/site-packages/pluggy/callers.py", line 187 in _multicall
  File "/usr/local/lib/python3.8/site-packages/pluggy/manager.py", line 84 in <lambda>
  File "/usr/local/lib/python3.8/site-packages/pluggy/manager.py", line 93 in _hookexec
  File "/usr/local/lib/python3.8/site-packages/pluggy/hooks.py", line 286 in __call__
  File "/usr/local/lib/python3.8/site-packages/_pytest/runner.py", line 256 in <lambda>
  File "/usr/local/lib/python3.8/site-packages/_pytest/runner.py", line 310 in from_call
  File "/usr/local/lib/python3.8/site-packages/_pytest/runner.py", line 255 in call_runtest_hook
  File "/usr/local/lib/python3.8/site-packages/_pytest/runner.py", line 216 in call_and_report
  File "/usr/local/lib/python3.8/site-packages/_pytest/runner.py", line 127 in runtestprotocol
  File "/usr/local/lib/python3.8/site-packages/_pytest/runner.py", line 110 in pytest_runtest_protocol
  File "/usr/local/lib/python3.8/site-packages/pluggy/callers.py", line 187 in _multicall
  File "/usr/local/lib/python3.8/site-packages/pluggy/manager.py", line 84 in <lambda>
  File "/usr/local/lib/python3.8/site-packages/pluggy/manager.py", line 93 in _hookexec
  File "/usr/local/lib/python3.8/site-packages/pluggy/hooks.py", line 286 in __call__
  File "/usr/local/lib/python3.8/site-packages/_pytest/main.py", line 338 in pytest_runtestloop
  File "/usr/local/lib/python3.8/site-packages/pluggy/callers.py", line 187 in _multicall
  File "/usr/local/lib/python3.8/site-packages/pluggy/manager.py", line 84 in <lambda>
  File "/usr/local/lib/python3.8/site-packages/pluggy/manager.py", line 93 in _hookexec
  File "/usr/local/lib/python3.8/site-packages/pluggy/hooks.py", line 286 in __call__
  File "/usr/local/lib/python3.8/site-packages/_pytest/main.py", line 313 in _main
  File "/usr/local/lib/python3.8/site-packages/_pytest/main.py", line 257 in wrap_session
  File "/usr/local/lib/python3.8/site-packages/_pytest/main.py", line 306 in pytest_cmdline_main
  File "/usr/local/lib/python3.8/site-packages/pluggy/callers.py", line 187 in _multicall
  File "/usr/local/lib/python3.8/site-packages/pluggy/manager.py", line 84 in <lambda>
  File "/usr/local/lib/python3.8/site-packages/pluggy/manager.py", line 93 in _hookexec
  File "/usr/local/lib/python3.8/site-packages/pluggy/hooks.py", line 286 in __call__
  File "/usr/local/lib/python3.8/site-packages/_pytest/config/__init__.py", line 164 in main
  File "/usr/local/lib/python3.8/site-packages/_pytest/config/__init__.py", line 187 in console_main
  File "/usr/local/lib/python3.8/site-packages/pytest/__main__.py", line 5 in <module>
  File "/usr/local/lib/python3.8/runpy.py", line 87 in _run_code
  File "/usr/local/lib/python3.8/runpy.py", line 194 in _run_module_as_main
./dev/pytest: line 56:    11 Segmentation fault      (core dumped) $PYTHON_EXECUTABLE -m pytest "$@"
databricks/koalas/frame.py .............................................F..FF

and xarray:

============================= test session starts ==============================
platform linux -- Python 3.8.4, pytest-6.1.1, py-1.9.0, pluggy-0.13.1
rootdir: /usr/src/app, configfile: setup.cfg
plugins: env-0.6.2, cov-2.10.1, custom-exit-code-0.3.0, hypothesis-5.38.1, record_api-1.3.0
collected 11772 items / 2 skipped / 11770 selected

xarray/tests/test_accessor_str.py ...................................... [  0%]
.................                                                        [  0%]
xarray/tests/test_backends.py .FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFxFFFFFFFFFF [  0%]
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFxFFFFFFFFFF [  1%]
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF...FFFFFFFFFFFFFFFFFFFFFFFFFFFFFxFFFFFFFF [  2%]
FFFFFFFFFFFFFFFFFFF.xFxFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFxFFFFFFFFFF [  2%]
FFFFFFFFFFFFFFFFF.xFxFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFxFFFFFFFFFFF [  3%]
FFFFFFF.FFFFFFFFFFFFFFFFFFFFFFFFFFFFFxFFFFFFFFFFFFFFFFFFssFFFFFFFFFFFFFF [  3%]
FFFFFFFFFFFFFFFFxFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFxFF [  4%]
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFxFFFFFFFFFFFFFFFFFFFFFFFF [  5%]
FFFFFFFFFFFFFFFFFFFFFFFxFFFFFFFFFFFFFFFFFF.FFFFFFFFFFFFFFFFFFFFFFFFFFFFF [  5%]
FFFFxFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [  6%]
FFFFFFFFxFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF.FFFFFFFFFFFFFFFFF [  6%]
FFFFFFFFFFFFFFxFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF..FFFFFFFFFF [  7%]
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [  8%]
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF..FF..FFFFFFFFFFxFFFFFFFFFFFFFFFFFF [  8%]
FFFFFFsssssssssssssssssssssssssssssssssssssssssssssssssssssssssFFFFFFFFF [  9%]
FFFFFFFFsFxFsFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [  9%]
FFFFFFFFFFFFFFFFF                                                        [ 10%]
xarray/tests/test_backends_api.py F                                      [ 10%]
xarray/tests/test_backends_common.py F                                   [ 10%]
xarray/tests/test_backends_file_manager.py FFFFFFFFFFFFFFFFFFFFFFFFFF... [ 10%]
..                                                                       [ 10%]
xarray/tests/test_backends_locks.py .                                    [ 10%]
xarray/tests/test_backends_lru_cache.py ........                         [ 10%]
xarray/tests/test_cftime_offsets.py .................................... [ 10%]
........................................................................ [ 11%]
........................................................................ [ 12%]
........................................................................ [ 12%]
........................................................................ [ 13%]
........................................................................ [ 13%]
........................................................................ [ 14%]
........................................................................ [ 15%]
........................................................................ [ 15%]
........................................................................ [ 16%]
........................................................................ [ 16%]
........................................................................ [ 17%]
........................................................................ [ 18%]
........................................................................ [ 18%]
........................................................................ [ 19%]
........................................................................ [ 19%]
........................................................................ [ 20%]
........................................................................ [ 21%]
........................................................................ [ 21%]
........................................................................ [ 22%]
........................................................................ [ 23%]
........................................................................ [ 23%]
........................................................................ [ 24%]
........................................................................ [ 24%]
........................................................................ [ 25%]
........................................................................ [ 26%]
........................................................................ [ 26%]
........................................................................ [ 27%]
........................................................................ [ 27%]
..........                                                               [ 28%]
xarray/tests/test_cftimeindex.py ....................................... [ 28%]
........................................................................ [ 28%]
........................................................................ [ 29%]
...........................................................FFFFFFFFFFFFF [ 30%]
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 30%]
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 31%]
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 32%]
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 32%]
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF............... [ 33%]
...............................................x........................ [ 33%]
.................                                                        [ 33%]
xarray/tests/test_cftimeindex_resample.py FFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 34%]
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 34%]
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 35%]
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 36%]
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 36%]
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 37%]
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 37%]
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF                                  [ 38%]
xarray/tests/test_coding.py FFFFFFFFFFFFF                                [ 38%]
xarray/tests/test_coding_strings.py FFFFFFFFFFFFFFFFFFFFF                [ 38%]
xarray/tests/test_coding_times.py FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 38%]
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF.......................... [ 39%]
.......................FFFFFFF.......................................... [ 40%]
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 40%]
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 41%]
F                                                                        [ 41%]
xarray/tests/test_combine.py FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF.xFFF [ 41%]
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF                                     [ 41%]
xarray/tests/test_computation.py FFFFFFFFFFFFFFFFFFFFF...............    [ 42%]
xarray/tests/test_concat.py FFFFFFFF..FFFFFFFFFFFFFFFFFFFFF.FFFFFFFFFFF  [ 42%]
xarray/tests/test_conventions.py FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 42%]
FFFFFFFFFxFFFFFFFFFFFFFFFs...                                            [ 43%]
xarray/tests/test_dataarray.py FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF..FFFFFF [ 43%]
FFFF...FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF...FFFFFF [ 44%]
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFsFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF. [ 44%]
.FFFF.FFFFFFFFFFFFFFFFFFFFFF....FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 45%]
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 46%]
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 46%]
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 47%]
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 47%]
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 48%]
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 49%]
FFFFFFFFFFFFFsssssssssssssFFFFFFFFFFF                                    [ 49%]
xarray/tests/test_dataset.py FFFFFFF.FF.F.FFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 49%]
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 50%]
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF.FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 50%]
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 51%]
FFFFFFFFFFFFFFFFFFFFFF.....FFFFFFFFF.FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 52%]
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 52%]
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 53%]
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 54%]
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 54%]
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 55%]
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 55%]
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 56%]
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF                     [ 56%]
xarray/tests/test_distributed.py FFFFFFFFFFFFFFFFFsF                     [ 57%]
xarray/tests/test_dtypes.py FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF      [ 57%]
xarray/tests/test_duck_array_ops.py FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 57%]
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 58%]
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 58%]
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 59%]
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 60%]
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 60%]
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 61%]
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 62%]
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 62%]
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 63%]
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [ 63%]
FFFFFFFFFFFFFFFFFFFFFFFFFFF                                              [ 64%]
xarray/tests/test_extensions.py FFFF                                     [ 64%]
xarray/tests/test_formatting.py FFFFFFFFFFFFFFFFF                        [ 64%]
xarray/tests/test_formatting_html.py FFFFFFFFFFFF                        [ 64%]
xarray/tests/test_groupby.py FFFFFFFFFFFFFFFFFFFFFFFFFFFFFF.F            [ 64%]
xarray/tests/test_indexing.py FFFFFFFFFFFFFFFFFFF....................... [ 64%]
Fatal Python error: Segmentation fault

Thread 0x00007f7542a0b700 (most recent call first):
  File "/usr/local/lib/python3.8/threading.py", line 306 in wait
  File "/usr/local/lib/python3.8/queue.py", line 179 in get
  File "/usr/local/lib/python3.8/site-packages/distributed/threadpoolexecutor.py", line 51 in _worker
  File "/usr/local/lib/python3.8/threading.py", line 870 in run
  File "/usr/local/lib/python3.8/threading.py", line 932 in _bootstrap_inner
  File "/usr/local/lib/python3.8/threading.py", line 890 in _bootstrap

Thread 0x00007f754320c700 (most recent call first):
  File "/usr/local/lib/python3.8/threading.py", line 306 in wait
  File "/usr/local/lib/python3.8/queue.py", line 179 in get
  File "/usr/local/lib/python3.8/site-packages/distributed/threadpoolexecutor.py", line 51 in _worker
  File "/usr/local/lib/python3.8/threading.py", line 870 in run
  File "/usr/local/lib/python3.8/threading.py", line 932 in _bootstrap_inner
  File "/usr/local/lib/python3.8/threading.py", line 890 in _bootstrap

Thread 0x00007f758268a700 (most recent call first):
  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 78 in _worker
  File "/usr/local/lib/python3.8/threading.py", line 870 in run
  File "/usr/local/lib/python3.8/threading.py", line 932 in _bootstrap_inner
  File "/usr/local/lib/python3.8/threading.py", line 890 in _bootstrap

Current thread 0x00007f7587164740 (most recent call first):
  File "/usr/local/lib/python3.8/site-packages/record_api/get_stack.py", line 70 in __getitem__
  File "/usr/local/lib/python3.8/site-packages/record_api/core.py", line 349 in TOS
  File "/usr/local/lib/python3.8/site-packages/record_api/core.py", line 423 in log_called_method
  File "/usr/local/lib/python3.8/site-packages/record_api/core.py", line 414 in __call__
  File "/usr/local/lib/python3.8/site-packages/record_api/core.py", line 634 in __call__
  File "/usr/src/app/xarray/tests/test_indexing.py", line 730 in test_create_mask_error
  File "/usr/local/lib/python3.8/site-packages/_pytest/python.py", line 184 in pytest_pyfunc_call
  File "/usr/local/lib/python3.8/site-packages/pluggy/callers.py", line 187 in _multicall
  File "/usr/local/lib/python3.8/site-packages/pluggy/manager.py", line 84 in <lambda>
  File "/usr/local/lib/python3.8/site-packages/pluggy/manager.py", line 93 in _hookexec
  File "/usr/local/lib/python3.8/site-packages/pluggy/hooks.py", line 286 in __call__
  File "/usr/local/lib/python3.8/site-packages/_pytest/python.py", line 1627 in runtest
  File "/usr/local/lib/python3.8/site-packages/_pytest/runner.py", line 163 in pytest_runtest_call
  File "/usr/local/lib/python3.8/site-packages/pluggy/callers.py", line 187 in _multicall
  File "/usr/local/lib/python3.8/site-packages/pluggy/manager.py", line 84 in <lambda>
  File "/usr/local/lib/python3.8/site-packages/pluggy/manager.py", line 93 in _hookexec
  File "/usr/local/lib/python3.8/site-packages/pluggy/hooks.py", line 286 in __call__
  File "/usr/local/lib/python3.8/site-packages/_pytest/runner.py", line 256 in <lambda>
  File "/usr/local/lib/python3.8/site-packages/_pytest/runner.py", line 310 in from_call
  File "/usr/local/lib/python3.8/site-packages/_pytest/runner.py", line 255 in call_runtest_hook
  File "/usr/local/lib/python3.8/site-packages/_pytest/runner.py", line 216 in call_and_report
  File "/usr/local/lib/python3.8/site-packages/_pytest/runner.py", line 127 in runtestprotocol
  File "/usr/local/lib/python3.8/site-packages/_pytest/runner.py", line 110 in pytest_runtest_protocol
  File "/usr/local/lib/python3.8/site-packages/pluggy/callers.py", line 187 in _multicall
  File "/usr/local/lib/python3.8/site-packages/pluggy/manager.py", line 84 in <lambda>
  File "/usr/local/lib/python3.8/site-packages/pluggy/manager.py", line 93 in _hookexec
  File "/usr/local/lib/python3.8/site-packages/pluggy/hooks.py", line 286 in __call__
  File "/usr/local/lib/python3.8/site-packages/_pytest/main.py", line 338 in pytest_runtestloop
  File "/usr/local/lib/python3.8/site-packages/pluggy/callers.py", line 187 in _multicall
  File "/usr/local/lib/python3.8/site-packages/pluggy/manager.py", line 84 in <lambda>
  File "/usr/local/lib/python3.8/site-packages/pluggy/manager.py", line 93 in _hookexec
  File "/usr/local/lib/python3.8/site-packages/pluggy/hooks.py", line 286 in __call__
  File "/usr/local/lib/python3.8/site-packages/_pytest/main.py", line 313 in _main
  File "/usr/local/lib/python3.8/site-packages/_pytest/main.py", line 257 in wrap_session
  File "/usr/local/lib/python3.8/site-packages/_pytest/main.py", line 306 in pytest_cmdline_main
  File "/usr/local/lib/python3.8/site-packages/pluggy/callers.py", line 187 in _multicall
  File "/usr/local/lib/python3.8/site-packages/pluggy/manager.py", line 84 in <lambda>
  File "/usr/local/lib/python3.8/site-packages/pluggy/manager.py", line 93 in _hookexec
  File "/usr/local/lib/python3.8/site-packages/pluggy/hooks.py", line 286 in __call__
  File "/usr/local/lib/python3.8/site-packages/_pytest/config/__init__.py", line 164 in main
  File "/usr/local/lib/python3.8/site-packages/_pytest/config/__init__.py", line 187 in console_main
  File "/usr/local/bin/pytest", line 8 in <module>
................................................................FFFFFFF

and pandas and skimage.

We should re-try that branch locally with one of those to track down the segfault and fix it.

aktech commented 4 years ago

Ah, I will take a look.