geosciz commented 7 years ago

cc: @rabernat, @nathanieltarshish

This pull request fixes #38 by removing the dtype option in df = dd.read_csv(input_path, names=float_columns, dtype=float_dtypes, header=None) and setting variable dtypes later.

codecov-io commented 7 years ago

Codecov Report

Merging #39 into master will decrease coverage by 0.21%. The diff coverage is 100%.

@@            Coverage Diff             @@
##           master      #39      +/-   ##
==========================================
- Coverage   67.31%   67.09%   -0.22%     
==========================================
  Files           4        4              
  Lines         465      462       -3     
  Branches       76       75       -1     
==========================================
- Hits          313      310       -3     
- Misses        126      127       +1     
+ Partials       26       25       -1

Impacted Files	Coverage Δ
floater/utils.py	`34.89% <100%> (-1.29%)`	:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update b4b2863...b8a70c1. Read the comment docs.

rabernat commented 7 years ago

What version of dask, pandas, and xarray are you using?

geosciz commented 7 years ago

@rabernat, here are my dask, pandas, and xarray versions: dask 0.14.1, pandas 0.19.2, and xarray 0.9.3.

geosciz commented 7 years ago

@rabernat Yes, I agree. I have tried to find the origin by looking into raw csv output. It seems that the only file causing this issue is /vega/physo/users/cz2397/global_lagrangian/run_test_rec/float_trajectories.0000000096.007.001.csv.

rabernat commented 7 years ago

I don't fully understand this issue, but if there is not a big performance penalty, I guess this is fine with me.

ocean-transport / floater

fix compound dtypes issue #39

Codecov Report