ddf-project / DDF

Distributed DataFrame: Productivity = Power x Simplicity For Scientists & Engineers, on any Data Engine
http://ddf.io
Apache License 2.0
168 stars 42 forks source link

[PE-1859] Fix filesystem closed exception #306

Closed lebinh closed 8 years ago

lebinh commented 8 years ago

Description and related tickets, documents

Don't close the FileSystem instance as it is cached and shared by default.

The FileSystem instance returned by FileSystem.get is cached and shared by default, see https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java#L376-L381. Thus closing it will cause "Filesystem closed" exception on other shared instance.

Also rewrite the method to use try-with-resource instead of manually closing the Writer instance.

Reviewers: @Huandao0812

Breaking changes & backward compatible issues

NA.

How to test

NA.

PR Progress

Make sure all checkboxes below are checked before merged