Create a file using standard Python: open("test.txt", "w").write("Hello World!")
Run a flow with dump_to_path: DF.Flow(({'a': 1},{'a': 2}), DF.dump_to_path('test')).process()
Expected
Default OS file permissions (from umask):
$ ls -l test.txt
-rw-rw-r-- 1 ori ori 12 Aug 23 19:03 test.txt
$ ls -l test/
-rw-rw-r-- 1 ori ori 816 Aug 23 19:03 datapackage.json
-rw-rw-r-- 1 ori ori 9 Aug 23 19:03 res_1.csv
Actual
dump_to_path files have permissions to owner only:
$ ls -l test/
-rw------- 1 ori ori 816 Aug 23 19:06 datapackage.json
-rw------- 1 ori ori 9 Aug 23 19:06 res_1.csv
Explanation
Files created with FileDumper are first created as temporary files which have permissions only for file owner, and are are later copied over to the final location. They are copied with the limited temporary file permissions which is unexpected and limited. This PR fixes it by setting their permissions to the default OS permissions as set by umask.
Reproduction Steps
open("test.txt", "w").write("Hello World!")
DF.Flow(({'a': 1},{'a': 2}), DF.dump_to_path('test')).process()
Expected
Actual
Explanation
Files created with FileDumper are first created as temporary files which have permissions only for file owner, and are are later copied over to the final location. They are copied with the limited temporary file permissions which is unexpected and limited. This PR fixes it by setting their permissions to the default OS permissions as set by umask.
See more details about this problem and the solution here: https://stackoverflow.com/questions/7150826/how-can-i-get-the-default-file-permissions-in-python