Closed xoen closed 6 years ago
Merging #54 into master will increase coverage by
0.6%
. The diff coverage is45.45%
.
@@ Coverage Diff @@
## master #54 +/- ##
=========================================
+ Coverage 73.87% 74.47% +0.6%
=========================================
Files 4 4
Lines 823 815 -8
=========================================
- Hits 608 607 -1
+ Misses 215 208 -7
Impacted Files | Coverage Δ | |
---|---|---|
etl_manager/etl.py | 58.48% <45.45%> (+1.58%) |
:arrow_up: |
Continue to review full report at Codecov.
Legend - Click here to learn more
Δ = absolute <relative> (impact)
,ø = not affected
,? = missing data
Powered by Codecov. Last update 1519f8e...c127e0d. Read the comment docs.
@isichei / @r4vi I probably moved more things than I originally planned so:
Regarding point 2, as said before, I think we should just go ahead and automatically format the code using black
which gives us the big advantage of not having to think about formatting. I can format using black and PR after this, I think this first step will make the diff less harsh so still good to have it.
@xoen nice refactoring if you're going all in on python 3.6 (fstrings) then you might want to look at pathlib for paths instead of os.path.exists stuff
also you could turn GlueJob into a context manager so client code can do something like
with GlueJob as j:
...do stuff
do_more_stuff()
and it'll do the cleanup when it comes out of the block
_create_glue_job_definition()
=>_job_definition()
(glue is redundant)_job_definition()
avoid unnecesary variables when building the dictionary unless they improve readabilityis_running
usesjob_run_state
instead of . Also removed unusedis_running
variable in it.glue_job_folder_split
in__init__()
)Style changes:
+
when code is still readable:
)NOTE: I'm wondering if we need to treat "RUNNING" as a special state and have the
is_running()
property instead of the client code using something likejob.job_run_state == 'RUNNING'
which is still clear.