search

moj-analytical-services / etl_manager

A python package to create a database on the platform using our moj data warehousing framework
21 stars 9 forks source link

added optional glue_specific metadata #24

Closed RobinL closed 6 years ago

RobinL commented 6 years ago

User can now optionally specify a glue_specific key in the table metadata. Any entries will override/append to default dicts, using the _dict_merge function.

I'm using this to allow our metadata to include things like this:

"glue_specific": {
        "Parameters": {
            "skip.header.line.count": "1"
        },
        "StorageDescriptor": {
            "Parameters": {
                "skip.header.line.count": "1"
            }
        }
    }
codecov-io commented 6 years ago

Codecov Report

Merging #24 into master will decrease coverage by 0.05%. The diff coverage is 60%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master      #24      +/-   ##
==========================================
- Coverage   67.57%   67.51%   -0.06%     
==========================================
  Files           4        4              
  Lines         660      665       +5     
==========================================
+ Hits          446      449       +3     
- Misses        214      216       +2
Impacted Files Coverage Δ
etl_manager/meta.py 65.58% <60%> (-0.12%) :arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update decc2d1...0f6ed42. Read the comment docs.