kcigeospatial / balt_co_ETL

This will provide a place to track internal issues around the Baltimore County NPDES NCT application ETL.
0 stars 0 forks source link

QuarterlyGradingPmtInfo - LAND_USE_AF No Values Populating in Export #20

Closed JuanitaMiles closed 7 years ago

JuanitaMiles commented 7 years ago

Steps to Reproduce: Reporting Year = 2017 On Failure = Continue on Errors Current User = KCI\Juanita.Miles

No values are being populated the MDE Export, QuarterlyGradingPmtInfo.LAND_USE_AF.

Source (QRT_GRADING_INFO.LAND_USE_AF): image MDE Export (QuarterlyGradingPmtInfo.LAND_USE_AF): image

MDE Export should populate with the MDPLandUse Domain Descriptions. 13 --> High Density Residential 17 --> Extractive 14 --> Commercial 13 --> High Density Residential 15 --> Industrial

gerrykelly commented 7 years ago

@JuanitaMiles I don't see that this comes from any issue with the ETL code. I'm not sure why it is happening so I'll turn this back to you with a test FGDB and some info to see if you can catch anything at the ArcGIS level.

The FGDB is \vm-wsscmta\MDEdbExports\LAND_USE_AF__test.gdb

The step in the ETL that should populate LAND_USE_AF is when QuarterlyGradingPmtInfo_p gets appended to the QuarterlyGradingPmtInfo table.

You'll see that the AF & BF field types (that started as text at the county) are both correct: image

If you went to ArcMap you'd see that the fields show like this ... image image

and when you show codes instead, like this: image

In the ETL, the Append TEST works, so I don't see why BF gets populated and AF does not. If you manually in ArcCatalog do the append you get the same result.

Looking at NO_TEST, the fields map so it looks like it should work image

And no error are returned in the geoprocessing results: image

Let me know if you can find anything

gerrykelly commented 7 years ago

@JuanitaMiles @leeensminger You can still look into this if you'd like, but later I happened to catch that LAND_USE_AF is a subtype field in the MDE FGDB but not for BaltCO. That's a schema difference and probably the cause of this. So, I'll remove my assignment until you get direction on dealing with that (I wonder if it is even intentional by MDE in the first place)

image

JuanitaMiles commented 7 years ago

@gerrykelly @leeensminger Nice catch Gerry. I'm glad you found this, since I, to be honest, don't know how to access that FGDB you created for me...I'll put this on my list of topics for Friday (both, the schema change and my lack of knowledge...)

JuanitaMiles commented 7 years ago

@gerrykelly @leeensminger Okay so I figured out how to access the LAND_USE_AF_test.gdb (Duh). I copied it to my working folder...tried the append once with the subtype applied got the same results...then removed the subtype, ran the append again and believe it or not EVERYTHING, including all the other domains that didn't display properly from the other issue I logged, comes out perfect!

image

So I think we can kill two birds with one stone by hopefully removing the LAND_USE_AF subtype.

leeensminger commented 7 years ago

Yes this is an erroneous subtype in the MDE schema. Should be removed from the target MDE database, as it is a mistake.