search

moj-analytical-services / etl_manager

A python package to create a database on the platform using our moj data warehousing framework
21 stars 9 forks source link

Updating package to point to new web-schema location. #102

Closed isichei closed 5 years ago

isichei commented 5 years ago

Also added unit test to check if local schema matches prescribed web-schema

RobinL commented 5 years ago

One question: shouldn't we just directly use the web version rather than having a local version as well?

isichei commented 5 years ago

Yeah was thinking that. Thought it might still be useful to be self contained let's say if you don't have internet access.

I've added this to make sure they do align. So will fail if they go out of sync.

If we wanted to reference the weblink directly we could do something like:

try:
    schema = get_url("https://schema.json")
except url_error:
    schema = get_local_table()
RobinL commented 5 years ago

No strong view, can see benefits of doing it both ways.

I think prob my preferance is to use your try except and raise a warning if it hits the except line.