data: Data Flow in app when raw items updated

[odscjames]

https://github.com/openactive/conformance-services/issues/79 https://github.com/openactive/conformance-services/issues/42

[odscjames]

@rhiaro This needs some tests work but you may want to look at what I'm doing here and chat with me to check out the proposed workflow

[odscjames]

Bug in process when raw item updated

[odscjames]

Make deletes cascade from source to normalised data

[odscjames]

I also run a manual test by putting some data in DB,

Running:

select count(*) from raw_data where normalised  = TRUE;  
select count(*) from raw_data where validation_done = TRUE;  
select count(*) from normalised_data where data_deleted = FALSE; 
select count(*) from normalised_data_profile_results ;

Then started again:

update publisher_feed set raw_next_url=NULL;

Then run only the download raw data process and you should see every one of those 4 counts going down - and I did!