felix-reichel / price-search-engine-seals-analysis

Produces a price search engine firm quality seal changes data set of (potentially) skewed index-spaced data cubes within a big data cube.
0 stars 0 forks source link

Impl Business Layer Abstraction 2.0 #29

Open felix-reichel opened 3 weeks ago

felix-reichel commented 3 weeks ago

variableDomain? (allDBDataForVariableDomainIsLoaded?) -> processInWeekBatches(variableDomain) processInMonthlyBatches(variableDomain)

(a) Determine Domain (b) Determine Render Strategy (c) Call Loaders (d) Compute From below (e) Store To Grid/err: rollback


CASE UNREALISTIC: GIVEN (1) the fully constructed (max. covering) (i,j,t)-results_space exists,

AND (2) ALL DATA IS CURRENTLY LOADED INTO DB

FOR EVERY possible args/(i,j,t) combination perform a:

(PSEUDO-CODE) ROUTINE PART 1: for a single args/(i,j,t)-variant variable in a fully constructed (max. covering) (i,j,t)-results_space:


Variable_name_arg = "clicks_"

// variable_does_even_exist_lookup()

variant_result =  ‘ijt’     # args, var variancy result; needed for WHERE-Clause

// Perform a check
check(*args, variant_result):
    i = args[0]
    i is Not Null & j is not null & t is not null?

Then calculate unix time bounds:

 u_lower_bound_from_t = u_lower(t) 
 u_upper_bound_from_t = u_upper(t)

 //collect_all_exclusion_criterions()
// buildQueryWith (Criterion(), get ijt_WHERE_CLAUSE, ….)

// q1 denotes an example query result
q1 :=
SELECT SUM(c.clicks), c.produkt_id, c.haendler_bez,    DB_FUNC_GET_T_FROM_U(c.timestamp, UNIX_ORIGIN) as db_t
 FROM clicks c
  WHERE
   c.produkt_id = i         # get ijt_WHERE_CLAUSE
  AND c.haendler_bez = j 
  AND c.timestamp IS BETWEEN u_lower_bound_from_t AND u_upper_bound_from_t
  AND NOT EXISTS ( SELECT 1 FROM scrapper_ips si WHERE c.user_ip = si.user_ip )     # Append EXLUSION CRITERION 
 GROUP BY  i, j, db_t;

getAsPl(q1)


// variable value realization found
var_result_value = SUM(c.clicks)            //can be any SQL aggr func. (MIN, MAX, AVG, SUM)

// Perform a sanity check
sanity_check():
    db_t == t ?
    -26 <= t <= 832 ? 
   
     -26 <= db_t <= 832 ?

// THEN PART 2 UPDATE, (ALL POSITIONS (ijt) HAVE TO EXIST) FOR EVERY POSITIONAL EXACT MATCH TO BE FOUND…

variable = Variable_name_arg + variant_result

upd :=
UPDATE results AS res
 SET res.variable = variable_value 
 WHERE res.i = i AND res.j = j and res.t = (db_t | t)    //
POSITIONAL EXACT MATCHES




IF NOTHING WORKED :



THEN
PERFORM_DB_ROLLBACK();