Tambahan feature extraction? - Githubissues

luthfianto / dmc-2016

Team: Uni Gadjah Mada 1. Our attempts and solutions for prudsys' Data Mining Cup 2016

1 stars 1 forks source link

Tambahan feature extraction? #15

Closed luthfianto closed 8 years ago

luthfianto commented 8 years ago

Sumber: https://github.com/xydrolase/dmc-2014/blob/master/featgen%2Ffeat_gen.R

Catatan:

cid = customer id
iid = item id

List:

[ ] mindisc.by.iid = min(disc , na.rm = T) ,
[ ] meandisc.by.iid = mean(disc , na.rm = T)
[ ] minprice.by.iid = min(price) ,
[ ] maxprice.by.iid = max(price) ,
[x] meanprice.by.iid = mean(price)). kita: usual_unit_price
[ ] keverage #considering the local(the nearest 15 order) disc/price trend for items smooth with neighbor to be 15
[ ] localdisc
[ ] localprice
[x] pricediff. kita: price_diff
[ ] discdiff
[ ] outday.by.iid
[ ] deal
[ ] nlowprice.by.cid = sum(price < 100) ,
[ ] nlowdisc.by.cid = sum(disc < 0.8) ,
[ ] ndeal.by.cid = sum(deal) ,
[x] norder.by.cid = n(). kita: order_order
[ ] nreturn.by.cid = sum(return , na.rm = T) ,
[ ] totalspend.by.cid = sum(price) ,
[x] meanspend.by.cid = mean(price). kita: customer_budget
[ ] nonreturnspend.by.cid = sum(price * (return==0) , na.rm = T) ,
[ ] returnspend.by.cid = sum(price * (return == 1) , na.rm = T))

some by.batch.cid feature(xin have done this)

[ ] mbspend = mean(price) , bsize = n()) %.%
[ ] nbc = length(unique(date)) , noc = n() , bspendc = sum(price))
[ ] #outseason.by.iid
[ ] check_order_before
[ ] check_keep_future
[ ] check_return_future
[ ] check_order_future
[ ] #ob.by.cid.iid.price
[ ] #of.by.cid.iid.price
[ ] rb.by.cid.iid.price = check_return_before(date,return),
[ ] kb.by.cid.iid.price = check_keep_before(date,return),
[ ] rf.by.cid.iid.price = check_return_future(date,return),
[ ] kf.by.cid.iid.price = check_keep_future(date,return)) %.%

If a cid returned/kept/ordered an exactly same item before/in the future

[ ] #ob.by.cid.iid.color.size
[ ] #of.by.cid.iid.color.size
[ ] rb.by.cid.iid.color.size = check_return_before(date,return),
[ ] kb.by.cid.iid.color.size = check_keep_before(date,return),
[ ] rf.by.cid.iid.color.size = check_return_future(date,return),
[ ] kf.by.cid.iid.color.size = check_keep_future(date,return)) %.%

If a cid returned/kept/ordered a same iid before/in the future

[ ] rb.by.cid.iid = check_return_before(date,return),
[ ] ob.by.cid.iid = check_order_before(date,return),
[ ] kb.by.cid.iid = check_keep_before(date,return),
[ ] rf.by.cid.iid = check_return_future(date,return),
[ ] of.by.cid.iid = check_order_future(date,return),
[ ] kf.by.cid.iid = check_keep_future(date,return)) %.%

If a cid returned/kept/ordered a same item with same price before/in the future

[ ] #ob.by.cid.price
[ ] #of.by.cid.price
[ ] rb.by.cid.price = check_return_before(date,return),
[ ] kb.by.cid.price = check_keep_before(date,return),
[ ] rf.by.cid.price = check_return_future(date,return),
[ ] kf.by.cid.price = check_keep_future(date,return)
[ ] rankprice.by.cid.iid = rank(price)) %.% #the price rank of price of the iid ordered by a cid
[ ] ntotal = nrow(train)
[ ] #llr.by.price
[ ] ##return+1/#keep+1
[ ] llr.by.price = log((sum(return , na.rm = T)+0.5) / (0.5+length(return)-sum(return,na.rm=T)))) %.%
[ ] k = sum(return , rm.na = T)))$k
[ ] n = n()))$n
[ ] llr.by.cid.price = log((sum(return , na.rm = T)+0.5) / (0.5+length(return)-sum(return,na.rm=T)))) %.%
[ ] rm(train)

item freshness

[ ] raw.tr$f1w <- fan.feats$outday.by.iid <= 7
[ ] raw.tr$f2w <- fan.feats$outday.by.iid <= 14
[ ] raw.tr$f1m <- fan.feats$outday.by.iid <= 30
[ ] raw.tr$f3m <- fan.feats$outday.by.iid <= 90
[ ] raw.tr$f6m <- fan.feats$outday.by.iid <= 180
[ ] raw.tr$oseas <- fan.feats$outseason.by.iid
[ ] raw.tr$isdisc <- fan.feats$disc < 1
[ ] raw.tr$deal <- fan.feats$deal
[ ] raw.tr$lowdisc <- fan.feats$disc <= 0.8

price ranges

[ ] raw.tr$pb25 <- raw.tr$price < 25
[ ] raw.tr$pb50 <- raw.tr$price < 50
[ ] raw.tr$pb100 <- raw.tr$price < 100
[ ] raw.tr$pb200 <- raw.tr$price < 200

compute counts and LLRs for given "feats", the combation of features.

[ ] counts.and.llrs <- function(df, feats, c1=0.5, c2=0.5) {

3way interaction: color_state_iid

[ ] .feats = counts.and.llrs(raw.tr, c("state", "iid", "color"))
[ ] names(.feats) <- c("all.cnt.state_iid_color", "all.llr.state_iid_color")
[ ] all.feats <- cbind(all.feats, .feats)
[ ] .feats = counts.and.llrs(raw.tr, c("state", "mid", "color"))
[ ] names(.feats) <- c("all.cnt.state_mid_color", "all.llr.state_mid_color")
[ ] all.feats <- cbind(all.feats, .feats)

ratio of low price / low discount

[ ] fan.feats$rlowprice.by.cid <- fan.feats$nlowprice.by.cid /
[ ] all.feats$all.cnt.cid
[ ] fan.feats$rlowprice.by.cid[all.feats$all.cnt.cid == 0] <- 0
[ ] fan.feats$rlowdisc.by.cid <- fan.feats$nlowdisc.by.cid /
[ ] all.feats$all.cnt.cid
[ ] fan.feats$rlowdisc.by.cid[all.feats$all.cnt.cid == 0] <- 0

batch features with some selected interactions

[ ] bat.n=length(oid),
[ ] bat.uniq.iid=length(unique(iid)),
[ ] bat.uniq.mid=length(unique(mid)),
[ ] bat.uniq.size=length(unique(size)),
[ ] bat.uniq.color=length(unique(color)),
[ ] bat.uniq.ztype=length(unique(ztype)),
[ ] bat.uniq.zsize=length(unique(zsize)),
[ ] bat.uniq.mid_zsize=nrow(unique(cbind(mid, zsize))),
[ ] bat.uniq.ztype_zsize=nrow(unique(cbind(ztype, zsize))),
[ ] bat.uniq.iid_color=nrow(unique(cbind(iid, color))),
[ ] bat.uniq.mid_color=nrow(unique(cbind(mid, color)))

batch counts / other counts / max counts

customer per batch features

[ ] rrate=sum(return)/length(return),
[ ] krate=1-sum(return)/length(return)) %.%

only set the first order of each batch to be the true rate, others set to be NA

[ ] srrate=c(sum(return, na.rm=T)/length(return), rep(NA, length(return)-1)),
[ ] skrate=c(1-sum(return, na.rm=T)/length(return), rep(NA, length(return)-1))) %.%
[ ] cb.ret.rates$srrate <- cb.ret.srates$srrate
[ ] cb.ret.rates$skrate <- cb.ret.srates$skrate

average return/keep rate, weighted and unweighted,

[ ] cbat.wavg.rrate=mean(rrate, na.rm=T),
[ ] cbat.wavg.krate=mean(krate, na.rm=T),

simple averages

[ ] cbat.avg.rrate=mean(srrate, na.rm=T),
[ ] cbat.avg.krate=mean(skrate, na.rm=T),
[ ] cbat.sum.rrate=sum(srrate, na.rm=T),
[ ] cbat.sum.krate=sum(skrate, na.rm=T))
[ ] # log-likelihood ratio of return over kept

luthfianto commented 8 years ago

Closed for awhile due to another high priority issue.