Forwarded mails / answers

louismartin commented 7 years ago

Some mails contain forwarded information which is both a curse and a blessing. It is a curse because the bag of words is completely polluted with all the information but a blessing because we have all the email addresses of the previous recipients !!!!!!!!

Example: mid: 51172

On July 16th, SDG&E filed a Motion requesting certain actions from theCommission necessary to implement its Memorandum of Understanding with DWR.There are ten CPUC Implementing Decisions provided for in the MOU. Themotion either requests that the action occur or references another CPUCproceeding where the matter is pending.Please let me know if Enron is interested in pursuing any matters related tothe SDG&E MOU.Jeanne Bennett-----Original Message-----From: Ruiz, Annie [mailto:ARuiz@sempra.com]Sent: Monday, July 16, 2001 5:33 PMTo: bfinkelstein@turn.org; bishop@jrwood.com; Brill, Thomas;bsl@cpuc.ca.gov; chiefskip@aol.com; christine.costa@sce.com; cjw5@pge.com;cwrmccv@worldnet.att.net; DavoodiKR@efaches.navfac.navy.mil;dlf@cpuc.ca.gov; douglass@energyattorney.com.; DVRONROCK@ADPLAW.COM;edf@cpuc.ca.gov; ek@a-klaw.com; epoole@adplaw.com; fmo@sdcity.sannet.gov;freedman@turn.org; furutanj@efawest.navfac.navy.mil; jbennett@gmssr.com;jguzman@nossaman.com; jleslie@luce.com; jmalkin@orrick.com;jsqueri@gmssr.com; jtachera@energy.state.ca.us; jweil@aglet.org;mbrubaker@consultbai.com; mdjoseph@adamsbroadwell.com; Melville, Keith;mjaske@energy.state.ca.us; mshames@ucan.org; msw@cpuc.ca.gov;mv1@cpuc.ca.gov; mzr@cpuc.ca.gov; pbray@newpower.com; raw@cpuc.ca.gov;rax@cpuc.ca.gov; riald@kindermorgan.com;

zaccharieramzi commented 7 years ago

Yes, true that. We could do 2 things:

get rid of everything behind "Original message" => not very smart but we could have a more accurate BoW.
everytime something like that appears get the sender and the recipients (except the sender of course) and put them as recipients. EXCEPT when it is a forward in which case we know it's none of them!!

zaccharieramzi commented 7 years ago

This kind of emails represent half of the dataset: 20860. Problem : We don't have the email addresses but shortcuts for them. Example: mid 881 (extract):

-----Original Message-----From: Stanton, Lon Sent: Thursday, July 12, 2001 12:49 PMTo: Miller, Mary Kay; Nelson, Michel; Neubauer, Dave; McGowan, Mike W.; Fossum, Drew; Talcott, Jim; Porter, Gregory J.; Gilbert, Tom; Poock, Brian; team.waterloo plant@enron.com; Dushinske, John; Brennan, Lorna; Lehan, Tom; Shepherd, Richard; Jensen, BethSubject:

zaccharieramzi commented 7 years ago

Same kind of issue for mails with a forward. We should remove that too ! (and maybe consider using the emails stated to lower their probabilities).

louismartin / email-classification-challenge

Forwarded mails / answers #12