*****Preliminaries // Remote debug config...

1. export VERTX_OPTS="-agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=5005"
2. vertx runMod modulename -conf jsonconf -ha -hagroup optional
3. run remote debug in IDE (intelij is preferred multiple tabs concurrently)

Geia sas to kainourio project gia to query processor einai etoimo opws epishs kai ta plugins.

To repo einai sto github kai einai private pros to paron to link einai https://github.com/vagvaz/leads-query-processor.git

Gia version control tha xrhsimopoihsoume to git. Akolouthoun merika tutorials to prwto einai syntomo kai periektiko to 2o gia to github kai to 3o genikes gnwseis xrhsimes!!! git tutorial http://www.vogella.com/tutorials/Git/article.html#gitdefintion_localrepositories github tutorial https://help.github.com/articles/set-up-git
general knowledge https://www.atlassian.com/git/tutorial/git-basics

o kathe enas sas tha exei 1 branch vagvaz -> vagvaz lefteris -> lefteris ioakeim -> ioakeim giannis -> giannis odysseas -> odysseas npavlakis -> npavlakis

thelw na mou steilete ta username sas sto github gia na sas kanw add. Tha douleyete sto proswpiko sas branch kai otan teleiwnete kapoio feature tha kanete pull request

pull request https://help.github.com/articles/creating-a-pull-request

an thelete mporeite na kanete locally osa branch thelete. Tha protimousa oti allages exete na tis pernate sto proswpika sas kai na kanete pull request wste na enswmatwnw tis allages sas.

*** gia na kanete push to branch sas git push -u origin branchname

Epishs sas parakalw issues provlhmata bugs pou den diorthwnete mhn mou ta lete opws pernate apo to grafeio dhmiourghste new issue sto repository.

*****Project

afou kanete clone to project tha deite oti uparxoun diaforoi fakeloi praktika ena gia kathe leitourgikh monada: common: koina xrhsima pragmata core: ta apolutws aparaithta crawler: o crawler pou eftiaksa gia na testarete ta plugins kai genika deployer: o deployer nqe: node query executor parent: voithitiko project gia na mporoume na kanoume build planner: o planner plugin-examples: aplws periexei to template maven project gia ta plugins kai to sentiment and transform plugin gia reference plugins: kwdikas related sta plugins

• Compile gia na kanete compile to project mvn compile

• create jar. prosekste auto to jar einai MONO o kwdikas kai ta resources oxi kai ta dependencies mvn package

• create a super jar, dhmiourgeitai ena jar -with-dependencies me auto mporeite na treksete to kwdika sas mvn assembly:single

• Developping Exw valei sxolia genika stis klasseis pou tha xreiasteite mind

---- in commmon ----- LQPConfiguration (kaleita panta thn initialize wste na exete working cluster ready infinispan) Tuple tuple utility class for handling json objects StringConstants classes in utils folder InfinispanManager interface that abstracts Infinispan operators getCache bla bla InfinispanClusterSingleton.java xrhsimopoihste authn thn klash gia na apokthsete prosvash se 1 InfinispanManager

--- in plugins ---- PluginBaseImpl ena basic plugin pou kanei log oti mpainei kai oti vgainei apo mia getCache PluginInterface to interface pou tha prepei na ulopoioun ta plugins PluginManager h klash pou xrhsimopoieite gia na kanete upload kai deploy plugins

--- plugin-examples --- plugins-template folder pou xrhsimopoieite san base gia na kanete to plugin sas (NA ALLAZETE to artifactId) mesa epishs uparxei kai enas fakelos test me 2 klaseis mia gia testing gia local kai gia clustered. sentiment plugin to gnwsto sentiment and entity extraction plugin kanei compile alla tou leipoun oi classifiers 77Mb ta exw anevasei edw http://goo.gl/4dfs8j ama thelete na to treksete

transform-plugin aplo plugin pou aplws filtrarei kapoia attributes

---- Pws uloipoieitai ena plugin

1. sto leads-query-processor project: mvn clean install etsi kanete compile kai egkathistate sto local maven repository sas ola ta dependencies tou project.
2. copy to fakelo plugin-examples/plugins-template se ena kainourio fakelo px myplugin
3. customize myplugin/pom.xml (ALLAZETE to artifactId, kai prostetete ta maven deps)
4. start happy coding....

ta plugins ta exw gia reference mporeite na deite pws exw ulopoihsei auta ta 3 plugins. gia test exw kanei modify ton crawler pou eixe ftiaksei o Pierre wste na gemizoume mia cache deite pws ton xrhsimmpoiw gia kapoio logo to crawler.seed configuration den douleyei

---- PROSOXH

1. epeidh evala na uparxoun file stores sta caches (shmainei oti swzontai se arxeio) sto /tmp/ tha vriskontai arxeia cacheName.dat ta svhnete...

2. mhn ksegelasteite kanontas iterations panw apo ena cache me to entrySet. Sto deployment version den tha exete ola ta data gia na douleyei swsta...(twra douleyei giati ola ta instances koitazoun to idio arxeio...)

3. to state sas an to allaksete prepei na to ksanavalete sto KVS

4. gia na testarete to clustered deployment prepei prwta na ftiaksete 1 jar me ola ta dependencies mvn assembly:single) kai ena xml configuration arxeio. auta

--UPLOADING (new) -Setup install swift, sudo apt-get install python-swiftclient get the LEADS-openrc.sh from the C&H Dashboard->Access & Security->API Access run it and set the password get the repo from https://github.com/skarab7/leads_query-engine.git (check the branches I used develop) make upload.sh executable

-Execution run "source pathto/tools/openstack_cli/bin/activate" run ./upload.sh

-Extended Setup (if above does not work) check the URL of the cluster tha you are uploading hamm5,6 ! run "source pathto/tools/openstack_cli/bin/activate" pip install -r openstack_cli/requirements.txt install necessary libraries if above fails eg. sudo apt-get install libffi-dev libssl-dev python-dev good luck