f4bD3v / humanitas

A price prediction toolset for developing countries
BSD 3-Clause "New" or "Revised" License
17 stars 7 forks source link

Data Mining: Regional Scope of Analysis #16

Closed albu89 closed 10 years ago

albu89 commented 10 years ago

Good morning team, quick question concerning the regional scope of the analysis. Basically I'd be able to create interferences between regions, however depending on how we present our system it might not make sense. Are we going to present information to specific regions, i.e. will the user click on a part of India and receive information about commodities to that spcecific area? In that case I would exclude interferences between regions to make it coherent with the rest of the system.

Happy easter!

Alex

f4bD3v commented 10 years ago

Yes, definitely!

albu89 commented 10 years ago

Association Rules is finished for weekly data: Sample output of the system for Delhi where big, small indicates the price increase:

supp conf rule 0.826 0.963 MeatMutton=small -> MaizeFAQ=big 0.826 0.915 MaizeFAQ=big -> MeatMutton=small 0.822 0.963 AttaFAQ=small -> MaizeFAQ=big 0.822 0.944 BajraFAQ=big -> MaizeFAQ=big 0.822 0.911 MaizeFAQ=big -> BajraFAQ=big 0.822 0.911 MaizeFAQ=big -> AttaFAQ=small 0.818 0.915 RiceCommonCoarse=small -> MaizeFAQ=big 0.818 0.906 MaizeFAQ=big -> RiceCommonCoarse=small 0.792 0.910 BajraFAQ=big -> RiceCommonCoarse=small 0.792 0.886 RiceCommonCoarse=small -> BajraFAQ=big 0.786 0.921 AttaFAQ=small -> MeatMutton=small 0.786 0.916 MeatMutton=small -> AttaFAQ=small 0.778 0.943 CoconutNA=small -> MaizeFAQ=big 0.778 0.862 MaizeFAQ=big -> CoconutNA=small 0.775 0.904 MeatMutton=small -> BajraFAQ=big 0.775 0.891 BajraFAQ=big -> MeatMutton=small 0.773 0.946 ArharSplit=small -> MaizeFAQ=big 0.773 0.906 AttaFAQ=small -> BajraFAQ=big 0.773 0.901 MeatMutton=small -> RiceCommonCoarse=small 0.773 0.888 BajraFAQ=big -> AttaFAQ=small 0.773 0.865 RiceCommonCoarse=small -> MeatMutton=small 0.773 0.857 MaizeFAQ=big -> ArharSplit=small 0.771 0.960 RedNA=small -> MaizeFAQ=big 0.771 0.903 AttaFAQ=small -> RiceCommonCoarse=small 0.771 0.863 RiceCommonCoarse=small -> AttaFAQ=small 0.771 0.854 MaizeFAQ=big -> RedNA=small 0.761 0.981 BajraFAQ=big MeatMutton=small -> MaizeFAQ=big 0.761 0.968 AttaFAQ=small MeatMutton=small -> MaizeFAQ=big 0.761 0.925 AttaFAQ=small MaizeFAQ=big -> MeatMutton=small 0.761 0.925 BajraFAQ=big MaizeFAQ=big -> MeatMutton=small 0.761 0.921 MaizeFAQ=big MeatMutton=small -> BajraFAQ=big 0.761 0.921 MaizeFAQ=big MeatMutton=small -> AttaFAQ=small 0.761 0.891 AttaFAQ=small -> MaizeFAQ=big MeatMutton=small 0.761 0.886 MeatMutton=small -> AttaFAQ=small MaizeFAQ=big 0.761 0.886 MeatMutton=small -> BajraFAQ=big MaizeFAQ=big 0.761 0.873 BajraFAQ=big -> MaizeFAQ=big MeatMutton=small 0.761 0.843 MaizeFAQ=big -> BajraFAQ=big MeatMutton=small 0.761 0.843 MaizeFAQ=big -> AttaFAQ=small MeatMutton=small 0.758 0.955 GramSplit=small -> MaizeFAQ=big 0.758 0.840 MaizeFAQ=big -> GramSplit=small 0.756 0.978 AttaFAQ=small BajraFAQ=big -> MaizeFAQ=big 0.756 0.942 RedNA=small -> SaltPacketiodized=small 0.756 0.937 SaltPacketiodized=small -> RedNA=small 0.756 0.937 SaltPacketiodized=small -> MaizeFAQ=big 0.756 0.920 BajraFAQ=big MaizeFAQ=big -> AttaFAQ=small 0.756 0.920 AttaFAQ=small MaizeFAQ=big -> BajraFAQ=big 0.756 0.886 AttaFAQ=small -> BajraFAQ=big MaizeFAQ=big 0.756 0.869 BajraFAQ=big -> AttaFAQ=small MaizeFAQ=big 0.756 0.838 MaizeFAQ=big -> AttaFAQ=small BajraFAQ=big 0.756 0.838 MaizeFAQ=big -> SaltPacketiodized=small 0.754 0.975 MeatMutton=small RiceCommonCoarse=small -> MaizeFAQ=big 0.754 0.922 MaizeFAQ=big RiceCommonCoarse=small -> MeatMutton=small 0.754 0.913 MaizeFAQ=big MeatMutton=small -> RiceCommonCoarse=small 0.754 0.879 MeatMutton=small -> MaizeFAQ=big RiceCommonCoarse=small 0.754 0.844 RiceCommonCoarse=small -> MaizeFAQ=big MeatMutton=small 0.754 0.836 MaizeFAQ=big -> MeatMutton=small RiceCommonCoarse=small 0.752 0.949 BajraFAQ=big RiceCommonCoarse=small -> MaizeFAQ=big 0.752 0.920 MaizeFAQ=big RiceCommonCoarse=small -> BajraFAQ=big 0.752 0.915 BajraFAQ=big MaizeFAQ=big -> RiceCommonCoarse=small 0.752 0.913 CoconutNA=small -> MeatMutton=small 0.752 0.877 MeatMutton=small -> CoconutNA=small 0.752 0.864 BajraFAQ=big -> MaizeFAQ=big RiceCommonCoarse=small 0.752 0.841 RiceCommonCoarse=small -> BajraFAQ=big MaizeFAQ=big 0.752 0.833 MaizeFAQ=big -> BajraFAQ=big RiceCommonCoarse=small 0.750 0.934 RedNA=small -> MeatMutton=small 0.750 0.917 ArharSplit=small -> MeatMutton=small 0.750 0.910 CoconutNA=small -> RiceCommonCoarse=small 0.750 0.910 CoconutNA=small -> AttaFAQ=small 0.750 0.878 AttaFAQ=small -> CoconutNA=small 0.750 0.874 MeatMutton=small -> RedNA=small 0.750 0.874 MeatMutton=small -> ArharSplit=small 0.750 0.839 RiceCommonCoarse=small -> CoconutNA=small

albu89 commented 10 years ago

@duynguyen with regards to the interface the association rules are specific to cities. i can include a finer granularity with respect to year and commodity if you wish. let me know about your requirements with regards to the interfaces so i can include all queries.

f4bD3v commented 10 years ago

@albu89, please post an update of the regional scope of analysis on this issue

albu89 commented 10 years ago

@fabbrix as mentioned i had problems extracting useful rules since the overwhelming majority of the data didn't show an increase or decrease in price over the time stamp of one week. i therefor decided to aggregate the price difference over a time period of 5 weeks which now gives me a much clearer picture on the correlation of the commodities. i'm running a script atm to output the top 10 rules of all cities. i'll upload it here shortly :-)

albu89 commented 10 years ago

@duynguyen here you go https://drive.google.com/folderview?id=0BzfXFPhJt9ILSjN2bGs2OEJjODg&usp=sharing

albu89 commented 10 years ago

@fabbrix uploaded documentation