Unable to run the fuzzy periodic frequent pattern (FPFP) algorithm

Tran-Long commented 1 year ago

Hi, thank you for developing such a wonderful open-source library for Pattern Mining. I am using the FPFP algorithm and face some problems:

The data format from the doc (https://udayrage.github.io/PAMI/fuzzyPeriodicFrequentPatternMining.html) does not work Particularly, for example, each row (transaction) from the website only has 1 colon(:) for separating between item and fuzzy value. However, with this format, the algorithm return an error (which inferred as an additional colon (:) is needed) ....
I have also read your paper (*) and fuzzying values from transactional database as written but it seems not right to your implemented algorithm (as I inspect to the code).
I have also visit your website to search for example of fuzzy database (https://u-aizu.ac.jp/~udayrage/datasets.html). However, nothing helps.

Can you please provide me the correct format of the data for this FPFP algorithm as well as explaination for how to create that format with a simple example?

Thanks in advance

(*) Kiran, R. Uday, et al. "Discovering fuzzy periodic-frequent patterns in quantitative temporal databases." 2020 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE). IEEE, 2020.

Likhitha-palla commented 1 year ago

Dear Tran-Long,

The data format currently used for fuzzy algorithms can be find in utility databases section in the following URL.
We will update and standardize the fuzzy database format and algorithms soon. Please, upgrade PAMI later and follow the updated manuals.
In the meanwhile, you can download the Utility databases from above URL. And follow the manual to run the FPFP-Miner algorithm

Tran-Long commented 1 year ago

Dear Likhitha-palla, Thank you for your reply. I have found and downloaded the file in the utility databases. I realize that there are 2 colons (:). Can you tell me what the number between those 2 colons is, please? Moreover, I am understanding that fuzzy values are values between (0,1). Why those values are integers? Please let me know

Thanks in advance

Likhitha-palla commented 1 year ago

Dear Trans-Long,

We ignore the part ":value:" in transaction. And consider the left side as items and right side as the quantitative values.
And convert the quantitative values in to fuzzy values using already defined fuzzy membership function.

Tran-Long commented 1 year ago

Dear Likhitha-palla, thanks for your reply. I am understanding that fuzzy values are between (0,1), which are created from quantitative values with specific thresholds (fuzzy membership funciton) However, the algorithm return error with float values and the file that you guide me to run with the FPFP contains integer instead of float. I'm just wondering how you transform those calculated fuzzy values (between (0,1)) into integer values ? Thank you

Likhitha-palla commented 1 year ago

Dear Trans-Long,

Please try with sample values like [0.008, 0.01, 0.012]. If you give parameters (minSup & periodicity) in float values, These are multiplied with total number of transaction in database.
Please follow the membership function and the mathematic in the paper.

Kiran, R. Uday, et al. "Discovering fuzzy periodic-frequent patterns in quantitative temporal databases." 2020 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE). IEEE, 2020.

Tran-Long commented 1 year ago

Dear Likhitha-palla, Sorry but maybe you are misunderstanding my point that, Firstly, I tried with correct values type:

Fuzzy value: FLOAT, such as 0.1, 0.0008, .....
minSup&periodicity: INT, such as 2,3,1000,....

Secondly, I did use the code provided in : basic-sample with the data sample copied from part 3 in: manual (which I added another colon for match the format) and the RUNTIME ERROR happen in line 404 of this FPFP code: source-code

The ERROR received is inferred as: ("ValueError: invalid literal for int() with base 10: '0.2'"), which means the value of fuzzy value must be integer instead of float in order to successfully run the FPFP algorithm.

Besides, what I mentioned in previous comments is that the data you provided in this URL has fuzzy values are all in integer format, not float as calculated with membership function in the paper.

So, I am wondering that why these fuzzy values are integer and how to transform those from float into integer ?

Thank you

udayRage commented 1 year ago

hai,

can you share your notebook with us?

it will help us in knowing whether ur getting problem.

reagrds prof. uday

On Fri, Feb 17, 2023 at 13:07 Parzival @.***> wrote:

Dear Likhitha-palla, Sorry but maybe you are misunderstanding my point that, Firstly, I tried with correct values type:

Fuzzy value: FLOAT, such as 0.1, 0.0008, .....

minSup&periodicity: INT, such as 2,3,1000,....

Secondly, I did use the code provided in : basic-sample https://github.com/udayRage/PAMI/blob/main/sampleManuals/fuzzyPeriodicFrequentPatterns/basic/FPFPMiner/FPFPMiner-st.md with the data sample copied from part 3 in: manual https://udayrage.github.io/PAMI/fuzzyPeriodicFrequentPatternMining.html (which I added another colon for match the format) and the RUNTIME ERROR happen in line 404 of this FPFP code: source-code https://github.com/udayRage/PAMI/blob/main/PAMI/fuzzyPeriodicFrequentPattern/basic/FPFPMiner.py

The ERROR received is inferred as: ("ValueError: invalid literal for int() with base 10: '0.2'"), which means the value of fuzzy value must be integer instead of float in order to successfully run the FPFP algorithm.

Besides, what I mentioned in previous comments is that the data you provided in this URL https://u-aizu.ac.jp/~udayrage/datasets.html has fuzzy values are all in integer format, not float as calculated with membership function in the paper.

So, I am wondering that why these fuzzy values are integer and how to transform those from float into integer ?

Thank you

— Reply to this email directly, view it on GitHub https://github.com/udayRage/PAMI/issues/22#issuecomment-1434068794, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMN3BXDN2HK54MKH5KWHFELWX32QRANCNFSM6AAAAAAU52S7IM . You are receiving this because you are subscribed to this thread.Message ID: @.***>

-- Regards

Prof. Rage Uday Kiran Associate Professor Database Systems Lab Division of Information Systems University of Aizu, Aizu-Wakamatsu, Japan.

Lab number: 145-A

Mobile no. +81 80 4834 0999 Phone no. (Lab) +81 242 37 2541 (Ext) 3115

When people throw stones at you, convert them into milestones.

udayRage commented 1 year ago

hai,

So, I am wondering that why these fuzzy values are integer and how to transform those from float into integer ?

ans) fuzzy pattern mining algorithms take quantitaive database, convert them into fuzzy databases, and then mine the patterns.

     the input dataset, i.e., utility datase, is a quantitaive dataset.

thus, it has values.

     the pattern mining alogirthm needs this quantitaive/utility

datset, internally transforms it into a fuzzy database, keeps it in the memory, mines the datase and generates patterns.

        so u cannot see the fuzzy database as it exists in the memory.

reagrds prof. uday

On Fri, Feb 17, 2023 at 13:07 Parzival @.***> wrote:

Dear Likhitha-palla, Sorry but maybe you are misunderstanding my point that, Firstly, I tried with correct values type:

Fuzzy value: FLOAT, such as 0.1, 0.0008, .....

minSup&periodicity: INT, such as 2,3,1000,....

Secondly, I did use the code provided in : basic-sample https://github.com/udayRage/PAMI/blob/main/sampleManuals/fuzzyPeriodicFrequentPatterns/basic/FPFPMiner/FPFPMiner-st.md with the data sample copied from part 3 in: manual https://udayrage.github.io/PAMI/fuzzyPeriodicFrequentPatternMining.html (which I added another colon for match the format) and the RUNTIME ERROR happen in line 404 of this FPFP code: source-code https://github.com/udayRage/PAMI/blob/main/PAMI/fuzzyPeriodicFrequentPattern/basic/FPFPMiner.py

The ERROR received is inferred as: ("ValueError: invalid literal for int() with base 10: '0.2'"), which means the value of fuzzy value must be integer instead of float in order to successfully run the FPFP algorithm.

Besides, what I mentioned in previous comments is that the data you provided in this URL https://u-aizu.ac.jp/~udayrage/datasets.html has fuzzy values are all in integer format, not float as calculated with membership function in the paper.

So, I am wondering that why these fuzzy values are integer and how to transform those from float into integer ?

Thank you

— Reply to this email directly, view it on GitHub https://github.com/udayRage/PAMI/issues/22#issuecomment-1434068794, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMN3BXDN2HK54MKH5KWHFELWX32QRANCNFSM6AAAAAAU52S7IM . You are receiving this because you are subscribed to this thread.Message ID: @.***>

-- Regards

Prof. Rage Uday Kiran Associate Professor Database Systems Lab Division of Information Systems University of Aizu, Aizu-Wakamatsu, Japan.

Lab number: 145-A

Mobile no. +81 80 4834 0999 Phone no. (Lab) +81 242 37 2541 (Ext) 3115

When people throw stones at you, convert them into milestones.

Tran-Long commented 1 year ago

Dear Prof, Uday,

Thank you for your replies. Here is my example notebook. So I am understanding your point that I must input a quantitative database into the FPFP algorithm to work, which is different from the manual ?

Thank you

udayRage commented 1 year ago

Likhitha, plz check the notebook and answer.

On Fri, Feb 17, 2023 at 13:30 Parzival @.***> wrote:

Dear Prof, Uday,

Thank you for your replies. Here is my example notebook https://colab.research.google.com/drive/1JsMSsUPmU_5ggUZHTBoo9C0YMw26l_l2?usp=share_link . So I am understanding your point that I must input a quantitative database into the FPFP algorithm to work, which is different the manual https://udayrage.github.io/PAMI/fuzzyPeriodicFrequentPatternMining.html ?

Thank you

— Reply to this email directly, view it on GitHub https://github.com/udayRage/PAMI/issues/22#issuecomment-1434088715, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMN3BXDMJFH3IREFPPVHV6LWX35EXANCNFSM6AAAAAAU52S7IM . You are receiving this because you commented.Message ID: @.***>

-- Regards

Prof. Rage Uday Kiran Associate Professor Database Systems Lab Division of Information Systems University of Aizu, Aizu-Wakamatsu, Japan.

Lab number: 145-A

Mobile no. +81 80 4834 0999 Phone no. (Lab) +81 242 37 2541 (Ext) 3115

When people throw stones at you, convert them into milestones.

Likhitha-palla commented 1 year ago

Dear Trans-Long,

Made changes in the notebook and hope it will clear your doubts. Please refer to this notebook

Tran-Long commented 1 year ago

Dear Prof. Rage Uday Kiran and Likhitha-palla, Thanks for your replies, I have read the changes in the notebook you sent as well as digging into your source code

I realize that your code is fixed with the fuzzy membership function with 3 regions of 1,6,11, respectively. Please confirm this for me. However, this is not my desire fuzzy membership function because I am trying to mine different features with different membership functions (i.e different regions and their corresponding thresholds). So for that reason, can you extract and send me the new code that only process the mining, which is not including the fuzzying value procedure, please? (i,e the new FPFP algorithm that works with the example data I've shown in the notebook)

Thank you

udayRage commented 1 year ago

hai

3 regions of 1,6,11, ans) yes

this is the old code used in the paper.

we are modifying the code so that it can take directly fuzzy database and extract patterns.

as Likhitha informed earlier, plz wait until this month release. we will update the code.

or, since you already read the code correctly, you can modify it and use it as it sees fit.

On Fri, Feb 17, 2023 at 16:03 Parzival @.***> wrote:

Dear Prof. Rage Uday Kiran and Likhitha-palla, Thanks for your replies, I have read the changes in the notebook you sent as well as digging into your source code https://github.com/udayRage/PAMI/blob/main/PAMI/fuzzyPeriodicFrequentPattern/basic/FPFPMiner.py#L109

I realize that your code is fixed with the fuzzy membership function with 3 regions of 1,6,11, respectively. Please confirm this for me. However,this is not my desire fuzzy membership function because I am trying to mine different features with different membership functions (i.e different regions and their corresponding thresholds). So for that reason, can you extract and send me the new code that only process the mining, which is not including the fuzzying value procedure, please? (i,e the new FPFP algorithm that works with the example data I've shown in the notebook https://colab.research.google.com/drive/1JsMSsUPmU_5ggUZHTBoo9C0YMw26l_l2?usp=share_link )

Thank you

— Reply to this email directly, view it on GitHub https://github.com/udayRage/PAMI/issues/22#issuecomment-1434185317, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMN3BXF4OFGHJLX5DZPR6HTWX4PF5ANCNFSM6AAAAAAU52S7IM . You are receiving this because you commented.Message ID: @.***>

-- Regards

Prof. Rage Uday Kiran Associate Professor Database Systems Lab Division of Information Systems University of Aizu, Aizu-Wakamatsu, Japan.

Lab number: 145-A

Mobile no. +81 80 4834 0999 Phone no. (Lab) +81 242 37 2541 (Ext) 3115

When people throw stones at you, convert them into milestones.

Tran-Long commented 1 year ago

Dear Prof. Rage Uday Kiran and Likhitha-palla,, thanks for your replies. Yes I'll wait and hope you guys can upload the new code as soon as possible. Thank you very much!

udayRage commented 1 year ago

Dear Tran-Long,

Good day!

           Thank you very much for your patience. 

           We have updated the FPFP algorithm.

           Also, fuzzy datasets needed for the experimentation are available https://u-aizu.ac.jp/~udayrage/datasets.html

          If you have any comments/feedback, please feel free to let us know.

Regards Uday

Tran-Long commented 1 year ago

Dear Prof. Rage Uday Kiran, Thank you for your replies. I will check it out later

UdayLab / PAMI

Unable to run the fuzzy periodic frequent pattern (FPFP) algorithm #22