feedzai / timeshap

TimeSHAP explains Recurrent Neural Network predictions.
Other
162 stars 30 forks source link

Issue in reproducing TimeSHAP Tutorial - TensorFlow - AReM dataset #9

Closed grzechowiak closed 2 years ago

grzechowiak commented 2 years ago

Foremost, thanks for the library, really great job!

I am trying to reproduce your TimeSHAP Tutorial for TF and I am having an issue in the section for Global Explanations when running the global_report() function - screen below:

image

The error refers to encoding the \u2264 character which is a sign: <=. I was trying to solve that myself by modifying the pruning.py function according to the error by adding encoding="utf-8" in line 326 with open(file_path, 'a', newline='') as file:, however it didn't solve the problem. Any advice is very welcome!

Also, for consistency, I want to mention that I had a problem loading the data - showing the error screens below. I was able to solve the problem only by deleting 2 datasets: cycling/dataset9.csv and cycling/dataset14csv and the rest of the code worked.

1/2 image 2/2 image

JoaoPBSousa commented 2 years ago

Hello @grzechowiak ,

Regarding the first issue, I suspect this is due to the OS you are using. On the persisted pruning data we write the character \u2264. A quick fix for your issue is to replace this character with another string/char. Replace symbols in lines:

About the dataset loading issue, this is known (documented on README ). This is due to the data itself being malformed on the UCI repo. We uploaded a fixed version of the datasets to the repo.

Please let me know if this fixes worked, and if you have any further question don't hesitate to contact us.

grzechowiak commented 2 years ago

Hello @JoaoPBSousa ,

Thank you for your quick answer! Your solution helped with the string, however, I have got another error (the same line), this time regarding the 'Shapley Value'. Any ideas why the error pop up? Please see the screen below.

image

JoaoPBSousa commented 2 years ago

Hello @grzechowiak,

This issue is due to the event data event_data passed to the plot_global_event method not having the Shapley Value key. We can run the code with no issues, so I cannot help you without further information. Make sure that the event_data variable has all the necessary keys, "Shapley Value" included. Is it possible that when you edited the code to fix the string issue, you changed something on the method?

If the issue persists, could you please provide further information so I can help you?

grzechowiak commented 2 years ago

Hello @JoaoPBSousa ,

Thanks for the answer! Finally, I was able to execute the function. I am not exactly sure what was causing the error, but I reinstalled the TimeSHAP and also I deleted the outputs in the output folder. Once the function had to recreate and save into the outputs folder again, the function worked. Anyway, thanks for the help!