LIME time consumption

[jithincheriyan]

Hi,

I am trying to use LIME to interpret my BERT model. The model prediction is good, and the prediction does not take much time to return the prediction probabilities. However, the LIME explainer takes nearly 10 minutes to make the explanation for a single comment. Is there anything wrong from my side or does LIME take that much time for interpretation?

Please find my code below:

def Lime_BERT_explanation(comment):

path = "H:\test"

# test_df = pd.read_csv(path + ".csv", encoding="ISO-8859-1")
# Comment_Text = test_df.Comment_Text.tolist()
# print(len(Comment_Text))
# for comment in Comment_Text:
print(comment)
bert_exp = explainer.explain_instance(str(comment),
                                      BERT_predict,
                                      num_features=6,
                                      labels=label_names
                                      )
print(bert_exp.as_list())

if name == 'main':

Lime_ML_explanation()

comment="Hello idiot"
# add this to test a single comment
Comment_Text = []
Comment_Text.append(comment)          # where Comment is the comment to be tested
print(BERT_predict(Comment_Text))
Lime_BERT_explanation(Comment_Text)

Here BERT_predict is the BERT module which returns the prediction probalbilities.

Thank you

[marcotcr]

The default value for num_samples (i.e. how many perturbed samples you get, and how many times BERT gets called) in explain_instance is 5000. I'm guessing your model takes 10 minutes to make that many predictions. If you're dealing with short texts (e.g. sentences), you can just use a smaller number of samples

[jithincheriyan]

Hi,

Thank you for the reply. But when I set num_samples to a lower number, say 100, it ends up with a new error: " labels_column = neighborhood_labels[:, label] TypeError: list indices must be integers or slices, not tuple".

What can we do to make it work?

Regards, Jithin

From: Marco Tulio Correia Ribeiro @.> Sent: Wednesday, 11 August 2021 5:40 AM To: marcotcr/lime @.> Cc: Jithin Cheriyan @.>; Author @.> Subject: Re: [marcotcr/lime] LIME time consumption (#632)

The default value for num_samples (i.e. how many perturbed samples you get, and how many times BERT gets called) in explain_instance is 5000. I'm guessing your model takes 10 minutes to make that many predictions. If you're dealing with short texts (e.g. sentences), you can just use a smaller number of samples

- You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmarcotcr%2Flime%2Fissues%2F632%23issuecomment-896183362&data=04%7C01%7Cjithin.cheriyan%40postgrad.otago.ac.nz%7Cf0e12b32d4374db880e508d95c25d191%7C0225efc578fe4928b1579ef24809e9ba%7C1%7C0%7C637642139784455160%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=crpErvvTK2uYxYIYx8LM5ALZ%2FpFMhbyYaos6MlOdNS4%3D&reserved=0, or unsubscribehttps://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAMOFAVNIRN34IKB75E6PJGTT4FP5HANCNFSM5BQCKZMA&data=04%7C01%7Cjithin.cheriyan%40postgrad.otago.ac.nz%7Cf0e12b32d4374db880e508d95c25d191%7C0225efc578fe4928b1579ef24809e9ba%7C1%7C0%7C637642139784455160%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=8oyaMb5wGbBfrTbaARyP0aOYkZH%2F9cLdFz39fckiMLo%3D&reserved=0. Triage notifications on the go with GitHub Mobile for iOShttps://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapps.apple.com%2Fapp%2Fapple-store%2Fid1477376905%3Fct%3Dnotification-email%26mt%3D8%26pt%3D524675&data=04%7C01%7Cjithin.cheriyan%40postgrad.otago.ac.nz%7Cf0e12b32d4374db880e508d95c25d191%7C0225efc578fe4928b1579ef24809e9ba%7C1%7C0%7C637642139784465152%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=%2Fwde1rN7URhwNkdDURHhNLwh9Ow2m34mcXnhFwVVBko%3D&reserved=0 or Androidhttps://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fplay.google.com%2Fstore%2Fapps%2Fdetails%3Fid%3Dcom.github.android%26utm_campaign%3Dnotification-email&data=04%7C01%7Cjithin.cheriyan%40postgrad.otago.ac.nz%7Cf0e12b32d4374db880e508d95c25d191%7C0225efc578fe4928b1579ef24809e9ba%7C1%7C0%7C637642139784465152%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=vN%2BOVQ0x14gPEl0GiYWfRwD%2FUsCS3qOj847Gwxm6Ug8%3D&reserved=0.