occasional screwup in send_activity without clear error messaging

[master0v]

Github issues should be used for bugs and feature requests. Use Stack Overflow for general "how-to" questions.

Version

What package version of the SDK are you using. 4.14.1

Describe the bug

Give a clear and concise description of what the bug is.

This is the traceback, there is no proper error message. So for starters the bug is the lack of clear exception messaging.

File "/usr/local/lib/python3.7/dist-packages/botbuilder/core/turn_context.py", line 174, in send_activity result = await self.send_activities([activity_or_text]) File "/usr/local/lib/python3.7/dist-packages/botbuilder/core/turn_context.py", line 226, in send_activities return await self._emit(self._on_send_activities, output, logic()) File "/usr/local/lib/python3.7/dist-packages/botbuilder/core/turn_context.py", line 304, in _emit return await logic File "/usr/local/lib/python3.7/dist-packages/botbuilder/core/turn_context.py", line 221, in logic responses = await self.adapter.send_activities(self, output) File "/usr/local/lib/python3.7/dist-packages/botbuilder/core/bot_framework_adapter.py", line 733, in send_activities
raise error File "/usr/local/lib/python3.7/dist-packages/botbuilder/core/bot_framework_adapter.py", line 719, in send_activities
activity.conversation.id, activity.reply_to_id, activity File "/usr/local/lib/python3.7/dist-packages/botframework/connector/aio/operations_async/_conversations_operations_async.py", line 525, in reply_to_activity request, stream=False, operation_config File "/usr/local/lib/python3.7/dist-packages/msrest/async_client.py", line 115, in async_send pipeline_response = await self.config.pipeline.run(request, kwargs) File "/usr/local/lib/python3.7/dist-packages/msrest/pipeline/async_abc.py", line 159, in run return await first_node.send(pipeline_request, kwargs) # type: ignore File "/usr/local/lib/python3.7/dist-packages/msrest/pipeline/async_abc.py", line 79, in send response = await self.next.send(request, kwargs) # type: ignore File "/usr/local/lib/python3.7/dist-packages/msrest/pipeline/async_requests.py", line 106, in send return await self.next.send(request, kwargs) File "/usr/local/lib/python3.7/dist-packages/msrest/pipeline/async_abc.py", line 79, in send response = await self.next.send(request, kwargs) # type: ignore File "/usr/local/lib/python3.7/dist-packages/msrest/pipeline/async_abc.py", line 79, in send response = await self.next.send(request, kwargs) # type: ignore File "/usr/local/lib/python3.7/dist-packages/msrest/pipeline/async_requests.py", line 85, in send await self.driver.send(request.http_request, kwargs) File "/usr/local/lib/python3.7/dist-packages/msrest/universal_http/async_requests.py", line 91, in send return await super(AsyncRequestsHTTPSender, self).send(request, **requests_kwargs) File "/usr/local/lib/python3.7/dist-packages/msrest/universal_http/async_requests.py", line 79, in send await future concurrent.futures._base.CancelledError

To Reproduce

Steps to reproduce the behavior:

1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

this is not reproducible consistently, it happens sporadically.

Expected behavior

Give a clear and concise description of what you expected to happen.

bot sends a message or at least gives a clear exception for why it can not do so.

Screenshots

If applicable, add screenshots to help explain your problem.

N/A

Additional context

Add any other context about the problem here.

web server is aiohttp, straight from your MS code samples.

[anishprasad01]

Hi @master0v,

I understand you're having trouble with the error messages, but it would be helpful if you could provide more information about exactly what behavior you're experiencing, and the conditions under which it occurs.

• Are messages simply not getting sent?
• Specific types of activities, or all of them?
• Which channel?
• Have you deployed to Azure? If yes, how did you do it?
• How are you testing the bot?
• Are you using a sample bot? If so, which one and what changes did you make to it (if any)?
• Have you tried debugging the bot?
• Have you noticed any patterns to the errors?

[master0v]

Hi Anish, thanks for your reply and your questions. Let me address all of them

Are messages simply not getting sent?

yes, the message doesn't get sent when this error happens

Specific types of activities, or all of them?

doesn't look like it has to do with activity type. I am simply calling "TurnContext.send_activity()" function

Which channel?

Microsoft Teams

Have you deployed to Azure? If yes, how did you do it?

No, and not planning to.

How are you testing the bot?

I have it deployed in my development msTeams environment. I found that it behaves differently in Teams than in other channels, or in testing, so I don't see the point in testing it with anything other than the Teams itself, since the final deployment is going to MS Teams.

Are you using a sample bot? If so, which one and what changes did you make to it (if any)?

yes, using modified EchoBot from BotBuilder-Samples

Have you tried debugging the bot?

yes, of course

Have you noticed any patterns to the errors?

No

[anishprasad01]

Teams does indeed behave differently. Have you tried using one of the Teams bot samples instead? They're written slightly differently. Give the Teams Conversation Bot a try as-is and let me know if the same issues persist, and what error messages appear, if any.

[master0v]

Hi Anish, thanks for the suggestion. I've examined that sample as well. My use case is really very simple: it is a 1:1 conversation with the bot. User says something, the bot replies. It would've been good if Teams supperted Suggested Actions, but it doesn't so I am forced to use a HeroCard with buttons. Other than that, I don't see anything in that sample, that I can use to improve my bot.

[anishprasad01]

As your bot is not currently functioning correctly, I would like to use the unmodified Teams sample to identify where the issues are coming from.

At this point, we don't know if the issue is in your bot, in the Azure Bot service, or in Teams. If you test connecting an unmodified sample written specifically for Teams to your Teams environment and we don't see the same behavior, we'll know the bug is in the code or vice versa.

[master0v]

ok, I will try that and report back. However, I must point out that even in my code the error doesn't happen consistently or frequently. I will have to write a stress test in order to really push that sample to break.

[anishprasad01]

I haven't been able to reproduce the issue thus far over the various Teams test bots I have. It could be specific to your environment, or it could be some other issue. Until we narrow it down, we can't begin to fix it.

[master0v]

Could you at least add some additional messaging, and make sure all of the exceptions get thrown with good/clear messaging?

[master0v]

btw, it looks like I am not the only one having this issue: https://github.com/microsoft/BotBuilder-Samples/issues/2738

[anishprasad01]

That's an old post on a much older version and it has no resolution, so unfortunately it may not be of much help to us. Regardless, were you able to test a clean sample? What were the results?

[stevkan]

@master0v - FYI, going forward, I'll be assisting you with this issue as @anishprasad01 will be unavailable for an extended amount of time.

To be clear, is the goal here to improve the error messaging, to identify what is causing your bot to sporadically fail sending messages, or both?

Also, is it possible for you to zip and post your project files here for me to review (minus any secrets/keys)? This way I can rely on your source files versus trying to accurately repro your setup.

[master0v]

Hey @stevkan , Good to meet you, man!

I don't think we can achieve the latter without the former. Iets focus on improving the error messaging, and hopefully that will help identify and solve what is causing the failure.

I can't send you the code, it really is no different from what is in the sample from the bot standpoint. I did do some experiments and added additional debugging information to it though, and I can report the following" The message actually gets sent to the bot, the Exception gets thrown despite the user receiving the message. I can't ignore this exception from my standpoint though, because there are other use cases for the when the message cannot be sent and I do need to know about it.

[stevkan]

@master0v - I don't mean to put the onus on you, but as repro'ing your issue hasn't worked, are you able to try building and accessing the botbuilder-python library locally in your bot with the following change? If the code change is successful and returns a proper error message, then we can look at getting the library updated.

The following code is located in botbuilder-core/botbuilder/core/turn_context.py, line 156 (ref code here):

async def send_activity(
        self,
        activity_or_text: Union[Activity, str],
        speak: str = None,
        input_hint: str = None,
    ) -> ResourceResponse:
        """
        Sends a single activity or message to the user.
        :param activity_or_text:
        :return:
        """
        try:
            if isinstance(activity_or_text, str):
                activity_or_text = Activity(
                    text=activity_or_text,
                    input_hint=input_hint or InputHints.accepting_input,
                    speak=speak,
                )
            result = await self.send_activities([activity_or_text])
            return result[0] if result else None
        except BaseException as err:
            print(f"Unexpected {err=}, {type(err)=}")
            raise

Naturally, let me know if you have any questions.

[stevkan]

Closing due to inactivity.

[master0v]

What "inactivity"? Is there any kind of requirement to reply during a certain time window? I was not aware of that. Please re-open, as the problem is still there, and don't close it until solved.

I've read up some more on the nature of concurrent.futures.CancelledError and it looks that a likely cause is task.cancel() In fact I am able to consistently reproduce this error by replacing the await localTurnContext.send_activity call with await asyncio.sleep(60).

Is there anything in the botbuilder library that requires TeamsActivityHandler.on_message_activity to return within a certain timeframe, and cancels the co-routine if it doesn't complete within that timeframe?

If that is there case, how do I increase that timeframe or eliminate it altogether?