Unable to Access Remote Function Logs Directly in Python

[MansurPro]

Description:

I am currently using the Modal client library to deploy and run functions remotely. While I can view the logs of these functions through the Modal dashboard and terminal when accessing the deployed functions, I am unable to access these logs directly within my Python code during runtime. This limitation hinders my ability to monitor the progress and debug remote functions efficiently.

Steps to Reproduce:

• 1) Deploy a function using the Modal client library.
• 2) Execute the remote function call.
• 3) Attempt to view the logs of the function directly in the Python code during execution.

Expected Behavior:

I expect to be able to poll and retrieve the logs of the remote function call directly within the Python code, allowing for real-time monitoring and debugging of the function's execution.

Actual Behavior:

Currently, the logs are only accessible through the Modal dashboard or the terminal where the deployed functions are managed, but not directly within the Python runtime environment.

Additional Information:

• Library Version: modal client version: 0.62.236
• Python Version: Python 3.11.3
• Operating System: macOS Sonoma - Version 14.4.1

• Code Snippet:

  def get_job_result(call_id):
  function_call = modal.functions.FunctionCall.from_id(call_id)
  
  try:
      result = function_call.get()
  except Exception as e:
      return None
  return result

Only status of the Job (running function) can we see:

• https://modal.com/docs/reference/modal.call_graph

Additional Context:

It would be beneficial to have a method or attribute to directly access the logs of a remote function call within the Python environment. This feature would greatly enhance the usability and debugging experience when working with remote functions.

[mwaskom]

Hi, that's correct, we don't support Python API access to logs currently. We will likely add support for it at some point, but no specific timeline estimate right now.

[MansurPro]

@mwaskom, Can you just make documentation about function from which you are using to display terminal logs? Isn't that already using Python API to show logs to users?

[mwaskom]

We use an internal ("private") function for that, as it may not work for all usecases (e.g., it currently only streams and doesn't handle disconnects well), and we do not expect the API to be stable over time. Limiting the official API helps us devote resources to core functionality — hopefully you can understand!

[mwaskom]

Closing discussion here but we do have this in our internal feature tracker and expect to add it at some point — hopefully soon, though no specific timeline has been assigned yet.