Closed djaym7 closed 1 year ago
Sorry @djaym7 still a little unclear to me. Can you explain the motivation a little more as well as the desired solution? This is for debugging purposes, right?
If so, we offer the ability to show errors in the console using the show_error
parameter, as described here: https://gradio.app/docs/#launch
I want to show all the output in console, not just the errors. The users of the app are technical folks and want to see console logs.
Not debugging, motivation is to see all the logs
Ok I'm starting to see the picture here. So you want a way to be able to pipe general console output into a Gradio component (specifically a Textbox)? If you don't mind me asking, why not look at the console directly?
Exactly yes
The users dont have access to server!, (output to text box or browser console .. )
Another motivation is to show the overall progress of the process. Step1 complete, step 2 running, ... and so on. Current version creates confusion with users if process is running properly or browser hung up or internet or something else went wrong
Oh I thought you were talking about the javascript console, but you're referring to the output from the Python terminal. Thanks for clarifying!
Similar to gr.error
, is there a usage of gr.log
to output the intermediate state of each run?
No we currently do not have a gr.log
, but you can run any arbitrary code within your function, so you could, e.g. log data onto the machine that is running the code.
Another motivation is to show the overall progress of the process. Step1 complete, step 2 running, ... and so on. Current version creates confusion with users if process is running properly or browser hung up or internet or something else went wrong
@djaym7 we now have a gr.Progress
that can be used to display arbitrary messages in the form of a progress bar, see https://gradio.app/docs/#progress
Let me know if this addresses your concerns and we can close this issue.
No we currently do not have a
gr.log
, but you can run any arbitrary code within your function, so you could, e.g. log data onto the machine that is running the code.
Thanks for your clarifications, but I still want to display my logs on the webpage (like in gr.Textbox
, not in the console). I wonder if there are any future plans for this feature.
No we currently do not have a
gr.log
, but you can run any arbitrary code within your function, so you could, e.g. log data onto the machine that is running the code.Thanks for your clarifications, but I still want to display my logs on the webpage (like in
gr.Textbox
, not in the console). I wonder if there are any future plans for this feature.
+1
I explored this and it is possible to achieve this functionality by piping sys.stdout
to a file and then reading from that file every second or so and then displaying the contents of that file in a gr.Textbox()
. Here's a full code snippet that accomplishes this:
import gradio as gr
import sys
class Logger:
def __init__(self, filename):
self.terminal = sys.stdout
self.log = open(filename, "w")
def write(self, message):
self.terminal.write(message)
self.log.write(message)
def flush(self):
self.terminal.flush()
self.log.flush()
def isatty(self):
return False
sys.stdout = Logger("output.log")
def test(x):
print("This is a test")
print(f"Your function is running with input {x}...")
return x
def read_logs():
sys.stdout.flush()
with open("output.log", "r") as f:
return f.read()
with gr.Blocks() as demo:
with gr.Row():
input = gr.Textbox()
output = gr.Textbox()
btn = gr.Button("Run")
btn.click(test, input, output)
logs = gr.Textbox()
demo.load(read_logs, None, logs, every=1)
demo.queue().launch()
Which produces this:
我对此进行了探索,可以通过管道传输
sys.stdout
到一个文件,然后每隔一秒左右从该文件中读取一次,然后在一个gr.Textbox()
. 这是完成此操作的完整代码片段:import gradio as gr import sys class Logger: def __init__(self, filename): self.terminal = sys.stdout self.log = open(filename, "w") def write(self, message): self.terminal.write(message) self.log.write(message) def flush(self): self.terminal.flush() self.log.flush() def isatty(self): return False sys.stdout = Logger("output.log") def test(x): print("This is a test") print(f"Your function is running with input {x}...") return x def read_logs(): sys.stdout.flush() with open("output.log", "r") as f: return f.read() with gr.Blocks() as demo: with gr.Row(): input = gr.Textbox() output = gr.Textbox() btn = gr.Button("Run") btn.click(test, input, output) logs = gr.Textbox() demo.load(read_logs, None, logs, every=1) demo.queue().launch()
产生这个:
![]()
It is useful. Thank you.
Any way to redirect output from another script which is running in a new process to this output.log file ? My gradio file starts a new process using os.system() and that script's stdout is not redirected here
figured out a way, using subprocess.call(command, stdout=sys.stdout.log,shell=True) and to truncate the output, in readlogs, using deque to keep last N lines.
This works awesome, though i am having an issue with having the textbox scroll be stuck at the top when max_lines is set. Is there currently a built in way to have the textbox force-scroll to the bottom; showing the latest output?
I explored this and it is possible to achieve this functionality by piping
sys.stdout
to a file and then reading from that file every second or so and then displaying the contents of that file in agr.Textbox()
. Here's a full code snippet that accomplishes this:import gradio as gr import sys class Logger: def __init__(self, filename): self.terminal = sys.stdout self.log = open(filename, "w") def write(self, message): self.terminal.write(message) self.log.write(message) def flush(self): self.terminal.flush() self.log.flush() def isatty(self): return False sys.stdout = Logger("output.log") def test(x): print("This is a test") print(f"Your function is running with input {x}...") return x def read_logs(): sys.stdout.flush() with open("output.log", "r") as f: return f.read() with gr.Blocks() as demo: with gr.Row(): input = gr.Textbox() output = gr.Textbox() btn = gr.Button("Run") btn.click(test, input, output) logs = gr.Textbox() demo.load(read_logs, None, logs, every=1) demo.queue().launch()
Which produces this:
![]()
Sorry, I got this error, and can not got logs. please help me, thanks:
Traceback (most recent call last):
File "/Users/luis/PycharmProjects/yamnet_test/app.py", line 49, in <module>
demo.load(read_logs, None, logs, every=1)
File "/Users/luis/miniconda3/envs/python38/lib/python3.8/site-packages/gradio/interface.py", line 110, in load
return super().load(name=name, src=src, api_key=api_key, alias=alias, **kwargs)
File "/Users/luis/miniconda3/envs/python38/lib/python3.8/site-packages/gradio/blocks.py", line 809, in load
return external.load_blocks_from_repo(name, src, api_key, alias, **kwargs)
File "/Users/luis/miniconda3/envs/python38/lib/python3.8/site-packages/gradio/external.py", line 30, in load_blocks_from_repo
tokens = name.split(
AttributeError: 'function' object has no attribute 'split'
import gradio as gr
from test import predict_uri
import sys
examples = [['miaow_16k.wav']]
title = "yamnet test"
description = "An audio event classifier trained on the AudioSet dataset to predict audio events from the AudioSet ontology."
class Logger:
def __init__(self, filename):
self.terminal = sys.stdout
self.log = open(filename, "w")
def write(self, message):
self.terminal.write(message)
self.log.write(message)
def flush(self):
self.terminal.flush()
self.log.flush()
def isatty(self):
return False
sys.stdout = Logger("output.log")
def test(x):
print("This is a test")
print(f"Your function is running with input {x}...")
return x
def read_logs():
sys.stdout.flush()
with open("output.log", "r") as f:
return f.read()
with gr.Interface(predict_uri, inputs=gr.inputs.Audio(type="filepath"), outputs=["text", 'plot']) as demo:
examples = examples,
title = title,
description = description,
allow_flagging = 'never'
logs = gr.Textbox()
demo.load(read_logs, None, logs, every=1)
demo.launch(enable_queue=True, show_error=True)
I explored this and it is possible to achieve this functionality by piping
sys.stdout
to a file and then reading from that file every second or so and then displaying the contents of that file in agr.Textbox()
. Here's a full code snippet that accomplishes this:import gradio as gr import sys class Logger: def __init__(self, filename): self.terminal = sys.stdout self.log = open(filename, "w") def write(self, message): self.terminal.write(message) self.log.write(message) def flush(self): self.terminal.flush() self.log.flush() def isatty(self): return False sys.stdout = Logger("output.log") def test(x): print("This is a test") print(f"Your function is running with input {x}...") return x def read_logs(): sys.stdout.flush() with open("output.log", "r") as f: return f.read() with gr.Blocks() as demo: with gr.Row(): input = gr.Textbox() output = gr.Textbox() btn = gr.Button("Run") btn.click(test, input, output) logs = gr.Textbox() demo.load(read_logs, None, logs, every=1) demo.queue().launch()
Which produces this:
![]()
Thanks for the solution @abidlabs, it works great!
However, just a heads up for anyone that may have dropdowns in their UIs, in my experience after each every
seconds they will close and you will need to open them again. This makes it impossible to use dropdowns if you have a very short refresh rate (e.g. <1s).
It's possible that other components may be affected in a similar way.
In any case, I understand this is just a workaround and not meant to work perfectly :) Thanks again for the snippet!
Thanks for pointing that out @dt-hicham-elboukkouri I'll create a separate issue so we don't lose track of it
Just in case anyone else can't live without colored terminal text, it is in fact possible incorporate Python rich Console to capture the output, export it to HTML, and really bring the command line to life in Gradio. Using builtins.print and a wrapper print function.
If you check this Bark fork it's not in it yet, was just testing this out locally, and I'm not sure it doesn't cause a terrible problem. But the code is simple, just Rich Console with capture set to true, builtins.print capturing and redirecting all print to the console.
The original motivation for the console was the typical workflow using Bark was to run large amounts of text against different splitting and processing parameters and there was a large volume of relevant feedback in the console that was otherwise difficult to bring into Gradio. Well that was the reason for the basic console, the colors here are just because they are delightful.
Thanks @JonathanFly for sharing! Very cool
This is awesome, hope it gets added in main
I explored this and it is possible to achieve this functionality by piping
sys.stdout
to a file and then reading from that file every second or so and then displaying the contents of that file in agr.Textbox()
. Here's a full code snippet that accomplishes this:import gradio as gr import sys class Logger: def __init__(self, filename): self.terminal = sys.stdout self.log = open(filename, "w") def write(self, message): self.terminal.write(message) self.log.write(message) def flush(self): self.terminal.flush() self.log.flush() def isatty(self): return False sys.stdout = Logger("output.log") def test(x): print("This is a test") print(f"Your function is running with input {x}...") return x def read_logs(): sys.stdout.flush() with open("output.log", "r") as f: return f.read() with gr.Blocks() as demo: with gr.Row(): input = gr.Textbox() output = gr.Textbox() btn = gr.Button("Run") btn.click(test, input, output) logs = gr.Textbox() demo.load(read_logs, None, logs, every=1) demo.queue().launch()
Which produces this:
![]()
Thanks for this! Just a note for others: if you use gr.Code() instead of gr.TextBox() you can change the "language" parameter it make the output more colorful (make sure to set interactive=False)
@JonathanFly would you have a pointer to the code of your beautiful demo?
I tried to invoke read_logs in a function, once I click the button I will trigger this function and then I would expect the log redirect to gradio web page, but it didn't work, anything i'm missing?
def evaluate(model_names, datasets):
logs = read_logs()
models = name2path(model_names)
print("-----")
print(models)
print(datasets)
entry = Entry()
entry.run_batch(models, datasets)
return logs
...
submit_btn = gr.Button("Evaluate")
with gr.Row():
with gr.Column():
gr.Markdown("## Log Output")
log_output = gr.TextArea(label="log output")
submit_btn.click(
fn=evaluate,
inputs=[model_names, datasets],
outputs=log_output,
every=1
)
Hey @Jeru2023 This might be of use; I created a Logger
class to be used with a gradio app (that has multiple sub-apps):
# Logger class in a shared python script called `common`, and imported as a module is the sub-app
class Logger:
def __init__(self, filename):
self.filename = f"{LOG_DIR}{filename}"
self.terminal = sys.stdout
self.reset_logs()
self.log = open(self.filename, "w")
self.flush()
def write(self, message):
self.terminal.write(message)
self.log.write(message)
def flush(self):
self.terminal.flush()
self.log.flush()
def isatty(self):
return False
def reset_logs(self):
with open(self.filename, 'w') as file:
file.truncate(0)
def read_logs(self):
sys.stdout.flush()
# Read the entire content of the log file
with open(self.filename, "r") as f:
log_content = f.readlines()
# Filter out lines containing null characters
log_content = [line for line in log_content if '\x00' not in line]
# Define the regex pattern for the progress bar
progress_pattern = re.compile(r'\[.*\] \d+\.\d+%')
# Find lines matching the progress bar pattern
progress_lines = [line for line in log_content if
progress_pattern.search(line) and " - Completed!\n" not in line]
# If there are multiple progress bars, keep only the last one in recent_lines
if progress_lines:
valid_content = [line for line in log_content if line not in progress_lines]
if log_content[-1] == progress_lines[-1]:
valid_content.append(progress_lines[-1].strip("\n"))
else:
valid_content = log_content
# Get the latest 30 lines
recent_lines = valid_content[-30:]
# Return the joined recent lines
return ''.join(recent_lines)
And here is an example of it's use in one of the sub-apps:
import gradio as gr
from common import *
from Tools.Patches import patches
EXIT_APP = False
log_file = "patches.log"
# ----------------------------------------------------------------------------------------------------------------------
# Module
# ----------------------------------------------------------------------------------------------------------------------
def module_callback(image_dir, annotation_file, image_column, label_column, patch_size, output_dir):
"""
"""
console = sys.stdout
sys.stdout = Logger(log_file)
args = argparse.Namespace(
image_dir=image_dir,
annotation_file=annotation_file,
image_column=image_column,
label_column=label_column,
patch_size=patch_size,
output_dir=output_dir,
)
try:
# Call the function
gr.Info("Starting process...")
patches(args)
print("\nDone.")
gr.Info("Completed process!")
except Exception as e:
gr.Error("Could not complete process!")
print(f"ERROR: {e}\n{traceback.format_exc()}")
sys.stdout = console
# ----------------------------------------------------------------------------------------------------------------------
# Interface
# ----------------------------------------------------------------------------------------------------------------------
def exit_interface():
"""
"""
global EXIT_APP
EXIT_APP = True
gr.Info("Please close the browser tab.")
gr.Info("Stopped program successfully!")
time.sleep(3)
def create_interface():
"""
"""
logger = Logger(log_file)
logger.reset_logs()
with gr.Blocks(title="Patches 🟩", analytics_enabled=False, theme=gr.themes.Soft(), js=js) as interface:
# Title
gr.Markdown("# Patches 🟩")
# Browse button
image_dir = gr.Textbox(f"{DATA_DIR}", label="Selected Image Directory")
dir_button = gr.Button("Browse Directory")
dir_button.click(choose_directory, outputs=image_dir, show_progress="hidden")
annotation_file = gr.Textbox(label="Selected Annotation File")
file_button = gr.Button("Browse Files")
file_button.click(choose_files, outputs=annotation_file, show_progress="hidden")
with gr.Row():
image_column = gr.Textbox("Name", label="Image Name Field")
label_column = gr.Dropdown(label="Label Name Field", multiselect=False, allow_custom_value=True,
choices=['Label'] + [f'Machine suggestion {n + 1}' for n in range(5)])
patch_size = gr.Number(112, label="Patch Size", precision=0)
# Browse button
output_dir = gr.Textbox(f"{DATA_DIR}", label="Selected Output Directory")
dir_button = gr.Button("Browse Directory")
dir_button.click(choose_directory, outputs=output_dir, show_progress="hidden")
with gr.Row():
# Run button (callback)
run_button = gr.Button("Run")
run = run_button.click(module_callback,
[image_dir,
annotation_file,
image_column,
label_column,
patch_size,
output_dir])
stop_button = gr.Button(value="Stop")
stop = stop_button.click(exit_interface)
with gr.Accordion("Console Logs"):
# Add logs
logs = gr.Code(label="", language="shell", interactive=False, container=True, lines=30)
interface.load(logger.read_logs, None, logs, every=1)
interface.launch(prevent_thread_lock=True, server_port=get_port(), inbrowser=True, show_error=True)
return interface
# ----------------------------------------------------------------------------------------------------------------------
# Main
# ----------------------------------------------------------------------------------------------------------------------
interface = create_interface()
try:
while True:
time.sleep(0.5)
if EXIT_APP:
break
except:
pass
finally:
Logger(log_file).reset_logs()
Here the output goes to console, the Code
or TextArea
, and the log file. Note that an instance of Logger is created multiple times, but it all directs to the same log file (patches.log
), so that if the use re-runs the sub-app, the log / console gets cleared. Hope that helps, it took me a while to get it just right. You'll need to modify the Logger class, as I'm doing some extra things to show a progress bar (like tqdm) and only showing the most recent 30 lines output from console.
Hey @Jeru2023 This might be of use; I created a
Logger
class to be used with a gradio app (that has multiple sub-apps):# Logger class in a shared python script called `common`, and imported as a module is the sub-app class Logger: def __init__(self, filename): self.filename = f"{LOG_DIR}{filename}" self.terminal = sys.stdout self.reset_logs() self.log = open(self.filename, "w") self.flush() def write(self, message): self.terminal.write(message) self.log.write(message) def flush(self): self.terminal.flush() self.log.flush() def isatty(self): return False def reset_logs(self): with open(self.filename, 'w') as file: file.truncate(0) def read_logs(self): sys.stdout.flush() # Read the entire content of the log file with open(self.filename, "r") as f: log_content = f.readlines() # Filter out lines containing null characters log_content = [line for line in log_content if '\x00' not in line] # Define the regex pattern for the progress bar progress_pattern = re.compile(r'\[.*\] \d+\.\d+%') # Find lines matching the progress bar pattern progress_lines = [line for line in log_content if progress_pattern.search(line) and " - Completed!\n" not in line] # If there are multiple progress bars, keep only the last one in recent_lines if progress_lines: valid_content = [line for line in log_content if line not in progress_lines] if log_content[-1] == progress_lines[-1]: valid_content.append(progress_lines[-1].strip("\n")) else: valid_content = log_content # Get the latest 30 lines recent_lines = valid_content[-30:] # Return the joined recent lines return ''.join(recent_lines)
And here is an example of it's use in one of the sub-apps:
import gradio as gr from common import * from Tools.Patches import patches EXIT_APP = False log_file = "patches.log" # ---------------------------------------------------------------------------------------------------------------------- # Module # ---------------------------------------------------------------------------------------------------------------------- def module_callback(image_dir, annotation_file, image_column, label_column, patch_size, output_dir): """ """ console = sys.stdout sys.stdout = Logger(log_file) args = argparse.Namespace( image_dir=image_dir, annotation_file=annotation_file, image_column=image_column, label_column=label_column, patch_size=patch_size, output_dir=output_dir, ) try: # Call the function gr.Info("Starting process...") patches(args) print("\nDone.") gr.Info("Completed process!") except Exception as e: gr.Error("Could not complete process!") print(f"ERROR: {e}\n{traceback.format_exc()}") sys.stdout = console # ---------------------------------------------------------------------------------------------------------------------- # Interface # ---------------------------------------------------------------------------------------------------------------------- def exit_interface(): """ """ global EXIT_APP EXIT_APP = True gr.Info("Please close the browser tab.") gr.Info("Stopped program successfully!") time.sleep(3) def create_interface(): """ """ logger = Logger(log_file) logger.reset_logs() with gr.Blocks(title="Patches 🟩", analytics_enabled=False, theme=gr.themes.Soft(), js=js) as interface: # Title gr.Markdown("# Patches 🟩") # Browse button image_dir = gr.Textbox(f"{DATA_DIR}", label="Selected Image Directory") dir_button = gr.Button("Browse Directory") dir_button.click(choose_directory, outputs=image_dir, show_progress="hidden") annotation_file = gr.Textbox(label="Selected Annotation File") file_button = gr.Button("Browse Files") file_button.click(choose_files, outputs=annotation_file, show_progress="hidden") with gr.Row(): image_column = gr.Textbox("Name", label="Image Name Field") label_column = gr.Dropdown(label="Label Name Field", multiselect=False, allow_custom_value=True, choices=['Label'] + [f'Machine suggestion {n + 1}' for n in range(5)]) patch_size = gr.Number(112, label="Patch Size", precision=0) # Browse button output_dir = gr.Textbox(f"{DATA_DIR}", label="Selected Output Directory") dir_button = gr.Button("Browse Directory") dir_button.click(choose_directory, outputs=output_dir, show_progress="hidden") with gr.Row(): # Run button (callback) run_button = gr.Button("Run") run = run_button.click(module_callback, [image_dir, annotation_file, image_column, label_column, patch_size, output_dir]) stop_button = gr.Button(value="Stop") stop = stop_button.click(exit_interface) with gr.Accordion("Console Logs"): # Add logs logs = gr.Code(label="", language="shell", interactive=False, container=True, lines=30) interface.load(logger.read_logs, None, logs, every=1) interface.launch(prevent_thread_lock=True, server_port=get_port(), inbrowser=True, show_error=True) return interface # ---------------------------------------------------------------------------------------------------------------------- # Main # ---------------------------------------------------------------------------------------------------------------------- interface = create_interface() try: while True: time.sleep(0.5) if EXIT_APP: break except: pass finally: Logger(log_file).reset_logs()
Here the output goes to console, the
Code
orTextArea
, and the log file. Note that an instance of Logger is created multiple times, but it all directs to the same log file (patches.log
), so that if the use re-runs the sub-app, the log / console gets cleared. Hope that helps, it took me a while to get it just right. You'll need to modify the Logger class, as I'm doing some extra things to show a progress bar (like tqdm) and only showing the most recent 30 lines output from console.
This is really cool, it worked like a charm, and i've learned a lot from your code, thank you so much!!!
I developed a custom component which can pipe some log file to the frontend, just use python logging function to write log to a file instead of print, and this component will do the job. And it can show colored log too.
code: https://github.com/louis-she/gradio-log
Very nice @louis-she !
Reopening this https://github.com/gradio-app/gradio/issues/2360
Original description from 2360 : When one submits, show live log of console outputs on gradio output box.
This is a feature request, not an issue