Closed HJFG closed 9 months ago
hi I wish to contribute to the issue
Contributions are welcom @kshitiz305 !
I'm encountering this same bug. transformers is 100% installed, and this code works fine outside of reload mode.
import langchain_helper as lch import gradio as gr import torch import sd_generator as sd import time from datetime import datetime from diffusers import StableDiffusionXLPipeline import os import gc from transformers import AutoModelForCausalLM, AutoTokenizer
def reclaim_mem():
torch.cuda.ipc_collect()
gc.collect()
torch.cuda.empty_cache()
time.sleep(0.01)
print(f"Memory after del {torch.cuda.memory_allocated()}")
reclaim_mem() def generate_datetime(): now = datetime.now() date_time = now.strftime("%m/%d/%Y, %H:%M:%S") return date_time
def make_folder(): foldertimestr = time.strftime("%Y%m%d-%H") folder_path = f"./output/{foldertimestr}" if not os.path.exists(folder_path): os.mkdir(folder_path) return foldertimestr
def gen_mon_desc(user_monster_type, user_monster_color,user_monster_size): user_log = [] user_monster_type = user_monster_type user_monster_color = user_monster_color user_monster_size = user_monster_size
response = lch.generate_monster_desc(user_monster_type, user_monster_color, user_monster_size)
user_log.append(response)
global sd_input
sd_input = response
reclaim_mem()
return response
demo = gr.Blocks() with gr.Blocks() as demo:
gr.Interface(
fn=gen_mon_desc,
inputs=[gr.Textbox(lines = 3, placeholder="What is the name or type of your monster, be descriptive..."),
gr.Textbox(lines = 3, placeholder="What is the color and some details?"),
gr.Dropdown(['Tiny','Small','Medium','Large','Huge','Gigantic','Titanic'], label = 'Size?')],
outputs="text"
)
if name == 'main': demo.launch()
CLI shows : Watching: 'C:\AI\StatBlockGenerator\streamlit-build\venv\lib\site-packages\gradio', 'C:\AI\StatBlockGenerator\experiments'
Traceback (most recent call last):
File "C:\AI\StatBlockGenerator\experiments\main.py", line 2, in
I'm not sure about rye (what the original issue comment mentions) but I think this problem is fixed in the v4 branch to be released in about two weeks.
I'm not sure about rye (what the original issue comment mentions) but I think this problem is fixed in the v4 branch to be released in about two weeks.
Reloading main failed with the following exception: ModuleNotFoundError: No module named '__main__'
The issues persists in gradio==4.0.2 @freddyaboulton I'm not using rye but simply with
gradio main.py
Can you share the code @cheulyop ? Will take a look - thanks!
Sure. Here's a snippet. Let me know if this isn't enough, @freddyaboulton Thanks.
import gradio as gr
from loguru import logger
log_path = f"logs/{datetime.now().strftime('%Y-%m-%d')}.log"
logger.add(log_path, rotation="1 day", retention="6 months", level="INFO")
with gr.Blocks() as demo:
with gr.Row():
with gr.Column():
chatbot = gr.Chatbot(show_copy_button=True, height=750)
msg = gr.Textbox(label="User message.", placeholder="Type here...")
clear = gr.ClearButton(
[
msg,
chatbot,
],
value="New Conversation",
)
if not IS_PROD:
with gr.Column():
search_query = gr.Textbox(
label="Search query",
interactive=False,
)
search_results = gr.Code(
label="Search results", language="json", interactive=False, lines=10
)
else:
search_query = None
search_results = None
@logger.catch
def submit_handler(*args, **kwargs):
return user_msg_submitted(*args, **kwargs)
msg.submit(
submit_handler,
[msg, chatbot],
[msg, chatbot, search_query, search_results] if not IS_PROD else [msg, chatbot],
)
clear.click()
if __name__ == "__main__":
# demo.queue()
logger.info("Starting demo server...")
demo.launch(
share=os.getenv("SHARE", "False") == "True",
server_name=os.getenv("SERVER_NAME", "127.0.0.1"),
server_port=int(os.getenv("SERVER_PORT", "7860")),
)
logger.info("Demo server stopped.")
@freddyaboulton freddyaboulton Can confirm this issue is fixed for me with 4.0, thank you!
EDIT: Update, actually this is technically working but seems to continue running in the background and in a new updated run. IE : It's making each code call once for each refresh from an updated save file. This led to a Blue Screen Of Death crash. Which leaves me having to kill the terminal to stop overloading my GPU memory. To reproduce I ran once, made an alteration to code to trigger a refresh. It started making each call to my LLM twice. Then after crashing the terminal, rebooting my venv, and running gradio main.py, with no inputs it starts right where it left off and runs in a loop. No input from me (user) from the UI. Refreshing the browser page seems to resolves this.
Example of first run with no code changes :
Prompt Template : You are an intelligent and very good writer of fantasy fiction that uses a diverse and wide ranging vocabulary. Write a very concise, visually descriptive, detailed and colorful paragraph description of a Eagle Headed, lion bodied, winged griffin, include description of its body, face, and limbs, it is with a dark skinned female knight mounted on it's back and size Large at on a castle wall overlooking an army of monsters . The with a dark skinned female knight mounted on it's back Eagle Headed, lion bodied, winged griffin LLM load took 10.821446657180786 Memory allocated : 8485902336 30.014198780059814 pipe load took load took 30.01427936553955 Memory allocated : 8494422016
The majestic eagle-headed, lion-bodied, winged griffin stood tall and proud on the castle wall, its regal presence casting a shadow over the army of monsters below. Its body was strong and muscular, resembling a lion's, with a thick, shaggy mane that rippled in the wind. Its wings, wide and powerful, stretched out from its back, ready to take flight at a moment's notice. Its head, adorned with piercing, golden eyes, was that of an eagle, sharp and cunning, scanning the horizon for potential threats. Its limbs were long and agile, ending in razor-sharp talons, perfect for grasping and tearing. Its torso was sleek and streamlined, a testament to its incredible strength and agility. The griffin's appendages, both wings and talons, were a sight to behold, their power and grace a testament to its divine origin. Atop the magnificent creature stood a dark-skinned female knight, her armor gleaming in the sunlight, her sword held high, ready to face the monstrous horde below. Together, the griffin and knight formed an unstoppable force, their bond unbreakable as they soared over the castle wall, casting a shadow of fear over the enemy below.
This is after updating code. It starts immediately calling the LLM, and does it in a loop, seems like once for each time I've updated the code. Refreshing the browser window seems to resolve this after it's complete the loop.
Prompt Template : You are an intelligent and very good writer of fantasy fiction that uses a diverse and wide ranging vocabulary. Write a very concise, visually descriptive, detailed and colorful paragraph description of a A feminine armored orc, include description of its body, face, and limbs, it is astride a golden and white eagle headed griffin and size Large at . The astride a golden and white eagle headed griffin A feminine armored orc LLM load took 7.31205677986145 Memory allocated : 8485902336 pipe load took load took 0.0 Memory allocated : 8485902336 35.304688453674316 pipe load took load took 35.304688453674316 Memory allocated : 8494422016
In the heart of the battlefield, a magnificent sight unfolds: a feminine armored Orc, standing tall and proud at a size Large. Her body, a testament to her strength and resilience, is encased in gleaming, intricately designed armor that shimmers in the sunlight. Her head, adorned with a helmet that bears the insignia of her clan, is a perfect blend of ferocity and grace, with piercing, almond-shaped eyes that seem to see right through you.
Her limbs, powerful and muscular, end in sharp, deadly claws that can tear through even the toughest of foes. Her torso, broad and imposing, houses a heart full of courage and determination. As she astride her loyal steed, a golden and white eagle-headed griffin, she commands an aura of dominance and authority.
The griffin, with its majestic wings spread wide, soars gracefully through the chaos of the battlefield, its eyes locked with those of its rider. The bond between the two is palpable, a testament to their unbreakable trust and loyalty. Together, they cut through the enemy ranks with ease, leaving a trail of destruction in their wake.
This feminine armored Orc, a force to be reckoned with, stands as a symbol of strength, courage, and unity, her presence alone enough to strike fear into the hearts of her adversaries. As she charges into battle, her golden and white eagle-headed griffin by her side, the battlefield trembles with anticipation of the carnage to come. Memory after del 8519680 Prompt Template : You are an intelligent and very good writer of fantasy fiction that uses a diverse and wide ranging vocabulary. Write a very concise, visually descriptive, detailed and colorful paragraph description of a A feminine armored orc, include description of its body, face, and limbs, it is astride a golden and white eagle headed griffin and size Large at . The astride a golden and white eagle headed griffin A feminine armored orc LLM load took 4.411355495452881 Memory allocated : 8494422016 pipe load took load took 0.0 Memory allocated : 8494422016
Code that includes gradio
import description_helper as dsh
import gradio as gr
import sd_generator as sd
import statblock_helper as sth
import utilities as u
import process_html
# Build functions that will be called by Gradio UI
# Take input from gradio user and call the llm in description_helper
def gen_mon_desc(user_monster_type, user_monster_color,user_monster_size, user_monster_loc):
# Prompt template with user inputs added to be printed to log and terminal
prompt_template = f"You are an intelligent and very good writer of fantasy fiction that uses a diverse and wide ranging vocabulary. Write a very concise, visually descriptive, detailed and colorful paragraph description of a {user_monster_type}, include description of its body, face, and limbs, it is {user_monster_color} and size {user_monster_size} at {user_monster_loc}. The {user_monster_color} {user_monster_type} "
print("Prompt Template : " + prompt_template)
# setting sd_input as global to be called and moved between models, this could probably be done better as a variable called from description helper
global sd_input
response = dsh.generate_monster_desc(user_monster_type, user_monster_color, user_monster_size,user_monster_loc)
sd_input = response
print(sd_input)
# A list to hold user inputs and start user log,
# Build static folders, this too needs to go to utilities
#Create user log path all this needs to be moved to utilities
user_log = []
folder_path = f"./output/{u.make_folder()}"
user_log_file = open(f"{folder_path}/userlog.txt","a")
user_log.append(prompt_template)
user_log.append(f"Output from LLM: {response}")
date_time = u.generate_datetime()
print(date_time, file = user_log_file)
print(user_log, file= user_log_file)
user_log_file.close()
u.reclaim_mem()
return response
# Pass sd_input and call function to generate image and save bu calling name making function in utilities
def gen_mon_img():
sd_input = dsh.generate_monster_desc.response
generated_image = sd.generate_image(sd_input)
generated_image.save(u.make_image_name())
return generated_image
# Pass modified input to statblock generator
def gen_mon_statblock(challenge_rating,abilities):
input = sd_input + f"Challenge Rating Must Be : {challenge_rating}, it has abilities {abilities}"
generated_statblock = sth.generate_statblock(input)
return generated_statblock
# Call the html process program and point to the file
def mon_html_process():
mon_file_name = sth.file_name_list[0]+'/' + sth.file_name_list[1] +'.html'
print("Ouput path = " + mon_file_name)
process_html.process_html(mon_file_name)
# Build the html and file path to pass to gradio to output html file in gr.html
def gen_link():
mon_file_path = sth.file_name_list[0]+'/' + sth.file_name_list[1] +'.html'
iframe = iframe = f"""<iframe src="file={mon_file_path}" width="100%" height="500px"></iframe>"""
link = f'<a href="file={mon_file_path}" target="_blank">{sth.file_name_list[1] +".html"}</a>'
return link, iframe
# Build gradio app
demo = gr.Blocks()
with gr.Blocks() as demo:
# Title, eventually turn this into an updated variable with Monster name
gr.HTML(""" <div id="inner"> <header>
<h1>Monster Statblock Generator</h1>
</div>""")
with gr.Tab("Generator"):
with gr.Column():
# Does this need to be a column? Building interface to receive input
gr.Interface(
fn=gen_mon_desc,
inputs=[gr.Textbox(label = "Step 1 : The Monster's Name or type", lines = 1, placeholder=f"Ex : A Royal Griffin Lancer"),
gr.Textbox(label = "Step 2 : Colors and description", lines = 3, placeholder="Ex: A female knight in shining armor in a saddle on the back of a golden and white Griffin with lance pointed up"),
gr.Dropdown(['Tiny','Small','Medium','Large','Huge','Gigantic','Titanic'], label = 'Step 3 : Size '),
gr.Textbox(label = "Step 4 Optional : Descirbe it's location", lines = 3, placeholder="on a castle wall overlooking a monsterous army " )],
outputs=gr.Textbox(lines = 16),
allow_flagging="never"
)
# Output object for image, and button to trigger
output_image = gr.Image()
image_Gen = gr.Button(value = "Generate Image" )
image_Gen.click(gen_mon_img, outputs = output_image)
# Create a tab to split off statblock generation
with gr.Tab("Statblock"):
# Interface to take in challenge rating and call statblock generating function(Needs explanation of what a challenge rating is)
gr.Interface(
fn=gen_mon_statblock,
inputs = [gr.Textbox(label="Challenge Rating, 1-20", lines =1),
gr.Textbox(label="Abilities", lines = 3)],
outputs = gr.Textbox(lines = 20,interactive=True),
allow_flagging="never"
)
# Build buttons to modify to html and show html
gen_html = gr.Button(value = "Generate html")
gen_html.click(mon_html_process,inputs =[], outputs=[])
markdown = gr.Markdown(label="Output Box")
html = gr.HTML(label="HTML preview", show_label=True)
new_link_btn = gr.Button("link")
new_link_btn.click(fn = gen_link, inputs = [], outputs = [markdown, html])
if __name__ == '__main__':
demo.launch(share = False)```
I encounter this error too ModuleNotFoundError: No module named 'main'
I am facing the same issue
ModuleNotFoundError: No module named '__main__'
Hi team - I am definitely running into the same issue as well. Is there anything we can do to accelerate the fix for this? As far as I can tell, auto reloading just does not work because of this issue and renders the "gradio cc dev" completely pointless. I am not using rye and just doing gradio cc dev
. These are my versions:
gradio==4.8.0 gradio_client==0.7.1
Thank you.
Same issue here!
Gradio version: 4.9.0
@abidlabs Hey this bug has existed in Gradio for quiet a while and it's still present in the latest version. It makes continues development really annoying because you have to rerun the app which each ui change.
Sorry about that! cc @freddyaboulton who I believe is looking into this issue
Hi @arian81 , @joaomorossini , @sangv, @MohamedAliRashad , and others!
First of all, apologies for the delay.
I have not been able to reproduce this issue in a fresh environment with gradio 4.10.0 installed (the latest version).
Here I am running the code @cheulyop shared. I am able to make a change and have the app reload automatically without restarting the whole app.
https://github.com/gradio-app/gradio/assets/41651716/42ed1f9c-70c9-4340-a7c7-38547216f5ee
I really want to fix this but it will be difficult if I can't reproduce the problem. Can you do the following to help me figure out what's happening?
gradio environment
gradio
command from. Maybe it's something strange like that.Thank you for your help in advanced!
Thanks @freddyaboulton for looking into this. I can confirm that I still have this issue with the versions shown below.
pip list | grep 'gradio' gradio 4.10.0 gradio_client 0.7.3
Followed by: gradio cc dev ♻ Launching demo/app.py in reload mode
And finally this:
Changed detected in: {my_home_dir_location}/demo/app.py
Reloading demo.app failed with the following exception:
ModuleNotFoundError: No module named 'main'
any progress or ETA on the issue?
Running version from #6983 I finally got a traceback of what's causing this issue. It's the python-dotenv
library failing.
File "C:\Users\.virtualenvs\lib\site-packages\dotenv\main.py", line 336, in load_dotenv
dotenv_path = find_dotenv()
File "C:\Users\.virtualenvs\lib\site-packages\dotenv\main.py", line 286, in find_dotenv
if usecwd or _is_interactive() or getattr(sys, 'frozen', False):
File "C:\Users\.virtualenvs\lib\site-packages\dotenv\main.py", line 283, in _is_interactive
main = __import__('__main__', None, None, fromlist=['__file__'])
ModuleNotFoundError: No module named '__main__
I can confirm not using dotenv package i.e not calling load_dotenv()
function will resolve the issue. Hopefully this is a thread for devs to pull on and figure out how to fix this bug. For now the only solution is to not use python-dotenv
package to load .env files. @freddyaboulton
When hotreload the server the gradio==4.14.0 I receive the error
AttributeError: module 'main' has no attribute 'demo'
here is my code :
def configure_demo_interface():
with gr.Blocks() as demo:
chatbot = gr.Chatbot(
avatar_images=(
None,
(os.path.join(os.path.dirname(__file__), "images/doc.jpg")),
)
)
tts_output = gr.Audio()
msg = gr.Textbox()
gr.ClearButton([msg, chatbot])
msg.submit(run_text_prompt, [msg, chatbot], [msg, chatbot, tts_output])
asr = None
with gr.Row():
audio = gr.Audio(type="filepath")
button = gr.Button("Sent audio")
if audio is not None:
asr = ModelASR()
button.click(
asr.run_audio_prompt, [audio, chatbot], [audio, chatbot, tts_output]
)
return demo
log_file = create_logfile()
chat = chatPlugin(log_file)
configure_logging(log_file)
if __name__ == "__main__":
demo = configure_demo_interface()
demo.launch(server_name="0.0.0.0") ```
Run code by command
`gradio main.py`
@sangv @joaomorossini @MohamedAliRashad Can you see what traceback you get with 4.14.0?
@Nghiauet Please create the demo object outside the if __name__ == "__main__"
clause.
@sangv @joaomorossini @MohamedAliRashad Can you see what traceback you get with 4.14.0?
@Nghiauet Please create the demo object outside the
if __name__ == "__main__"
clause.
@freddyaboulton This is what I got here:
Changes detected in: /Users/morossini/Documents/1.Projects/arxiv_openai_assistant/main.py
Reloading main failed with the following exception:
Traceback (most recent call last):
File "/Users/morossini/Documents/1.Projects/arxiv_openai_assistant/venv/lib/python3.10/site-packages/gradio/utils.py", line 207, in watchfn
module = importlib.import_module(reloader.watch_module_name)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 883, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "/Users/morossini/Documents/1.Projects/arxiv_openai_assistant/main.py", line 29, in <module>
load_dotenv()
File "/Users/morossini/Documents/1.Projects/arxiv_openai_assistant/venv/lib/python3.10/site-packages/dotenv/main.py", line 336, in load_dotenv
dotenv_path = find_dotenv()
File "/Users/morossini/Documents/1.Projects/arxiv_openai_assistant/venv/lib/python3.10/site-packages/dotenv/main.py", line 286, in find_dotenv
if usecwd or _is_interactive() or getattr(sys, 'frozen', False):
File "/Users/morossini/Documents/1.Projects/arxiv_openai_assistant/venv/lib/python3.10/site-packages/dotenv/main.py", line 283, in _is_interactive
main = __import__('__main__', None, None, fromlist=['__file__'])
ModuleNotFoundError: No module named '__main__'
Seems to be the same problem with dotenv that @arian81 encountered comment
yes confirm the same that @arian81 crashing when from dotenv import load_dotenv
is used
today, the issues persist with the versions gradio==4.14.0 python-dotenv==1.0.0
Changes detected in: ******.py
Reloading ui failed with the following exception:
Traceback (most recent call last):
File "/Users/cesar.reyes/miniconda3/lib/python3.11/site-packages/gradio/utils.py", line 207, in watchfn
module = importlib.import_module(reloader.watch_module_name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/cesar.reyes/miniconda3/lib/python3.11/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 940, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "****/ui.py", line 37, in <module>
load_dotenv()
File "/Users/cesar.reyes/miniconda3/lib/python3.11/site-packages/dotenv/main.py", line 336, in load_dotenv
dotenv_path = find_dotenv()
^^^^^^^^^^^^^
File "/Users/cesar.reyes/miniconda3/lib/python3.11/site-packages/dotenv/main.py", line 286, in find_dotenv
if usecwd or _is_interactive() or getattr(sys, 'frozen', False):
^^^^^^^^^^^^^^^^^
File "/Users/cesar.reyes/miniconda3/lib/python3.11/site-packages/dotenv/main.py", line 283, in _is_interactive
main = __import__('__main__', None, None, fromlist=['__file__'])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ModuleNotFoundError: No module named '__main__'
Code:
import os
import autogen
import gradio as gr
from dotenv import load_dotenv
load_dotenv()
with gr.Blocks() as demo:
chatbot = gr.Chatbot()
msg = gr.Textbox()
clear = gr.ClearButton([msg, chatbot])
def respond(message, chat_history):
# core.init_chat(message)
bot_message = "Hello Hello 1sqq2"
chat_history.append((message, bot_message))
print(chat_history)
return "", chat_history
msg.submit(respond, [msg, chatbot], [msg, chatbot])
demo.launch(server_port=8080)
Command:
gradio ui.py
Not sure if this is fixable from gradio or is a dotenv bug but it seems for now not using dotenv during development will solve the issue.
Potential fix in dotenv here: https://github.com/theskumar/python-dotenv/pulls
This has been fixed in python-dotenv version 1.0.1!
Adding a new env variable to a dotenv file is picked up by reload mode! Modifying a env variable is not. That's expected dotenv behavior, it doesn't overwrite env variables that are already set.
Thanks for your patience everyone.
Describe the bug
I’m using rye as a Python package management,here's the link of documention https://rye-up.com/ Auto-reloading by running
rye run gradio src/run.py
raised a ModuleNotFoundErrorHave you searched existing issues? 🔎
Reproduction
Screenshot
No response
Logs
System Info
Severity
Blocking usage of gradio