openOutpaint is spamming Automatic API regardless if its active

[vladmandic]

openOutpaint-webUI-extension is constantly polling Automatic API which can cause some issues on Automatic side as its not designed for that. This is done in app/js/index.js:checkAgain

And it does that regardless if openOutpaint tab is active or not

Even more, it uses expensive /sdapi/options API and it only looks at results on first connect, rest of them are simply used as checkConnection

Ask-1: Any kind of polling should be done only when tab is actually active & visible

Take a look at https://github.com/vladmandic/sd-extensions/blob/main/extensions/info-tab/javascript/info-tab.js
for simple example - my extension polls torch gpu memory stats every second only if tab is active/visible

Ask-2: Use / endpoint for simple is-alive checkConnection as that is responded by Gradio directly and doesnt need to be parsed by Automatic API parser

[zero01101]

moving to main repo as this is definitely more in its domain

[zero01101]

re: 1 - yep agreed, working on that right now; actual tab focus/blur is simple enough to implement so far but i'm kind of a moron and the gradio one is fighting with me ;) post-first-connection checks are also getting updated to GET /sdapi/v1/progress as that seems to be one of the lightest API calls and sure sounds like it's inherently ready to answer repeated demands for information

re: 2 - soooo yeah, we can't use a bare / endpoint as that'll return something as long as webUI is running regardless of --api flag :/ both the standalone self-hosted "website" and extension version rely on the API being exposed so we definitely gotta check something that confirms or denies that... ergo the /progress endpoint mentioned above :)

should have a testing branch push up in a few minutes with a basic tab focus/blur check at least, gradio to come after - if you just want to update your .gitmodules to point at the testing branch instead of main and restart webUI i believe that should get you the updates

ok maybe not in a few minutes as it seems to not be working lol

[vladmandic]

re: 1 - yeah, not as trivial as in pure-js, took me a bit to figure it out, that's why i've sent you the example re: 2 - you're right, progress api is a good choice

and no rush, its not critical. i'll check if its ready tomorrow and switch to testing branch if its still not in main.

[zero01101]

tab focus check/endpoint update/slight adjustment to how often it checks if connection is good is up on testing now :) moving to gradio tab check

[zero01101]

hmmmm ok so since it's the main app/submodule actually doing the connectivity/status checks against the API, my feeble attempts at stealing your example and making the extension tell the app/submodule to chill out effectively kill off the standalone app :| this one might be... uh... difficultish

[seijihariki]

Is window focus properly detected on iframes? If so, we can try using that.

[seijihariki]

Also has the benefit of solving the issue for the standalone app...

[zero01101]

well let's find out!

[vladmandic]

hmmmm ok so since it's the main app/submodule actually doing the connectivity/status checks against the API, my feeble attempts at stealing your example and making the extension tell the app/submodule to chill out effectively kill off the standalone app :| this one might be... uh... difficultish

for standalone app, you need to skip initInitial and go straight to initLoading on DOMContentLoaded. and you can do that if typeof gradioApp === 'undefined' since your standalone app doesnt run in gradio?

[seijihariki]

for standalone app, you need to skip initInitial and go straight to initLoading on DOMContentLoaded. and you can do that if typeof gradioApp === 'undefined' since your standalone app doesnt run in gradio?

gradioApp is probably never available in the context as we are running inside an iframe...

If window focus works, I think we are in the clear. We could even detect when the tab is clicked on and forcibly shift focus to the iframe.

[zero01101]

gradioApp is probably never available in the context as we are running inside an iframe...

precisely, i ran into that initially by stupidly putting @vladmandic 's gradio tab code in index.js which resulted in the same kind of contextless, undefined gradioApp() as you're having fun with in that enormous colab thread - trying to getElementsByTagName("iframe") in index.js:checkFocus() just returns the configuration page unfortunately :(

[seijihariki]

So, no dice with document.hasFocus()?

[zero01101]

not yet but that might be 100% my idiocy lol, still throwing garbage at the wall to see what sticks

[seijihariki]

It seems to work:

https://user-images.githubusercontent.com/7552751/211173386-d14fa768-8760-4230-90ef-9b7176b68e6b.mp4

[seijihariki]

I am using the window events:

[seijihariki]

With some patches to openoutpaint-ext:

Automatic focusing of the frame on tab enter is also doable:

https://user-images.githubusercontent.com/7552751/211173688-f4486542-776a-42b8-b19e-49e96b21668d.mp4

Though clicking outside the iframe unfocuses openOutpaint.

[zero01101]

sorry, had to step out for a bit; i was using document events at first but also moved to window events which didn't work at first like i was expecting, but it took me way too long to notice that you were also clicking inside the iframe to refocus in the first video and that made all the difference lol, it does seem to work and i still stand by my absolute 100% certainty that i'm an idiot :D

i'm sure you'll be just shocked to hear that today's the first i've heard of/used this whole mutationobserver thingy so yeah this is more fancy magic to me so far :| but yep, your openoutpaint-ext modification does seem to be pretty effective at popping focus back onto it in most eventualities :)

i'm gonna push my garbage to testing but i'm still obviously doing something WILDLY wrong - i have a checkFocus() function that literally stops working if i don't do the console.debug() at the bottom which should be interpreted as "i do not understand a tiny bit of what i'm doing" and i apologize

[edit]

literally stops working if i don't do the

i told you i'm an idiot -_-

[zero01101]

@vladmandic care to check if this is more to your liking? since we popped the submodule update out of main.py you'll probably have to fully shut down and restart webUI after updating the extension itself to pull in all the latest bits

[vladmandic]

@vladmandic care to check if this is more to your liking? since we popped the submodule update out of main.py you'll probably have to fully shut down and restart webUI after updating the extension itself to pull in all the latest bits

sure. main or testing branch?

[zero01101]

it's on main now, it seemed "safe" enough :)

[vladmandic]

i see progress polls every second while dreaming and veeeery infrequently otherwise - up to a point im not sure what even triggers them anymore (switching tab focus or even window focus does not trigger them)?

btw, you can monitor api requests on automatic side with python launch.pt --api --api-log, i've added that few days ago.

[zero01101]

yeah, that sounds about right - first launch will GET /sdapi/v1/options, and when a dream is processing we query /progress to get the ETA regularly, but if it's "idle" then you should only see a call to /progress while the tab (standalone or extension) is active/focused once a minute; refocusing the tab doesn't immediately instantiate a status check or anything, but you should see the check within a minute of the tab refocusing.

[vladmandic]

closing as resolved - thanks!