wandb / wandb

The AI developer platform. Use Weights & Biases to train and fine-tune models, and manage models from experimentation to production.
https://wandb.ai
MIT License
8.94k stars 658 forks source link

[App]: lots of errors in the app #5367

Open davidgilbertson opened 1 year ago

davidgilbertson commented 1 year ago

Current Behavior

Over the past few hours I've gotten several errors. Sometimes individual panels not rendering (after having rendered for a long time, so the config is fine) sometimes the Sentry "Crash report" window, I get that a few times a day.

image

In the dev tools console, a quartet of GraphQL errors occurred with the latest "It looks like we're having issues." crash report .

GraphQL error: An internal error occurred. Please contact support.
GraphQL error: An internal error occurred. Please contact support.
GraphQL error: Error 1040 (08004): Too many connections
GraphQL error: An internal error occurred. Please contact support.

Sometimes if flicks up a 404 message (replacing the whole page) when I haven't even touched anything.

And today a new one about it not being able to display my custom panel (after having rendered for a long time, and refreshing fixed it, so the error is incorrect in only giving me one option: to lose my custom charts).

There was a problem rendering these panels.
Click this button to reset your workspace to the default layout.
This preserves runs, but custom charts will be lost.

image

And in the CLI network errors a few times a day (I have a rock solid network, usually coincides with some error in the web app so I assume WandB servers failing somewhere).

E.g. An internal error occurred. Please contact support

image

Expected Behavior

No response

Steps To Reproduce

No response

Screenshots

No response

Environment

OS: Windows 11

Browsers: Chrome

Version: 0.14

Additional Context

No response

CeHao1 commented 1 year ago

+1, I also faced this error many times.

rsanandres-wandb commented 1 year ago

Hello @CeHao1 and @davidgilbertson!

Are you still experiencing these errors? We had an outage yesterday causing slowness and errors in our SDK and UI. Feel free to let me know if you are still experiencing these errors and I would be happy to look into this for you.

CeHao1 commented 1 year ago

Now it is fixed. No problem any more. Thanks.

ZedRover commented 1 year ago

Now there still exists this problem for me. It happened 2 times today, and each one lasts about 10 minutes.

davidgilbertson commented 1 year ago

It's no longer happening for me, but I'll leave the issue open since @ZedRover appears to be having issues still.

rsanandres-wandb commented 1 year ago

@ZedRover

Could you provide me with the following so I can debug your issue:

exalate-issue-sync[bot] commented 1 year ago

Raphael San Andres commented: Hello, since we have not heard back from you we are going to close this request. If you would like to re-open the conversation, please let us know!

SimiPixel commented 1 year ago

I get this bug extremely frequently since a couple of days. About once every 1-2 minutes. It's very annoying. Any ideas?

leonmkim commented 1 year ago

Just to chip in, I have also encountered this issue recently in the past few days and it's still persisting beyond the big outage last week. On the main runs page of my project, I get the "There was a problem rendering these panels." message whenever I try to edit the filtering settings. And I get the "An internal error occurred. Please contact support" whenever opening up an individual run, which goes away after I refresh the page.