wandb / server

W&B Server is the self hosted version of Weights & Biases
MIT License
261 stars 21 forks source link

Error: Could not save view. #101

Closed jjerry-k closed 3 months ago

jjerry-k commented 1 year ago

Hi, I'm Jerry.

When I view my metrics on my project page, I got an error message like "Error: Could not save view." on the web page.

Can you tell me about the situation that can cause this error?

Is it about db connection?

luisbergua commented 1 year ago

Hi @jjerry-k, thanks for reporting this! Would you mind sharing with me a link to the page where this issue is raising? Also, have you tried other browsers, clearing cache/cookies and incognito window to see if the same error appears?

luisbergua commented 1 year ago

Hi @jjerry-k, I wanted to follow up here! If this issue is still appearing, would you mind sharing with me a link to the Workspace and try other browsers, clearing up cache/cookies and incognito window?

jjerry-k commented 1 year ago

Hi @luisbergua, Sorry, about the late reply. I tried several ways that using another browser and clearing cache/cookies. But, didn't repair. So, I want to get about wandb local web page's log. Could you explain where the log saved

luisbergua commented 1 year ago

Hi @jjerry-k, thanks for your answer! So this error could happen if the view exceeds a certain size threshold, could you explain me what's the size of the metrics? Also, are you running this by using wandb local? And could you send me a sudo code of how are you logging your metrics?

jjerry-k commented 1 year ago

Hi @luisbergua, thank you for your reply! I'm running wand by using docker! First, I'm logging 2029 items for each run with 40 epochs.

metrics = {
... 2029 items
}
wandb.log(metrics)

While looking at the log files, I got this message in mysql.log file.

2022-12-29T01:02:58.101968Z 22452 [Note] Aborted connection 22452 to db: 'wandb_local' user: 'wandb_local' host: '127.0.0.1' (Got an error reading communication packets)
2022-12-29T01:03:10.724457Z 22433 [Note] Aborted connection 22433 to db: 'wandb_local' user: 'wandb_local' host: '127.0.0.1' (Got a packet bigger than 'max_allowed_packet' bytes)

Did mysql's max_allowed_packet option change when 0.9.40 to 0.9.49 was released?

jjerry-k commented 1 year ago

I solved this issue.

There was no max_allowed_packet option in the mysql configuration file.

This is the content of the file. /etc/mysql/mysql.cond.d/mysqld.cnf

# Copyright (c) 2014, 2021, Oracle and/or its affiliates.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License, version 2.0,
# as published by the Free Software Foundation.
#
# This program is also distributed with certain software (including
# but not limited to OpenSSL) that is licensed under separate terms,
# as designated in a particular file or component or in included license
# documentation.  The authors of MySQL hereby grant you an additional
# permission to link the program and your derivative works with the
# separately licensed software that they have included with MySQL.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License, version 2.0, for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301 USA

#
# The MySQL  Server configuration file.
#
# For explanations see
# http://dev.mysql.com/doc/mysql/en/server-system-variables.html

[mysqld]
pid-file        = /var/run/mysqld/mysqld.pid
socket          = /var/run/mysqld/mysqld.sock
datadir         = /vol/mysql
#log-error      = /var/log/mysql/error.log
# By default we only accept connections from localhost
#bind-address   = 127.0.0.1
# Disabling symbolic-links is recommended to prevent assorted security risks
symbolic-links=0
max_connections = 500
max_prepared_stmt_count = 65536

# I added this line
max_allowed_packet      = 16M
luisbergua commented 1 year ago

Hi @jjerry-k, thanks a lot for sharing this! You can check the changes in the different releases here, I cannot see any changes regarding max_allowed_packet. Let me know if I can help you in any other way!

jjerry-k commented 1 year ago

Please close this issue!