search

jacobalberty / unifi-docker

Unifi Docker files
MIT License
2.16k stars 458 forks source link

Unifi-docker doesn't create backup #722

Closed JacopoMolteni closed 7 months ago

JacopoMolteni commented 9 months ago

Operating system

AlmaLinux 9

UniFi Tag

latest

Docker run

version: '2.3' services: mongo: image: mongo:3.6 container_name: ${COMPOSE_PROJECT_NAME}_mongo networks: - unifi restart: always volumes: - db:/data/db - dbcfg:/data/configdb controller: image: "jacobalberty/unifi:${TAG:-latest}" container_name: ${COMPOSE_PROJECT_NAME}_controller depends_on: - mongo init: true networks: - unifi restart: always volumes: - dir:/unifi - data:/unifi/data - log:/unifi/log - cert:/unifi/cert - init:/unifi/init.d - run:/var/run/unifi # Mount local folder for backups and autobackups - ./backup:/unifi/data/backup user: unifi sysctls: net.ipv4.ip_unprivileged_port_start: 0 environment: DB_URI: mongodb://mongo/unifi STATDB_URI: mongodb://mongo/unifi_stat DB_NAME: unifi ports: - "3478:3478/udp" # STUN - "6789:6789/tcp" # Speed test - "8080:8080/tcp" # Device/ controller comm. - "8443:8443/tcp" # Controller GUI/API as seen in a web browser - "8880:8880/tcp" # HTTP portal redirection - "8843:8843/tcp" # HTTPS portal redirection - "10001:10001/udp" # AP discovery logs: image: bash container_name: ${COMPOSE_PROJECT_NAME}_logs depends_on: - controller command: bash -c 'tail -F /unifi/log/*.log' restart: always volumes: - log:/unifi/log volumes: db: dbcfg: data: log: cert: init: dir: run: networks: unifi:

Bug description

If I try to export a backup the application hangs with a notification that says "fetching".

Steps to reproduce

Run the docker image using Portainer.io and try to make a backup.

Relevant log output

No response

JacopoMolteni commented 9 months ago

I found a guide explaining that the issue may be solved changing the owner of the folder backup to unifi 999:999, but on my system that user id and that group id are already assigned to the user systemd-coredump and the group input

vinyli85 commented 9 months ago

Same problem here. Webpage stays with message "fetching" and no download of file happens but a file is generated in the backup folder.

Edit Autobackup is working as intended.

TD-Community commented 9 months ago

I have the same issue for a while now. It seems started after upgrade to the lastest Unifi version 8.0.28. (I'm running docker on Synology NAS)

Looking around and checking user permissions on host (synology) it seems ok.

One strange thing is that the work "autobackup" is never mentioned in all of the logs. I have setup logging level debug for all items and checked "debug log" also.

I would expect that when starting the controller an autobackup file would be created automatically. It does not.

Manually starting download backup (settings only) shows popup "Fetching..." and does not close or give any error. When seaching for "autobackup" as text in all the logs gives zero results.

What I do see is this in server.log

[2024-02-14 08:38:14,787] <backup> INFO  system - [server backup] started
[2024-02-14 08:38:14,799] <backup> INFO  system - using 8.793446487482637% memory before backup
[2024-02-14 08:38:14,881] <backup> INFO  db     - Connecting to mongodb://localhost:27117
[2024-02-14 08:38:14,954] <backup> DEBUG db     - export ...heatmap
[2024-02-14 08:38:14,991] <backup> DEBUG db     - export ...wireguard_user
[2024-02-14 08:38:14,997] <backup> DEBUG db     - export ...voucher
[2024-02-14 08:38:15,003] <backup> DEBUG db     - export ...networkconf
[2024-02-14 08:38:15,012] <backup> DEBUG db     - export ...rogueknown
[2024-02-14 08:38:15,016] <backup> DEBUG db     - export ...traffic_rule
[2024-02-14 08:38:15,020] <backup> DEBUG db     - export ...apgroup
[2024-02-14 08:38:15,025] <backup> DEBUG db     - export ...pingtest
[2024-02-14 08:38:15,062] <backup> DEBUG db     - export ...setting
[2024-02-14 08:38:15,079] <backup> DEBUG db     - export ...notification
[2024-02-14 08:38:15,084] <backup> DEBUG db     - export ...teleport_token
[2024-02-14 08:38:15,088] <backup> DEBUG db     - export ...threat_log_view
[2024-02-14 08:38:15,095] <backup> DEBUG db     - export ...diagnostics_config
[2024-02-14 08:38:15,099] <backup> DEBUG db     - export ...payment
[2024-02-14 08:38:15,103] <backup> DEBUG db     - export ...alert_setting
[2024-02-14 08:38:15,107] <backup> DEBUG db     - export ...tag
[2024-02-14 08:38:15,112] <backup> DEBUG db     - export ...spatialrecord
[2024-02-14 08:38:15,115] <backup> DEBUG db     - export ...dashboard
[2024-02-14 08:38:15,120] <backup> DEBUG db     - export ...verification
[2024-02-14 08:38:15,125] <backup> DEBUG db     - export ...systemevent
[2024-02-14 08:38:15,129] <backup> DEBUG db     - export ...traffic_route
[2024-02-14 08:38:15,133] <backup> DEBUG db     - export ...dpiapp
[2024-02-14 08:38:15,138] <backup> DEBUG db     - export ...virtualdevice
[2024-02-14 08:38:15,142] <backup> DEBUG db     - export ...appstate_log
[2024-02-14 08:38:15,146] <backup> DEBUG db     - export ...firewallgroup
[2024-02-14 08:38:15,150] <backup> DEBUG db     - export ...hotspotop
[2024-02-14 08:38:15,155] <backup> DEBUG db     - export ...portforward
[2024-02-14 08:38:15,159] <backup> DEBUG db     - export ...dhcpoption
[2024-02-14 08:38:15,163] <backup> DEBUG db     - export ...task
[2024-02-14 08:38:15,167] <backup> DEBUG db     - export ...usergroup
[2024-02-14 08:38:15,172] <backup> DEBUG db     - export ...guest
[2024-02-14 08:38:15,176] <backup> DEBUG db     - export ...device
[2024-02-14 08:38:15,180] <backup> DEBUG db     - export ...doh_servers
[2024-02-14 08:38:15,184] <backup> DEBUG db     - export ...admin_activity_log
[2024-02-14 08:38:15,188] <backup> DEBUG db     - export ...admin
[2024-02-14 08:38:15,193] <backup> DEBUG db     - export ...magic_site_to_site_vpn_cloud_config
[2024-02-14 08:38:15,198] <backup> DEBUG db     - export ...privilege
[2024-02-14 08:38:15,203] <backup> DEBUG db     - export ...radiusprofile
[2024-02-14 08:38:15,209] <backup> DEBUG db     - export ...channelplan
[2024-02-14 08:38:15,213] <backup> DEBUG db     - export ...scheduletask
[2024-02-14 08:38:15,219] <backup> DEBUG db     - export ...routing
[2024-02-14 08:38:15,223] <backup> DEBUG db     - export ...trigger_log
[2024-02-14 08:38:15,244] <backup> DEBUG db     - export ...portconf
[2024-02-14 08:38:15,249] <backup> DEBUG db     - export ...alert
[2024-02-14 08:38:15,254] <backup> DEBUG db     - export ...wifi_connection_event
[2024-02-14 08:38:15,259] <backup> DEBUG db     - export ...featuremigration
[2024-02-14 08:38:15,262] <backup> DEBUG db     - export ...firewallrule
[2024-02-14 08:38:15,266] <backup> DEBUG db     - export ...wlangroup
[2024-02-14 08:38:15,271] <backup> DEBUG db     - export ...dpi
[2024-02-14 08:38:15,274] <backup> DEBUG db     - export ...map
[2024-02-14 08:38:15,278] <backup> DEBUG db     - export ...stat
[2024-02-14 08:38:15,283] <backup> DEBUG db     - export ...wifiman_feedback
[2024-02-14 08:38:15,287] <backup> DEBUG db     - export ...portalfile
[2024-02-14 08:38:15,292] <backup> DEBUG db     - export ...wlanconf
[2024-02-14 08:38:15,299] <backup> DEBUG db     - export ...config_meta
[2024-02-14 08:38:15,303] <backup> DEBUG db     - export ...uisettings_backup
[2024-02-14 08:38:15,307] <backup> DEBUG db     - export ...teleport_client
[2024-02-14 08:38:15,311] <backup> DEBUG db     - export ...heatmappoint
[2024-02-14 08:38:15,315] <backup> DEBUG db     - export ...downtime
[2024-02-14 08:38:15,319] <backup> DEBUG db     - export ...site
[2024-02-14 08:38:15,325] <backup> DEBUG db     - export ...ssooauthtoken
[2024-02-14 08:38:15,329] <backup> DEBUG db     - export ...hotspot2conf
[2024-02-14 08:38:15,332] <backup> DEBUG db     - export ...dynamicdns
[2024-02-14 08:38:15,336] <backup> DEBUG db     - export ...hotspotpackage
[2024-02-14 08:38:15,340] <backup> DEBUG db     - export ...user
[2024-02-14 08:38:15,345] <backup> DEBUG db     - export ...wall
[2024-02-14 08:38:15,350] <backup> DEBUG db     - export ...account
[2024-02-14 08:38:15,354] <backup> DEBUG db     - export ...dpigroup
[2024-02-14 08:38:15,360] <backup> DEBUG db     - Config export completed
[2024-02-14 08:38:15,361] <backup> DEBUG db     - export ...crashlog
[2024-02-14 08:38:15,365] <backup> DEBUG db     - export ...rogue
[2024-02-14 08:38:15,370] <backup> DEBUG db     - export ...alarm
[2024-02-14 08:38:15,376] <backup> DEBUG db     - export ...event
[2024-02-14 08:38:15,381] <backup> DEBUG db     - export ...ipsalert
[2024-02-14 08:38:15,420] <tracer> DEBUG tasks  - Started task[tracer]
[2024-02-14 08:38:15,421] <tracer> DEBUG tasks  - Finished task[tracer] in 1 ms
[2024-02-14 08:38:15,432] <backup> INFO  db     - Connecting to mongodb://localhost:27117
[2024-02-14 08:38:15,457] <backup> DEBUG db     - export ...doh_servers
[2024-02-14 08:38:15,458] <backup> DEBUG db     - export ...wifiman_feedback
[2024-02-14 08:38:15,458] <backup> DEBUG db     - export ...traffic_route
[2024-02-14 08:38:15,458] <backup> DEBUG db     - export ...wireguard_user
[2024-02-14 08:38:15,458] <backup> DEBUG db     - export ...appstate_log
[2024-02-14 08:38:15,459] <backup> DEBUG db     - export ...admin_activity_log
[2024-02-14 08:38:15,459] <backup> DEBUG db     - export ...config_meta
[2024-02-14 08:38:15,459] <backup> DEBUG db     - export ...magic_site_to_site_vpn_cloud_config
[2024-02-14 08:38:15,459] <backup> DEBUG db     - export ...traffic_rule
[2024-02-14 08:38:15,460] <backup> DEBUG db     - export ...teleport_client
[2024-02-14 08:38:15,460] <backup> DEBUG db     - export ...apgroup
[2024-02-14 08:38:15,460] <backup> DEBUG db     - export ...pingtest
[2024-02-14 08:38:15,460] <backup> DEBUG db     - export ...downtime
[2024-02-14 08:38:15,461] <backup> DEBUG db     - export ...notification
[2024-02-14 08:38:15,461] <backup> DEBUG db     - export ...trigger_log
[2024-02-14 08:38:15,461] <backup> DEBUG db     - export ...teleport_token
[2024-02-14 08:38:15,461] <backup> DEBUG db     - export ...threat_log_view
[2024-02-14 08:38:15,462] <backup> DEBUG db     - export ...alert
[2024-02-14 08:38:15,462] <backup> DEBUG db     - export ...wifi_connection_event
[2024-02-14 08:38:15,462] <backup> DEBUG db     - export ...diagnostics_config
[2024-02-14 08:38:15,462] <backup> DEBUG db     - export ...alert_setting
[2024-02-14 08:38:15,462] <backup> DEBUG db     - export ...dpi
[2024-02-14 08:38:15,463] <backup> DEBUG db     - export ...systemevent
[2024-02-14 08:38:15,463] <backup> DEBUG db     - Config export completed
[2024-02-14 08:38:15,463] <backup> DEBUG db     - export ...stat_5minutes
[2024-02-14 08:38:15,463] <backup> DEBUG db     - export ...stat_life
[2024-02-14 08:38:15,464] <backup> DEBUG db     - export ...stat_hourly
[2024-02-14 08:38:15,464] <backup> DEBUG db     - export ...stat_minute
[2024-02-14 08:38:15,465] <backup> DEBUG db     - export ...stat_monthly
[2024-02-14 08:38:15,465] <backup> DEBUG db     - export ...stat_archive
[2024-02-14 08:38:15,465] <backup> DEBUG db     - export ...stat_dpi
[2024-02-14 08:38:15,465] <backup> DEBUG db     - export ...stat_daily
[2024-02-14 08:38:15,687] <backup> DEBUG system - zipDir: timestamp
[2024-02-14 08:38:15,703] <backup> DEBUG system - zipDir: version
[2024-02-14 08:38:15,705] <backup> DEBUG system - zipDir: format
[2024-02-14 08:38:15,706] <backup> DEBUG system - zipDir: sites/
[2024-02-14 08:38:15,707] <backup> DEBUG system - zipDir: system.properties
[2024-02-14 08:38:15,709] <backup> DEBUG system - zipDir: db_stat.gz
[2024-02-14 08:38:15,710] <backup> DEBUG system - zipDir: db.gz
[2024-02-14 08:38:15,792] <backup> INFO  system - using 9.350554337027638% memory after backup
[2024-02-14 08:38:16,760] <backup> INFO  system - using 7.350733858540377% memory after cleanup
[2024-02-14 08:38:16,761] <backup> INFO  system - [server backup][BACKUP] end
[2024-02-14 08:38:16,769] <backup> INFO  event  - [event] Backup has been created

NO warning/error and even states that backup was created. No file was created in autobackup folder when looking in the folder.

I tried everything. Setting user permissions to unifi, created unifi user in synology and used the group and user ID's in the container variables. Used GUID and UID to 999:999 when creating the container etc etc.

Never any mention of autobackup in logs.

While running the controller I removed the autobackup folder and started a manual backup. Nothing in the log. I would expect an error it can not find the folder but the server.log gives the same log result that the backup was performed ok. Clearly it could not.

All those tests were done on clear new containers and did tests on different users and user permissions.

So what is happening here?

mongod.log

2024-02-14T08:35:36.873+0000 I CONTROL  [initandlisten] MongoDB starting : pid=82 port=27117 dbpath=/usr/lib/unifi/data/db 64-bit host=Unifi-Controller-New
2024-02-14T08:35:36.873+0000 I CONTROL  [initandlisten] db version v3.6.8
2024-02-14T08:35:36.873+0000 I CONTROL  [initandlisten] git version: 8e540c0b6db93ce994cc548f000900bdc740f80a
2024-02-14T08:35:36.873+0000 I CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.1.1f  31 Mar 2020
2024-02-14T08:35:36.874+0000 I CONTROL  [initandlisten] allocator: tcmalloc
2024-02-14T08:35:36.874+0000 I CONTROL  [initandlisten] modules: none
2024-02-14T08:35:36.874+0000 I CONTROL  [initandlisten] build environment:
2024-02-14T08:35:36.874+0000 I CONTROL  [initandlisten]     distarch: aarch64
2024-02-14T08:35:36.874+0000 I CONTROL  [initandlisten]     target_arch: aarch64
2024-02-14T08:35:36.874+0000 I CONTROL  [initandlisten] options: { net: { bindIp: "127.0.0.1", port: 27117, unixDomainSocket: { pathPrefix: "/usr/lib/unifi/run" } }, processManagement: { pidFilePath: "/usr/lib/unifi/run/mongod.pid" }, storage: { dbPath: "/usr/lib/unifi/data/db", wiredTiger: { engineConfig: { configString: "cache_size=256M" } } }, systemLog: { destination: "file", logAppend: true, logRotate: "reopen", path: "/usr/lib/unifi/logs/mongod.log" } }
2024-02-14T08:35:36.901+0000 I -        [initandlisten] Detected data files in /usr/lib/unifi/data/db created by the 'wiredTiger' storage engine, so setting the active storage engine to 'wiredTiger'.
2024-02-14T08:35:36.901+0000 I STORAGE  [initandlisten] 
2024-02-14T08:35:36.901+0000 I STORAGE  [initandlisten] ** WARNING: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine
2024-02-14T08:35:36.901+0000 I STORAGE  [initandlisten] **          See http://dochub.mongodb.org/core/prodnotes-filesystem
2024-02-14T08:35:36.901+0000 I STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=256M,session_max=20000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),cache_cursors=false,compatibility=(release="3.0",require_max="3.0"),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),statistics_log=(wait=0),verbose=(recovery_progress),cache_size=256M
2024-02-14T08:35:38.259+0000 I STORAGE  [initandlisten] WiredTiger message [1707899738:209645][82:0x7fb7323040], txn-recover: Main recovery loop: starting at 2/19840
2024-02-14T08:35:39.158+0000 I STORAGE  [initandlisten] WiredTiger message [1707899739:158877][82:0x7fb7323040], txn-recover: Recovering log 2 through 3
2024-02-14T08:35:39.673+0000 I STORAGE  [initandlisten] WiredTiger message [1707899739:673174][82:0x7fb7323040], txn-recover: Recovering log 3 through 3
2024-02-14T08:35:40.083+0000 I STORAGE  [initandlisten] WiredTiger message [1707899740:83463][82:0x7fb7323040], txn-recover: Set global recovery timestamp: 0
2024-02-14T08:35:41.865+0000 I CONTROL  [initandlisten] 
2024-02-14T08:35:41.865+0000 I CONTROL  [initandlisten] ** WARNING: Access control is not enabled for the database.
2024-02-14T08:35:41.865+0000 I CONTROL  [initandlisten] **          Read and write access to data and configuration is unrestricted.
2024-02-14T08:35:41.865+0000 I CONTROL  [initandlisten] 
2024-02-14T08:35:43.902+0000 I FTDC     [initandlisten] Initializing full-time diagnostic data capture with directory '/usr/lib/unifi/data/db/diagnostic.data'
2024-02-14T08:35:43.930+0000 I NETWORK  [initandlisten] waiting for connections on port 27117

server.log, startup

[2024-02-14 08:35:04,565] <launcher> INFO  launcher - Initializing on Linux(4.4.302+), jre17(17.0.9)
[2024-02-14 08:35:09,218] <launcher> INFO  tomcat - Adding basic REST API support during the startup
[2024-02-14 08:35:12,715] <launcher> INFO  system - Tomcat startup took 6419ms
[2024-02-14 08:35:12,801] <launcher> INFO  tomcat - Adding basic REST API support during the startup
[2024-02-14 08:35:34,314] <launcher> INFO  db     - Starting database service initialization...
[2024-02-14 08:35:34,317] <launcher> INFO  db     - waiting for db connection...
[2024-02-14 08:35:34,317] <db-server> INFO  db     - Checking if database needs to be shut down
[2024-02-14 08:35:35,938] <db-server> INFO  db     - Database was not running
[2024-02-14 08:35:35,939] <db-server> INFO  db     - Database configuration, dir=/usr/lib/unifi/data/db, port=27117
[2024-02-14 08:35:36,711] <db-server> INFO  db     - Starting database process...
[2024-02-14 08:35:36,757] <db-server> INFO  db     - 2024-02-14T08:35:36.757+0000 I STORAGE  [main] Engine custom option: cache_size=256M
[2024-02-14 08:35:36,820] <launcher> INFO  db     - Connecting to mongodb://localhost:27117
[2024-02-14 08:35:36,832] <launcher> INFO  db     - db connection established...
[2024-02-14 08:35:45,037] <launcher> INFO  db     - db connected (v3.6.8@mongodb://localhost:27117, journal enabled)
[2024-02-14 08:35:45,687] <launcher> INFO  db     - Database service initialized...
[2024-02-14 08:35:45,726] <launcher> INFO  system - ======================================================================
[2024-02-14 08:35:45,727] <launcher> INFO  system - UniFi 8.0.28 (build atag_8.0.28_24416 - release/release) is started
[2024-02-14 08:35:45,727] <launcher> INFO  system - Environment: UniFi-OS[false], UniFi-Cloud[false], UniFi-MongoService[false]
[2024-02-14 08:35:45,728] <launcher> INFO  system - ======================================================================
[2024-02-14 08:35:45,728] <launcher> INFO  system - BASE dir:/usr/lib/unifi
[2024-02-14 08:35:45,754] <launcher> INFO  system - Current System IP: 192.168.1.90
[2024-02-14 08:35:45,900] <launcher> INFO  system - Hostname: UnknownHost
[2024-02-14 08:35:45,901] <launcher> INFO  system - ubic.env: prod
github-actions[bot] commented 8 months ago

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

jslegers73 commented 7 months ago

This issue is stil there on the latest version.

drwtsn32x commented 7 months ago

I have never had an issue with autobackup working for me. I'm running on a Synology NAS as well.

On my system "Everyone" permissions were changed on the "backup" folder level to "Custom" (according to Synology File Manager) to grant most read and write privs.

If I ssh to the NAS and view the perms at that level, it is mode 777 on that directory and owned by root:root.

I have been running versions of this image for years on my NAS, not sure when those permissions were adjusted or by which image version.