search

teamdigitale / anpr-dashboard-server

Service to expose data about the migration status of Italian municipalities to the National Registry
GNU Affero General Public License v3.0
2 stars 4 forks source link

Lock error #27

Closed LucaPrete closed 4 years ago

LucaPrete commented 4 years ago

The application keeps a lock on the database for its entire life-span. This blocks writes on the database.

For example, every time a user tries to update a municipality takeover, the following error is returned:

jquery.min.js:5 PUT https://dashboard.anpr.it/comune/update 500
send @ jquery.min.js:5
ajax @ jquery.min.js:5
(anonymous) @ subentro:519
dispatch @ jquery.min.js:4
r.handle @ jquery.min.js:4

To reproduce

  1. you should be a user authorized to update municipalities takeovers
  2. go to https://dashboard.anpr.it/subentro
  3. look for a municipality, for example Pianopoli
  4. insert takeover dates, for example 19/05/2020 for all the three dates
  5. confirm (looking at the browser console)

At server side, the following error is returned:

[GIN] 2020/05/04 - 09:04:17 | 500 |  5.714830378s |   134.19.174.87 | PUT      "/comune/update"
2020/05/04 09:04:17 [Recovery] 2020/05/04 - 09:04:17 panic recovered:
database is locked
/usr/local/go/src/runtime/panic.go:513 (0x431ff8)
/go/src/github.com/teamdigitale/anpr-dashboard-server/sqlite/sqlite3_manager.go:484 (0x7dbf7a)
/src/server/storage.go:484 (0xac2545)
/src/server/goospel.go:232 (0xacb8e3)
/src/server/urlmanager.go:267 (0xaca31e)
/go/src/github.com/gin-gonic/gin/context.go:156 (0x9d7de9)
/go/src/github.com/gin-contrib/logger/logger.go:50 (0xa05352)
/go/src/github.com/gin-gonic/gin/context.go:156 (0x9d7de9)
/go/src/github.com/gin-gonic/gin/recovery.go:83 (0x9ebba9)
/go/src/github.com/gin-gonic/gin/context.go:156 (0x9d7de9)
/go/src/github.com/gin-contrib/sessions/sessions.go:52 (0x9f50a3)
/go/src/github.com/gin-gonic/gin/context.go:156 (0x9d7de9)
/go/src/github.com/gin-gonic/gin/recovery.go:83 (0x9ebba9)
/go/src/github.com/gin-gonic/gin/context.go:156 (0x9d7de9)
/go/src/github.com/gin-gonic/gin/logger.go:241 (0x9ead60)
/go/src/github.com/gin-gonic/gin/context.go:156 (0x9d7de9)
/go/src/github.com/gin-gonic/gin/gin.go:409 (0x9e1ebe)
/go/src/github.com/gin-gonic/gin/gin.go:367 (0x9e157c)
/usr/local/go/src/net/http/server.go:2741 (0x76505a)
/usr/local/go/src/net/http/server.go:1847 (0x7612a5)
/usr/local/go/src/runtime/asm_amd64.s:1333 (0x45f2b0)

This is also true for DB updates from the sqlite CLI.

The error disappears as soon as the application is turned off. This is a known workaround to manually update data.

LucaPrete commented 4 years ago

Turns out it's a known issue with azurefires, which uses cifs 3 to expose the fs. There's an issue opened here

Meanwhile, we've moved the db from an azurefile-based PVC to an azuredisk-based PVC, which is a WRO block storage. The only downside of this is that multiple containers cannot read from the same storage, so rolling updates are not possible. This causes downtime for users while the container gets updated.

We'll close the issue for now, but we'll keep investigating with Microsoft on a possible solution.