InfluxDB on NFS Storage: Unable to Create New Token and Write Data

[Sanidhya30]

Hi All,

Recently, I switched my InfluxDB data directory from local storage to NFS storage and encountered an issue. Initially, I set up InfluxDB with local storage, created an API token, and everything worked fine. However, after migrating the data to NFS storage, I'm unable to create new tokens, and data writing to the InfluxDB bucket using the token does not seem to work. Below is my Docker Compose configuration. The user with UID 3000 has the necessary permissions for the NFS storage.

services:
  influxdb:
    image: influxdb:latest
    container_name: influxdb
    user: "3000:3000"
    network_mode: bridge
    volumes:
      # Mount for influxdb data directory and configuration
      # - /var/dockerData/Influxdb/InfluxdbData:/var/lib/influxdb2   # Local Storage
      - /var/dockerData/Influxdb/InfluxdbConfig:/etc/influxdb2
      - /mnt/truenas-share/Debian/Database/Influxdb/InfluxdbData:/var/lib/influxdb2    # nfs Storage
    ports:
      - 8086:8086
    environment:
      - INFLUXD_LOG_LEVEL=debug
    restart: unless-stopped

Here are the debug logs when I attempt to create a new token:

ts=2024-08-04T14:44:53.084912Z lvl=debug msg="urm find" log_id=0qoTCYZ0000 store=new took=0.044ms
ts=2024-08-04T14:44:53.085068Z lvl=debug msg="session find" log_id=0qoTCYZ0000 service=session took=0.216ms
ts=2024-08-04T14:44:53.085088Z lvl=debug msg="session renew" log_id=0qoTCYZ0000 service=session took=0.011ms
ts=2024-08-04T14:44:53.085100Z lvl=debug msg="user find by ID" log_id=0qoTCYZ0000 store=new took=0.008ms
ts=2024-08-04T14:44:53.085322Z lvl=debug msg="user find by ID" log_id=0qoTCYZ0000 store=new took=0.020ms
ts=2024-08-04T14:44:53.085349Z lvl=debug msg="user find by ID" log_id=0qoTCYZ0000 store=new took=0.009ms
ts=2024-08-04T14:44:53.085370Z lvl=debug msg="org find by ID" log_id=0qoTCYZ0000 store=new took=0.015ms
ts=2024-08-04T14:44:53.106467Z lvl=debug msg="failed to create authorization" log_id=0qoTCYZ0000 handler=authorization error="input/output error" took=21.131ms
ts=2024-08-04T14:44:53.106480Z lvl=error msg="api error encountered" log_id=0qoTCYZ0000 error="input/output error"
ts=2024-08-04T14:44:53.106594Z lvl=debug msg=Request log_id=0qoTCYZ0000 service=http method=POST host=192.168.1.63:8086 path=/api/v2/authorizations query= proto=HTTP/1.1 status_code=500 response_size=63 content_length=3379 referrer=http://192.168.1.63:8086/orgs/e18b40e982cda0d2/load-data/tokens remote=192.168.1.11:50868 user_agent=Chrome authenticated_id=0d7375c056484000 user_id=0d6e081cec55e000 took=21.779ms error="internal error" error_code="internal error" body="{\"orgID\":\"e18b40e982cda0d2\",\"description\":\"ss\",\"permissions\":[{\"action\":\"read\",\"resource\":{\"type\":\"annotations\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"write\",\"resource\":{\"type\":\"annotations\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"read\",\"resource\":{\"type\":\"authorizations\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"write\",\"resource\":{\"type\":\"authorizations\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"read\",\"resource\":{\"type\":\"buckets\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"write\",\"resource\":{\"type\":\"buckets\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"read\",\"resource\":{\"type\":\"checks\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"write\",\"resource\":{\"type\":\"checks\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"read\",\"resource\":{\"type\":\"dashboards\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"write\",\"resource\":{\"type\":\"dashboards\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"read\",\"resource\":{\"type\":\"dbrp\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"write\",\"resource\":{\"type\":\"dbrp\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"read\",\"resource\":{\"type\":\"documents\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"write\",\"resource\":{\"type\":\"documents\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"read\",\"resource\":{\"type\":\"labels\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"write\",\"resource\":{\"type\":\"labels\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"read\",\"resource\":{\"type\":\"notebooks\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"write\",\"resource\":{\"type\":\"notebooks\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"read\",\"resource\":{\"type\":\"notificationEndpoints\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"write\",\"resource\":{\"type\":\"notificationEndpoints\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"read\",\"resource\":{\"type\":\"notificationRules\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"write\",\"resource\":{\"type\":\"notificationRules\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"read\",\"resource\":{\"type\":\"orgs\",\"id\":\"e18b40e982cda0d2\"}},{\"action\":\"read\",\"resource\":{\"type\":\"remotes\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"write\",\"resource\":{\"type\":\"remotes\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"read\",\"resource\":{\"type\":\"replications\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"write\",\"resource\":{\"type\":\"replications\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"read\",\"resource\":{\"type\":\"scrapers\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"write\",\"resource\":{\"type\":\"scrapers\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"read\",\"resource\":{\"type\":\"secrets\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"write\",\"resource\":{\"type\":\"secrets\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"read\",\"resource\":{\"type\":\"sources\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"write\",\"resource\":{\"type\":\"sources\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"read\",\"resource\":{\"type\":\"tasks\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"write\",\"resource\":{\"type\":\"tasks\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"read\",\"resource\":{\"type\":\"telegrafs\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"write\",\"resource\":{\"type\":\"telegrafs\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"read\",\"resource\":{\"type\":\"users\",\"id\":\"0d6e081cec55e000\"}},{\"action\":\"write\",\"resource\":{\"type\":\"users\",\"id\":\"0d6e081cec55e000\"}},{\"action\":\"read\",\"resource\":{\"type\":\"variables\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"write\",\"resource\":{\"type\":\"variables\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"read\",\"resource\":{\"type\":\"views\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"write\",\"resource\":{\"type\":\"views\",\"orgID\":\"e18b40e982cda0d2\"}}]}"

Currently, I've reverted to using local storage, which works fine, but I prefer to use NFS storage due to space constraints. Any guidance or suggestions on what might be wrong or how to resolve this issue would be greatly appreciated. Thanks.