influxdata / influxdb

Scalable datastore for metrics, events, and real-time analytics
https://influxdata.com
Apache License 2.0
28.59k stars 3.54k forks source link

Getting permission denied error when trying to write to influx #23954

Open bawilc01 opened 1 year ago

bawilc01 commented 1 year ago

I am very new to influxdb and I am struggling to write sample data to a dev influxdb via python. I have a retention policy set to INF with replication factor 2. My code is written in python and loops through a list of pandas of items that formats each line item as a point to be written to the db using line protocol formatting.

I keep getting the following error when attempting to write to influxdb:

influxdb.exceptions.InfluxDBServerError: {"error":"write failed: mkdir /data/database_name: permission denied"}

Sample code:

df['tag_id1'] = df['tag_id1'].astype("string")
df['tag_id2'] = df['tag_id2'].astype("string")
df['tag_id3'] = df['tag_id3'].astype("string")
df['field_id1'] = df['field_id1'].astype("float")
df['field_id2'] = df['field_id2'].astype("float")

empty_list = []

for idx, row in df.iterrows():
    tag_id1 = df['tag_id1'].astype("string")
    tag_id2 = df['tag_id2'].astype("string")
    tag_id3 = df['tag_id3'].astype("string")
    tag_id3 = tag_id3.replace(" ", "\\ ") -- this handles a space in a datetime object being written as a string; do not want these to be indexed timestamps
    field_id1 = df['field_id1'].astype("float")
    field_id2= df['field_id2'].astype("float")
    measurement = config['DATABASE']['Measurement']
    write_time = dt.utcnow()
    utc_unix_timestamp = calendar.timegm(write_time.utctimetuple())

    empty_list.append(f'{measurement},tag_id1="{tag_id1}",tag_id2="{tag_id2}",tag_id3="{tag_id3}" field_id1={field_id1},field_id2={field_id2} {utc_unix_timestamp}')

  client = InfluxDBClient(host=config['SERVER']['Hostname'], 
                      port=config['SERVER']['Port'], 
                      username=config['LOGIN']['Username1'], 
                      password=config['LOGIN']['Password1'],
                      database=config['DATABASE']['Database'],
                      ssl=True, 
                      verify_ssl=False)

  for x in empty_list: 
      client.write_points(
                      x, 
                      time_precision='n', 
                      database=config['DATABASE']['Database'], 
                      retention_policy=config['DATABASE']['Retention_Policy'], 
                      protocol=u'line', 
                      consistency='all')

I think my spacing is correct for the line protocol formatting. On my end, it looks like the influx client is attempting to read, then write each point, but is getting stuck on something and denying me permission. I have also seen partial write errors and also what looks like my tags being recognized, but not my fields. Not sure where to go look next but I'm struggling to find a solution to get these points written to the influxdb. I'm trying to keep this line protocol format because when using json protocol and writing directly from pandas gives me python deprecation warnings. Thanks for any help.

jeffreyssmith2nd commented 1 year ago

influxdb.exceptions.InfluxDBServerError: {"error":"write failed: mkdir /data/database_name: permission denied"}

This looks like an issue from the InfluxDB server in that it is trying to create the directory to store your database in and failing. Make sure that the user you are running influxdb as has the correct permissions to read/write to that directory.

bawilc01 commented 1 year ago

Hi @jeffreyssmith2nd. I'm not sure how to check this. I'm able to create in the chronograf UI but I'm not sure how to check user write permissions when using influx. Any info on where to look would be appreciated. Thanks!

jeffreyssmith2nd commented 1 year ago

Sorry, I meant the user at the operating system level. write failed: mkdir /data/database_name: permission denied is the key part. However you are running influxdb does not have access to write to /data/database_name (which probably means it doesn't have access to /data). It looks like you might be on linux based on the path, so you will need to properly chown and chmod that directory to make sure it is writeable.

bawilc01 commented 1 year ago

Apologies for the delay @jeffreyssmith2nd , I've been out due to illness. I've looked for this on my machine (a Mac) and I do not see it, even when typing ls -la. Where is this located? This is all completely new to me and I don't quite understand a lot about the infrastructure. Also, I've had a few instances where I've noticed some incorrect formatting on my part. After fixing it, sometimes this error goes away and I question if it's always a permissions issue or if influx doesn't like the format of what I'm sending. And if so, that makes resolving this very difficult for me since the error being given and the actual issue don't appear related.