Azure Blob Storage managed identity

[Steve887]

I'm deploying Jaeger to an Azure Container App and want to use Blob storage to persist traces. I think I have the config right as per the docs but I want to use Managed Identity to connect. With the config below it appears like the plugin doesn't start as I get the following in the console logs:

2023-09-19T04:26:26.573629757Z 2023-09-19T04:26:26.572Z [DEBUG] starting plugin: path=/var/jaeger/jaeger-objectstorage args=["/var/jaeger/jaeger-objectstorage", "--config", "/var/jaeger/jaeger.yaml"]
2023-09-19T04:26:26.602759128Z 2023-09-19T04:26:26.602Z [DEBUG] plugin started: path=/var/jaeger/jaeger-objectstorage pid=12
2023-09-19T04:26:26.602770199Z 2023-09-19T04:26:26.602Z [DEBUG] waiting for RPC address: plugin=/var/jaeger/jaeger-objectstorage
2023-09-19T04:26:28.044038971Z 2023-09-19T04:26:28.043Z [DEBUG] jaeger-objectstorage: level=warn ts=2023-09-19T04:26:28.043705418Z caller=experimental.go:20 msg="experimental feature in use" feature="In-memory (FIFO) cache"
2023-09-19T04:26:28.044055733Z 2023-09-19T04:26:28.043Z [DEBUG] jaeger-objectstorage: level=warn ts=2023-09-19T04:26:28.043817477Z caller=experimental.go:20 msg="experimental feature in use" feature="Azure Blob Storage"
2023-09-19T04:27:26.603084267Z {"level":"fatal","ts":1695097646.6029377,"caller":"./main.go:82","msg":"Failed to init storage factory","error":"grpc-plugin builder failed to create a store: error attempting to connect to plugin rpc client: timeout while waiting for plugin to start","stacktrace":"main.main.func1\n\t./main.go:82\ngithub.com/spf13/cobra.(*Command).execute\n\tgithub.com/spf13/cobra@v1.7.0/command.go:940\ngithub.com/spf13/cobra.(*Command).ExecuteC\n\tgithub.com/spf13/cobra@v1.7.0/command.go:1068\ngithub.com/spf13/cobra.(*Command).Execute\n\tgithub.com/spf13/cobra@v1.7.0/command.go:992\nmain.main\n\t./main.go:155\nruntime.main\n\truntime/proc.go:267"}

I can do this in the latest version of Loki, is it supported in this plugin? Or does the Loki reference for config need to be updated?

My config:

schema_config:
  configs:
    - from: 2020-07-01
      store: boltdb-shipper
      object_store: azure
      schema: v11  
      index:
        prefix: index_
        period: 24h
      row_shards: 32

storage_config:
  boltdb_shipper:
    active_index_directory: /var/jaeger/index
    shared_store: azure
    cache_location: /var/jaeger/boltdb-cache
  azure:
    use_managed_identity: true
    account_name: ${STORAGE_ACCOUNT_NAME}
    container_name: ${LOGS_CONTAINER_NAME}

compactor:
  working_directory: /var/jaeger/boltdb-shipper-compactor
  shared_store: azure

limits_config:
  enforce_metric_name: false
  reject_old_samples: true
  reject_old_samples_max_age: 168h

memberlist:
  abort_if_cluster_join_fails: false

  max_join_backoff: 1m
  max_join_retries: 10
  min_join_backoff: 1s

distributor:
  ring:
    kvstore:
      store: memberlist

ingester:
  lifecycler:
    ring:
      kvstore:
        store: memberlist
      replication_factor: 1
    final_sleep: 0s
  chunk_idle_period: 5m
  chunk_retain_period: 30s

[muhammadn]

@Steve887 What version of loki did you tried and work? Just need more information.

[muhammadn]

@Steve887 We will be releasing a new version soon but with a breaking change to how we store the traces.

We will update it here:

https://community.flitnetics.com/t/upcoming-update-of-jaeger-objectstorage/9

[muhammadn]

@Steve887 I've never used managed identity but i just want to update that we've at v3.0.0. We are no longer using loki as the backend interface to the storage(s) and using tempo instead. You can use Jaeger UI as per normal to get traces from AzureBlob.

Please try v3.0.0 out and tell me how it goes.

Closing for now. Do comment if you feel i'm missing something.