[Bug] Not able to set default backlog-quota retention policy , default retention policy and default TTL

[kunj-bosamia]

Search before asking

• [X] I searched in the issues and found nothing similar.

Version

pulsar helm chart version - 3.3.0 k8s version - v1.28.5+k3s1

Minimal reproduce step

When installing pulsar using pulsar helm chart added these values to broker.configData

    backlogQuotaDefaultRetentionPolicy: "consumer_backlog_eviction"
    backlogQuotaDefaultLimitBytes: "322122547200" 
    backlogQuotaDefaultLimitSecond: "1209600"
    defaultRetentionTimeInMinutes: "2880"
    defaultRetentionSizeInMB: "-1"
    ttlDurationDefaultInSeconds: "259200"
    topicLevelPoliciesEnabled: "true"

exec in pulsar broker pod do bin/pulsar-admin namespaces get-backlog-quotas public/functions

What did you expect to see?

the output should be -> destination_storage BacklogQuotaImpl(limit=322122547200, limitSize=322122547200, limitTime=1209600, policy=consumer_backlog_eviction)

What did you see instead?

I get no output

same goes for retention policy and ttl -> i get null as output , but it should return the default ttl and default return policy values as output respectively

Anything else?

No response

Are you willing to submit a PR?

• [ ] I'm willing to submit a PR!

[lhotari]

Please notice that 2.10.x is out of support: https://pulsar.apache.org/contribute/release-policy/#supported-versions

[lhotari]

The latest Pulsar Helm chart version is 3.3.0 , https://github.com/apache/pulsar-helm-chart/releases . All older Helm chart versions aren't supported.

[kunj-bosamia]

yep tried using helm chart version 3.3.0 which has a appVersion of 3.0.2 . Still giving the same result. I have edited the version in the description of this issue to 3.3.0.

[croemmich]

I've been trying to debug this for a couple hours as well. We're currently on appVersion 3.0.0 managed via helm.

Set a quota:

./bin/pulsar-admin namespaces set-backlog-quota test-tenant/test-ns --limit 3G --policy producer_request_hold

Get the quotas back

./bin/pulsar-admin namespaces get-backlog-quotas test-tenant/test-ns

Neither command fails or returns any errors, but the output of the get-backlog-quotas is empty.

19267 appears to be related

[croemmich]

Just updated to pulsar 3.0.3 and setting/getting backlog quotas is working now... 🤷

[kunj-bosamia]

Nope it doesnt work for pulsar 3.0.3 as well (note : p here is a alias for /bin/pulsar-admin )

pulsar@pulsar-toolset-0:/pulsar$ p namespaces create public/functions1
pulsar@pulsar-toolset-0:/pulsar$ p namespaces get-message-ttl public/functions1
null
pulsar@pulsar-toolset-0:/pulsar$ p namespaces get-backlog-quotas public/functions1
pulsar@pulsar-toolset-0:/pulsar$ p namespaces get-retention public/functions1
null
pulsar@pulsar-toolset-0:/pulsar$ 

The default values I set in for broker in values.yaml under broker.configData->

    backlogQuotaDefaultRetentionPolicy: "consumer_backlog_eviction"
    backlogQuotaDefaultLimitBytes: "322122547200" 
    backlogQuotaDefaultLimitSecond: "86400"
    defaultRetentionTimeInMinutes: "4320"
    defaultRetentionSizeInMB: "-1"
    ttlDurationDefaultInSeconds: "259200"
    topicLevelPoliciesEnabled: "true"

[lhotari]

I don't think that you can use the API to get the effective policy. It would be a useful feature, but that doesn't exist currently. Apache Pulsar is an open source projects, so contributions are welcome!

@kunjbosamiaFalkonry What type of problem does the lack of this feature (checking effective namespace / topic policies) cause to you?