[objectstorage_objects_data_source.go] not list all objects if there are more than 1000 objects under the bucket.

[ShifengHuGit]

Code : internal/service/objectstorage/objectstorage_objects_data_source.go Issue, If you using the data source [objectstorage_objects_data_source] to fetch the list of Objects where stored under a bucket, it will not return the correct infor. if there are more than 1000 objects . and only the last List API results are given.
like, if 4900 pieces in a BucketA, there will be 5 APIs sent out due to the 1000 limitation,like 1000,1000,1000,1000,900. and only the last try result threw out. that means you only see the 900 objects.

I believe it was caused by the code at Line 190, the definition shouldn't be outside the for loop? Otherwise it will be overwrite to empty for each iteration.

Please have a check . thanks .

182 for { 183 request.RequestMetadata.RetryPolicy = tfresource.GetRetryPolicy(false, "object_storage") 184 185 response, err := s.Client.ListObjects(context.Background(), request) 186 if err != nil { 187 return err 188 } 189 190 s.Res = &oci_object_storage.ListObjects{Objects: []oci_objectstorage.ObjectSummary{}} 191 for , objectSummary := range response.Objects { 192 s.Res.Objects = append(s.Res.Objects, objectSummary) 193 } 194 195 if response.NextStartWith == nil || *response.NextStartWith == "" { 196 break 197 } 198 199 request.Start = response.NextStartWith 200 }

[tf-oci-pub]

Thank you for reporting the issue. We observed the affected resources are not provided in the description or it's incorrect. We request you to add it in issue description as mentioned in below format. Example: affected_resources = oci_core_instance , oci_core_instances

If it's not related to any particular resource then mention affected resource as terraform. Example: affected_resources = terraform

As this works through automation, request you to follow exact syntax.

[ShifengHuGit]

Thanks for your reply. Let me clarify more details of this issue.

The Problem was using the dataSource :oci_objectstorage_objects is not able to fetch all the objects list at all, and as per my observation , only the last API result was threw out.

tf file is simply, plz refer to below:

variable "tenancy_ocid" {
}

data "oci_objectstorage_objects" "objects1" {
    # Required, the bucket name is equal to your tenancy id
    bucket = var.tenancy_ocid
    namespace = "bling"
}

output "objects" {
   value = "Namespace : bling\r\nBucket : ${var.tenancy_ocid}\r\nThe total number of objects under this bucket: ${length(data.oci_objectstorage_objects.objects1.objects)}"
}

See my log file as well. There are more than thousand objects actually under that bucket, but currently not all files info. returned.

[root@arm-instance-01 bug]# [root@arm-instance-01 bug]# pwd /root/bug [root@arm-instance-01 bug]# [root@arm-instance-01 bug]# [root@arm-instance-01 bug]# ls ListAllObjects.tf [root@arm-instance-01 bug]# [root@arm-instance-01 bug]# cat ListAllObjects.tf variable "tenancy_ocid" { }

data "oci_objectstorage_objects" "objects1" {

Required, the bucket name is equal to your tenancy id

bucket = var.tenancy_ocid
namespace = "bling"

}

output "objects" { value = "Namespace : bling\r\nBucket : ${var.tenancy_ocid}\r\nThe total number of objects under this bucket: ${length(data.oci_objectstorage_objects.objects1.objects)}" } [root@arm-instance-01 bug]# [root@arm-instance-01 bug]# terraform -v Terraform v1.2.9 on linux_arm64

• provider registry.terraform.io/hashicorp/oci v4.123.0

Your version of Terraform is out of date! The latest version is 1.4.6. You can update by downloading from https://www.terraform.io/downloads.html [root@arm-instance-01 bug]# [root@arm-instance-01 bug]# terraform plan data.oci_objectstorage_objects.objects1: Reading... data.oci_objectstorage_objects.objects1: Read complete after 3s [id=ObjectStorageObjectsDataSource-4115693968]

Changes to Outputs:

• objects = <<-EOT Namespace : bling Bucket : ocid1.tenancy.oc1..aaaaaaaaro7aox2fclu4urtpgsbacnrmjv46e7n4fw3sc2wbq24l7dzf3kba The total number of objects under this bucket: 955 EOT

You can apply this plan to save these new output values to the Terraform state, without changing any real infrastructure.

───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

Note: You didn't use the -out option to save this plan, so Terraform can't guarantee to take exactly these actions if you run "terraform apply" now. [root@arm-instance-01 bug]#

meanwhile , tested by oci cli via Cloud shell, same bucket, same namespace to see how many objects under there.

 shifeng_hu@cloudshell:~ (us-ashburn-1)$ oci os object list -ns bling -bn ocid1.tenancy.oc1..aaaaaaaaro7aox2fclu4urtpgsbacnrmjv46e7n4fw3sc2wbq24l7dzf3kba --all | grep -w name| wc -l
**4955**
shifeng_hu@cloudshell:~ (us-ashburn-1)$ 

if enable the oci cli debug mode, there were 5 times API sent out, response like 1000,1000,1000,1000, 955 objects returned.

I did a little bit of debug , Code might be wrong , I put on the 1st post. plz have a check.

Plz reach me feel free. thanks