pulibrary / tiger-data-app

TigerData is a comprehensive set of data storage and management tools and services that provides storage capacity, reliability, functionality, and performance to meet the needs of a rapidly changing research landscape and to enable new opportunities for leveraging the power of institutional data.
Apache License 2.0
6 stars 0 forks source link

File Count and Quota amount used are not showing #759

Closed carolyncole closed 3 weeks ago

carolyncole commented 1 month ago

Expected behavior

On https://tigerdata-staging.princeton.edu/projects/34184/contents the file count and Quota amount used are not showing. Amount of Quota is showing.

The asset was created by Chuck with the script below (or something similar).

## Namespace 
asset.namespace.create :namespace /princeton/tigerdataNS/PENANS/ffcwsNS -all true :store dell-ps-1
## Collection
asset.create :name ffcws :member-store dell-ps-1 \
    :type application/arc-asset-collection \
    :namespace princeton/tigerdataNS/PENANS/ffcwsNS \
    :collection \
       -unique-name-index true \
       -cascade-contained-asset-index true \
       -contained-asset-index true \
       true \
    :owner < :domain princeton :user cpena > \
    :member-acl < \
       :actor -type user princeton:cpena \
       :access < \
          :asset ACCESS \
          :asset CREATE \
          :asset MODIFY \
          :asset DESTROY \
          :collection ACCESS \
          :collection EXECUTE \
          :collection CREATE \
          :collection MODIFY \
          :collection DESTROY \
          :asset-content ACCESS \
          :asset-content MODIFY \
          :asset-content EXECUTE \
          > \
        > \
     :member-acl < \
       :actor -type user princeton:siena \
       :access < \
          :asset ACCESS \
          :asset CREATE \
          :asset MODIFY \
          :asset DESTROY \
          :collection ACCESS \
          :collection EXECUTE \
          :collection CREATE \
          :collection MODIFY \
          :collection DESTROY \
          :asset-content ACCESS \
          :asset-content MODIFY \
          :asset-content EXECUTE \
          > \
       > \
       :member-acl < \
       :actor -type role pu-hpc-group:pena \
       :access < \
          :asset ACCESS \
          :collection ACCESS \
          :collection EXECUTE \
          :asset-content ACCESS \
          :asset-content EXECUTE \
          > \
       > \
        :quota < :allocation 4 TB \
            :on-overflow fail \
            :description "Project Quota" \
            > \
    :pid "path=/princeton/tigerdata/PENA" \
    :enqueue < :name ffcws-copy-to-ecs :queue tigerdata-copy-to-dell-ecs-1 :event content-create :event content-modify > \
    :meta < :tigerdata:project < :DataSponsor cpena :DataManager siena :DataUser aminerva, fr1510, md6347, ee2591 :Department PNI :ProjectID 10.60803/362n-6f70 :ProjectDirectory /tigerdata/PENA/ffcws :Title ffcws :Description Contains data and code for a publication related to differential methylation in the future of families study. > >
    ## Accumulators
    asset.collection.accumulator.add :id path=princeton/tigerdata/PENA/ffcws :accumulator < :name ffcws-count :type collection.asset.count > :accumulator < :name ffcws-size :type content.all.size > :accumulator < :name ffcws-size-by-store :type content.all.store.size > 

It is possible that we are not parsing the quota correctly, or creating our quotas in the same way as Chuck.

Additionally the file count should be showing, but is not

carolyncole commented 1 month ago

Looks like we are expecting ther to be a size accumulator, but we should be able to get the size out of the quota. And Chuck is not creating a size accumulator. And the quota is added during the collection create and we are creating the quota separately.

carolyncole commented 1 month ago

Turns out the user had no access to the accumulators. But it did surface that we have two numbers that currently exist in the system for Storage used, the active data (Current files) and residual data (Old versions, items that are soft deleted, ect.)

We want to show the user both numbers inthe format Storage Usage 30 MB / 10 GB active data: 5.29 MB residual data: 24.71 MB

This means the page will change from showing only active data to showing quota usage (active & residual from the quota), active data (from the size accumulator), and residual data (quota - size accululator)

See slack conversation