dmwm / das2go

Go implementation of Data Aggregation System (DAS) for CMS experiment
MIT License
2 stars 3 forks source link

Cannot see a site of a dataset #46

Closed michael-pitt closed 2 years ago

michael-pitt commented 2 years ago

Clicking on "Site" for the following dataset give me an error:

dataset: /EphemeralZeroBias0/Run2022C-PromptReco-v1/AOD

Thanks, Michael

vkuznet commented 2 years ago

Michael, it is possible that site information is not available. The Site query on web interface returns nothing. And, you can easily verify this dasgoclient CLI:

dasgoclient -query="site dataset=/EphemeralZeroBias0/Run2022C-PromptReco-v1/AOD" -verbose 1
DBSUrl:  https://cmsweb.cern.ch/dbs/prod/global/DBSReader
SitedbUrl:  https://cmsweb.cern.ch/sitedb/data/prod
CricUrl w/ site API:  https://cms-cric.cern.ch/api/cms/site/query
RucioUrl:  https://cms-rucio.cern.ch/
RucioAuthUrl:  https://cms-rucio-auth.cern.ch/auth/x509
### download dasmaps
DAS GET https://raw.githubusercontent.com/dmwm/DASMaps/master/js/das_maps_dbs_prod.js 328.834169ms
Load dasmaps /Users/vk/.dasmaps/das_maps_dbs_prod.js

site dataset=/EphemeralZeroBias0/Run2022C-PromptReco-v1/AOD

### unique true
rucio token valya-/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=valya/CN=443502/CN=Valentin Y Kuznetsov-unknown-4051e56f191a495b93140445d7a064cb <nil>
### selected services [combined:site4dataset_pct combined:site4dataset] [site.name site.name]
### selected urls map[]
### selected localApis [map[das_map:[map[das_key:site pattern:^T[0-3]_ rec_key:site.name] map[api_arg:dataset das_key:dataset pattern:/[\w-]+/[\w-]+/[A-Z-]+ rec_key:dataset.name]] expire:10800 format:JSON hash:edcc7e5dda8dcf1336a4cb1e7544697e instances:<nil> lookup:site params:map[dataset:required] system:combined ts:1.639421069e+09 type:service url:combined plugin urn:site4dataset_pct] map[das_map:[map[das_key:site pattern:^T[0-3]_ rec_key:site.name] map[api_arg:dataset das_key:dataset pattern:/[\w-]+/[\w-]+/[A-Z-]+ rec_key:dataset.name]] expire:10800 format:JSON hash:4cba2df4bd952426c45444be2d65eb5f instances:<nil> lookup:site params:map[dataset:required] system:combined ts:1.639421069e+09 type:service url:combined plugin urn:site4dataset]]
DAS local API combined_site4dataset_pct
DAS GET https://cmsweb.cern.ch/dbs/prod/global/DBSReader/filesummaries?dataset=/EphemeralZeroBias0/Run2022C-PromptReco-v1/AOD&validFileOnly=1 1.547956107s
DAS GET https://cmsweb.cern.ch/dbs/prod/global/DBSReader/blocks?dataset=/EphemeralZeroBias0/Run2022C-PromptReco-v1/AOD 211.33019ms
DAS GET https://cms-rucio.cern.ch//replicas/cms//EphemeralZeroBias0/Run2022C-PromptReco-v1/AOD#e1016302-e79d-4d00-b4a5-b09e793f1964/datasets 877.561693ms
DAS GET https://cms-rucio.cern.ch//replicas/cms//EphemeralZeroBias0/Run2022C-PromptReco-v1/AOD#64881132-11f3-45d6-815b-0e670f2b348e/datasets 837.435605ms
DAS GET https://cms-rucio.cern.ch//replicas/cms//EphemeralZeroBias0/Run2022C-PromptReco-v1/AOD#d2ba9927-e9a9-4f01-9f53-baff6f148255/datasets 870.977315ms
DAS GET https://cms-rucio.cern.ch//replicas/cms//EphemeralZeroBias0/Run2022C-PromptReco-v1/AOD#d65bde6c-4101-480e-85da-2dcdc6406a5f/datasets 860.831373ms
DAS GET https://cms-rucio.cern.ch//replicas/cms//EphemeralZeroBias0/Run2022C-PromptReco-v1/AOD#c9c4ea42-0f41-4e21-a6c0-354a636c11de/datasets 894.500149ms
DAS GET https://cms-rucio.cern.ch//replicas/cms//EphemeralZeroBias0/Run2022C-PromptReco-v1/AOD#42a9836f-e39d-499a-bff5-596dd7e33ca7/datasets 858.256995ms
DAS GET https://cms-rucio.cern.ch//replicas/cms//EphemeralZeroBias0/Run2022C-PromptReco-v1/AOD#1de0b465-5f71-49c4-abf7-48a0ba0dd240/datasets 913.348083ms
siteInfo {}
### LOCAL APIS site4dataset_pct combined 0 map[das_map:[map[das_key:site pattern:^T[0-3]_ rec_key:site.name] map[api_arg:dataset das_key:dataset pattern:/[\w-]+/[\w-]+/[A-Z-]+ rec_key:dataset.name]] expire:10800 format:JSON hash:edcc7e5dda8dcf1336a4cb1e7544697e instances:<nil> lookup:site params:map[dataset:required] system:combined ts:1.639421069e+09 type:service url:combined plugin urn:site4dataset_pct] combined_site4dataset_pct 0x109d760 0
DAS local API combined_site4dataset
### LOCAL APIS site4dataset combined 0 map[das_map:[map[das_key:site pattern:^T[0-3]_ rec_key:site.name] map[api_arg:dataset das_key:dataset pattern:/[\w-]+/[\w-]+/[A-Z-]+ rec_key:dataset.name]] expire:10800 format:JSON hash:4cba2df4bd952426c45444be2d65eb5f instances:<nil> lookup:site params:map[dataset:required] system:combined ts:1.639421069e+09 type:service url:combined plugin urn:site4dataset] combined_site4dataset 0x109d760 0
#### processLocalApis 0
Received 0 records
WARNING: No site records found in Rucio, will look-up original sites in DBS
### site query returns nothing, will look-up data in DBS

So in verbose mode you can see all DAS queries it made to upstream services, if no data is provided it is not DAS issue per-se. In this case, Rucio reports no sites for this dataset and no site info exist in DBS. Please ask this question in Rucio forum. The DAS correctly displays that there is no site info.

Unless you provide more about the error you see, e.g. screen shot, etc. I see that nothing can be done in this case for DAS.