dmwm / DAS

Data Aggregation System
11 stars 7 forks source link

DAS server error 500 #2930

Closed DMWMBot closed 12 years ago

DMWMBot commented 12 years ago

https://cmsweb.cern.ch/das/request?view=list&limit=10&instance=cms_dbs_prod_global&input=dataset+dataset%3D%2FQCD_Pt-30to50_MuPt5Enriched_TuneZ2_7TeV-pythia6%2FFall10-START38_V12-v1%2FAODSIM returns 500 Internal Server Error

The server encountered an unexpected condition which prevented it from fulfilling the request.

yes, this was an attempt to list a deprecated dataset. Same query was completing this afternoon reporting "nothing found". Maybe different cache content triggers different reports ?

On the mundate side, it is a bit confusing that e.g. status dataset=/QCD_Pt-30to50_MuPt5Enriched_TuneZ2_7TeV-pythia6/Fall10-START38_V12-v1/AODSIM and site dataset=/QCD_Pt-30to50_MuPt5Enriched_TuneZ2_7TeV-pythia6/Fall10-START38_V12-v1/AODSIM work well, esp. the latter, while simply asking for the dataset existance, not the sites, gives error. Now activating color maps helps and I understand that site dataset=... works because it asks phedex which does not care for DBS status flag. But some users may be confused sort of hopelessly. It is odd that DBS can return status value, but nothing else. E.g. those all work block dataset=/QCD_Pt-30to50_MuPt5Enriched_TuneZ2_7TeV-pythia6/Fall10-START38_V12-v1/AODSIM config dataset=/QCD_Pt-30to50_MuPt5Enriched_TuneZ2_7TeV-pythia6/Fall10-START38_V12-v1/AODSIM status dataset=/QCD_Pt-30to50_MuPt5Enriched_TuneZ2_7TeV-pythia6/Fall10-START38_V12-v1/AODSIM and possibly more, I did not try all, but dataset dataset=... fails !

vkuznet commented 12 years ago

valya: Stefano, please open different ticket for different issues, rather then putting everything together. The server issue has been reported today and I it happens due to bug in DAS server code, see #2925.

now about issues you raised. I'll answer them here and close the ticket since it reports 500 server error, which was addressed #2925. If you need to address other issues, please open separate tickets, one per issue.

So, the status field. By default, the agreement is to look-up VALID datasets. If you want to check some dataset with another status you need to use status keyword, e.g. dataset dataset=/a/b/c status=DEPRECATED so your example dataset dataset=/QCD_Pt-30to50_MuPt5Enriched_TuneZ2_7TeV-pythia6/Fall10-START38_V12-v1/AODSIM status=DEPRECATED produces dataset information.

the site dataset=/a/b/c query works because it queries phedex, not dbs the status dataset=/a/b/c query works because you asked for a status and not dataset details, it queries DBS. If users confused, they will speak, but so far I got only one reply from you. Color map helps to understand the records.

The other queries, e.g. block dataset=/QCD_Pt-30to50_MuPt5Enriched_TuneZ2_7TeV-pythia6/Fall10-START38_V12-v1/AODSIM works because DBS API I use (listBlocks4path) does not check status. So the real problem with confusion is "default status assignment for dataset lookup". But the detault covers majority of use cases, e.g. only a few people may look datasets with other statuses. Therefore you just need to learn to supply status keyword. The server error fix will be scheduled in next upgrade window.

DMWMBot commented 12 years ago

belforte: thanks for explanation Valentin, sorry I did not notice that error 500 was reported alreday. About the discussion, you summarize very well the issue:

the real problem with confusion is "default status assignment for dataset lookup".

of course it is only relevant to the extent that datasets with status other then VALID are used/interesting. Which depends also on DataOps policies and is changing. We'll see.