Update schema for WMArchive documents

[amaltaro]

Marco, Stefano, we would like to update the document schema on the WMArchive server side and for that we need to update the clients posting documents to it.

Can you please make sure the documents posted by CRAB have the following changes:

• send only selected parameters to WMArchive from performance data.
• use int for NumberOfThreads and NumberOfStreams

You may have a look at this PR Seangchan @ticoann created for WMCore https://github.com/dmwm/WMCore/pull/8209

otherwise let me know where exactly the schema is enforced and I can make a PR for you. Thanks

[ericvaandering]

This is somewhat urgent because right now CRAB and WMAgent are providing the data in the wrong format and we can't update WMArchive to begin storing this data until all the data sources are providing the right format. WMAgent is going to address this by a patch, I'm hoping CRAB can do the same.

[ticoann]

@mmascher, @belforte, Just to make sure this reaches to you guys.

[belforte]

only @mmascher knows. If in dire need i (or you) can try to find the code and decode it.. but if he can get to this he can surley be faster.

[ticoann]

It looks like it uses WMCore code.

https://github.com/dmwm/CRABServer/blob/cc4e32b967f070110246188a3fc7e071e786ffc3/src/python/TaskWorker/Actions/PostJob.py#L90

I can create new crab branch patch, if that is what it needs be done. Otherwise current master contains this patch.

[belforte]

thanks @ticoann , yes it calls createArchiverDoc in https://github.com/dmwm/CRABServer/blob/cc4e32b967f070110246188a3fc7e071e786ffc3/src/python/TaskWorker/Actions/PostJob.py#L2753 so it is a matter of updating WMcore branch in CRAB and create new TaskWorker rpm which is a procedure I do not know myself (and wish we could get rid of)

[mmascher]

I can create a new WMCore for CRAB3 that cherrypicks https://github.com/dmwm/WMCore/pull/8209/commits/81c7b4890f6d965b278ee5c1e0f7f105f69ad249

Should be enough right?

When do we need to deploy this, I guess we need to synchronize a little bit.

[ticoann]

Hi Marco, Actually, you have to cherrypick this first https://github.com/dmwm/WMCore/pull/8195/commits/bfc38dca781e019c538dd3e8c4dc85a39adf95b3 then https://github.com/dmwm/WMCore/commit/81c7b4890f6d965b278ee5c1e0f7f105f69ad249

When it is ready I will create a new tag for crab. Thanks a lot. For deployment, After we deploy everything crab and agents. we probably wait a day to previous docs go into WMArchive, then update the WMArchive.

[ericvaandering]

You don't need to wait before deploying a new CRAB, though. However, after you have done it, only then can WMArchive step forward.

[mmascher]

I made the PR here https://github.com/dmwm/WMCore/pull/8234 for the crab WMCore branch

However, I think I'll need to patch the uploader script as well. Documents are created by postjobs, and the postjob code of the tasks already in the schedd won't be updated. And the lifetime of a postjob is 30 days..

I can force a default value (0?) for all the docs for the new keys (*) in the uploader script

(*) 'TotalInitTime': float 'TotalInitCPU': float 'NumberOfThreads': int 'NumberOfStreams': int

[ticoann]

I think you can force to default value. I think the data won't be consistent or correct for a while anyway until a fresh new workflows start to generate. What do you think? @ericvaandering, @vkuznet

[ericvaandering]

I'm a bit confused @mmascher . Can you use the uploader script to get around the 30 day lifetime by making sure these new values are either ints or not present? Both of those are fine, the problem is when they are strings.

[mmascher]

Ok @ericvaandering , I just checked the schedd. If it works I can make NumberOfThreads and NumberOfStreams integers since they are present in the reports, and ignore TotalInitTime and TotalInitCPU that are not there.

[ericvaandering]

That would be great. So a temporary hack that eventually gets removed as the WMCore code provides the right format.

[mmascher]

FWI I also found TotalInitCPU and TotalInitTime , so I will update all the 4 params

[mmascher]

@todor-ivanov I put the hacked uploader in data/srv/SubmissionInfrastructureScripts/WMArchiveUploaderNew.py on 0198.

As far as I understand we can already deploy it on all schedd, righ @ericvaandering ?

[ericvaandering]

Correct

[todor-ivanov]

Done! The new uploader is in all the schedds now.

[belforte]

@mmascher @amaltaro can this be closed ?

[amaltaro]

I think it can, though Valentin has other metrics that he would like to have populated. https://github.com/dmwm/WMCore/issues/8279

[belforte]

OK. Let's open one ad hoc issue then