dmwm / CRABServer

16 stars 39 forks source link

Modification in periodic remove expression for CMS LPC at FNAL #5165

Closed farrukh-aftab-khan closed 8 years ago

farrukh-aftab-khan commented 8 years ago

Hi,

CRAB3 periodic remove expression checks for "MemoryUsage" classAd in the running jobs. When a job gets routed to "Grid Universe", this classAd is dropped causing CRAB3 job to go to hold with the following reason: The job attribute PeriodicRemove expression

'( ( ( JobStatus =?= 5 ) && ( time() - EnteredCurrentStatus > 7 * 60 ) ) || ( ( JobStatus =?= 1 ) && ( time() - EnteredCurrentStatus > 7 * 24 * 60 * 60 ) ) || ( ( JobStatus =?= 2 ) && ( ( DiskUsage > 100000000 ) || ( ( MaxWallTimeMins * 60 ) < ( time() - EnteredCurrentStatus ) ) || ( MemoryUsage > RequestMemory ) ) ) || ( ( JobStatus =?= 1 ) && ( time() > ( x509UserProxyExpiration + 86400 ) ) ) )' evaluated to UNDEFINED

Can we modify this expression on one of the testbed taskworkers? Maybe replace

( MemoryUsage > RequestMemory )

with

ifthenelse(MemoryUsage is undefined, false, MemoryUsage > RequestMemory)

I am open to suggestions on how else to fix this. Right now I think hard coding any value should be okay? I need to be sure that we don't hit another bottleneck.

Farrukh

mmascher commented 8 years ago

Hi, I am sure @jmarra13 can help you with this now that we are done with the release and we don't need the preprod TW for a couple of weeks!