Improve condor submission

[kreczko]

The current condor submission is still a bit wonky and needs improvement

• [x] balanced jobs: currently the grouping function will split jobs in groups of N, but the last group will be the smallest. The solution to this problem is

def n_even_chunks(l, n):
    last = 0
    for i in range(1, n+1):
        cur = int(round(i * (len(l) / n)))
        yield l[last:cur]
        last = cur

• [x] faster AT output: currently the ntuple layer has to finish to 100% before AT can run. Instead we should start AT jobs as soon as their input files are ready (i.e. make them dependent on a sub-group of ntuple jobs).
• [x] only central AT for data: no need to run other AT modes for data
• [ ] add metadata:
  • [ ] each job should store the input files, output files, lumi JSON used (data only)
  • [ ] run create cutflow, create lumi_json (data-only) and scan number of processed events as the last step of an ntuple job
• [x] summarise DAS calls to improve submission performance (caching added in PR #244)

Please add missing items

[kreczko]

To add:

• [x] do what CRAB does for the compilation: https://github.com/dmwm/CRABClient/blob/master/src/python/CRABClient/JobType/UserTarball.py#L40

In short:

• ship the libraries directories = ['lib', 'biglib', 'module']
• only zip data and interface directories from src we can adopt this for the creation of the cmssw.tar.gz. which in turn should reduce the compilation time to 0. Both ntp create tarball and ntp setup from=tarball need adjustments.