cernopendata / opendata.cern.ch

Source code for the CERN Open Data portal
http://opendata.cern.ch/
GNU General Public License v2.0
666 stars 148 forks source link

Update to 2011/2012 VM #1959

Closed katilp closed 6 years ago

katilp commented 6 years ago

We can update the 2011 VM with a newer version with an increased cache volume for #1877 The VM is available from https://cernbox.cern.ch/index.php/s/SkLCf42lo6DqdAE with the following changes with respect to the old one (from Jakob)

I made the following changes:

1) I used the slightly updated base image

http://cernvm.cern.ch/releases/production/cernvm3-micro-2.8-6.hdd

2) In the cms-user-data.txt, I changed

echo "CVMFS_QUOTA_LIMIT=10000 # 10G cache" > /etc/cvmfs/default.local

to

echo "CVMFS_QUOTA_LIMIT=20000 # 20G cache" > /etc/cvmfs/default.local

3) I used the modified cernvm script

cvm2ova -n "CMS-Open-Data-1.3.0" -d 40000 -m 2048 \
  -i cernvm3-micro-2.8-6.hdd \
  -u cms-user-data.txt

Testing

katilp commented 6 years ago

Test as in #1877

Start 17:49:

[cms-opendata@localhost ~]$ df
Filesystem     1K-blocks   Used Available Use% Mounted on
aufs            40182924 901688  37216968   3% /
tmpfs            1022272      0   1022272   0% /dev/shm
cvmfs2          20480000 539998  19940002   3% /cvmfs/cvmfs-config.cern.ch
cvmfs2          20480000 539998  19940002   3% /cvmfs/cms.cern.ch
cvmfs2          20480000 539998  19940002   3% /cvmfs/cms-ib.cern.ch
cvmfs2          20480000 539998  19940002   3% /cvmfs/cms-opendata-conddb.cern.ch

end 18:36:

[cms-opendata@localhost ~]$ df
Filesystem     1K-blocks     Used Available Use% Mounted on
aufs            40182924 11269640  26849016  30% /
tmpfs            1022272        0   1022272   0% /dev/shm
cvmfs2          20480000 10869733   9610268  54% /cvmfs/cvmfs-config.cern.ch
cvmfs2          20480000 10869733   9610268  54% /cvmfs/cms.cern.ch
cvmfs2          20480000 10869733   9610268  54% /cvmfs/cms-ib.cern.ch
cvmfs2          20480000 10869733   9610268  54% /cvmfs/cms-opendata-conddb.cern.ch
katilp commented 6 years ago

Testing the RAW reco with this VM, see http://opendata-dev.web.cern.ch/record/463 or direct in https://github.com/cms-opendata-validation/2011-minimumbias-raw-recotest/blob/master/README.md Interested to see whether this solves the problem of killed jobs for rereco UPDATE: does not help, killed after close to 20000 evts

katilp commented 6 years ago

@tiborsimko Can we add another version of 2011/2012 VM to the 2011 record? I would prefer to have it there for the sake of simplicity for users (instructions why to choose the new VM can reside in the VM record and not be spread all over in different instructions). It will have its own contextualization file record, though,

katilp commented 6 years ago

Tested by @zenaiev, no problem with the new VM

katilp commented 6 years ago

Closing as superseded by #2044