Migrate from Hadoop to ceph

[ktht]

At the very least implies that we update sample dictionaries and clean up the code from hdfs modules and commands.

[saswatinandan]

Are you creating new dictionaries? I need to submit new jobs asap

[ktht]

I'll look into this. ETA by tomorrow. We need to double-check with our sysadmin that we can proceed with the job submission, though.

[ktht]

AFAICT, everything should be now using the Ceph paths instead of the old Hadoop paths and modules. Just to reiterate what I had written in the commit messages:

• Changed the paths from /hdfs/local to /local and from /hdfs/cms/store to /store
  • Note that the migration of /hdfs/cms/store hasn't begun at the time of writing this post, so those paths are currently invalid. This is not a big deal since we only store pre-processed Ntuples on /hdfs/cms/store, which are not directly used anywhere in the analysis;
  • Did not bother changing anything in the macros directories since they host code that refers to already deleted files;
• Replaced calls to the hdfs module with the standard calls to the os module;
  • However, I did not bother to test any of the scripts, since the likelihood of using them is very low

I still kept HDFS ROOT plugin, hdfs python module and TFileOpenWrapper for historical reasons since this repository will be deprecated anyways once the bbWW analysis has concluded.

Reading from and writing to /local works as intended and the FW is able to figure out the correct input paths, so at least something is guaranteed to function at this point in time.

Also note that the same best practices apply to Ceph as they applied to Hadoop:

• keep the number of files per directory reasonable (1k or so);
• avoid small files (smaller than 10 MB) as much as possible.

There are no special commands (other than those listed here), which enable to bypass FUSE (since there is no FUSE to begin with) or have otherwise a more "direct" access to the underlying file system. Use the standard POSIX commands instead.

Closing this issue with a disclaimer that not everything has been fully tested, plus we need to coordinate with our sysadmin before we can submit anything substantial to the cluster.