Create a plan for updating PDB-related software

[brindakv]

• IHMCIF

  • https://github.com/ihmwg/IHMCIF
  • Data dictionary
  • Not included in PDB-Dev deployment directly

• python-ihm

  • https://github.com/ihmwg/python-ihm/
  • Used for reading and writing IHMCIF files
  • python-ihm is used in the PDB-Dev backend submission workflow and in validation report generation.
  • python-ihm gets updated when new kinds of entries are deposited, e.g., branched polymers.
  • Both backend submission workflow and validation report generation (singularity) should use updated python-ihm
  • make-mmcif.py is part of the python-ihm package
  • Consolidate how the package is updated and used
  • Use tagged version of python-ihm for both dev and production
  • Use make-mmcif.py from python-ihm
  • Remove make-mmcif.py for our protein-database git repository
  • Brinda to ask Ben to include make-mmcif.py as part of the python package( see https://github.com/ihmwg/python-ihm/issues/134) (DONE)

• IHMValidation

  • Validation software: https://github.com/salilab/IHMValidation
  • Maintained by @aozalevsky
  • External dependencies (Molprobity, ATSAS)
  • Use latest code of dev and tagged version on production

• py-rcsb_db

  • Hosted by @aozalevsky at https://salilab.org/~arthur/ihmv/packages/py-rcsb_db_v0.86.tar.gz
  • Also contains two yml config files
  • Used to convert mmCIF to JSON in the workflow
  • Package will be updated offline by @brindakv and will be tagged
  • Use tagged version on both dev and production

• cpp-dict-pack (@svoinea to update how this software is deployed)

  • Used for validating IHMCIF files against the dictionary
  • https://github.com/informatics-isi-edu/protein-database/wiki/data-submission-workflow#mmcif-validator
  • cpp-dict-pack software is stable
  • sdb file is provided by @brindakv
  • Will be handled later

• Molstar

  • Used for visualization
  • https://github.com/informatics-isi-edu/protein-database/tree/master/scripts/molstar

[svoinea]

1. Currently, make-mmcif.py is not part of the python-ihm Python Package. It can be got from the python-ihm repository:

   git clone https://github.com/ihmwg/python-ihm.git

   The make-mmcif.py file is on the python-ihm/util directory.

2. If you have versions preference, please go to https://github.com/informatics-isi-edu/protein-database/blob/master/scripts/ubuntu/www/deploy.sh#L81-L93 and specify them. So far, there was never a problem by using the latest version of them.

3. The *.yml files are now present in the py-rcsb_db_v0.86.tar.gz file, and those are the ones used. In case an update occurs to them, how should we get them? A new *.tar.gz file? Or a different way?

4. Regarding CifCheck, it was built only once as described at https://github.com/informatics-isi-edu/protein-database/blob/master/scripts/ubuntu/www/deploy.sh#L105-L113 and the binary code was comitted into the protein-database repository. It was working on all the new Fedora versions as well as now on Ubuntu. So far, no problem was reported by using only the very first build of CifCheck.

[aozalevsky]

from https://github.com/informatics-isi-edu/protein-database/issues/163#issuecomment-2010307830 by @svoinea

1. On production, the script is /root/ihm_validation_checkout.sh. The checkout is for the tag 20240315:

Each time you want to apply a new tag, set its value in the /root/ihm_validation_checkout.sh script and run it.

2. On dev, we have the daily script /etc/cron.daily/dev-update. The checkout for IHMValidation is for the latest dev_2.0:

Each time you want to apply an update, run manually the script /etc/cron.daily/dev-update. Or if want a specific tag, replace origin/dev_2.0 by the tag value, and then run the script.

The generated log file is /root/pdb-cron-hourly-update.log.