tinyRNA provides an all-in-one solution for precision analysis of sRNA-seq data. At the core of tinyRNA is a highly flexible counting utility, tiny-count, that allows for hierarchical assignment of reads to features based on positional information, extent of feature overlap, 5’ nucleotide, length, and strandedness.
The tinyRNA environment and its dependencies have been upgraded to use Python 3.10. This PR also improves the developer experience for managing dependencies and updating lockfiles.
Environment Management
The update-lockfiles.sh script has been replaced with env.py. The previous script only converted an environment.yml file (manually created) to lockfiles for osx and linux. The new script offers the following commands:
new-env: Automated creation of new environment.yml files for a specified version of Python and a template file. The file environment_template.yml has been provided. If additional dependencies or pinned specifications are required, they should be manually added to the template before passing it to the command. All unpinned specifications are solved with mamba before the new file is written.
rebuild-locks: Rebuilds the (OS-specific) lockfiles from the (OS-agnostic) environment file using conda-lock. Once complete, the new lockfiles are scanned for vulnerable Conda packages using jake. Lockfiles produced using this command will overwrite any existing lockfiles so that they are immediately available for use with setup.sh (i.e. unlike the environment.yml files produced above, their filenames do not indicate the version of Python)
make-archive: Produces a compressed archive of an existing Conda environment using conda-pack. The archive can be used to perform a fully offline installation of the environment on another machine, even if the requested packages don't exist in the target machine's local package cache. The archive is OS-specific, however.
Installation Script Changes and Improvements
The installation script no longer supports in-place upgrades for existing tinyRNA environments. The environment, and any packages the user may have installed, must now be removed by the script during an update. This functionality had been introduced when dependency updates were frequent but the Python version remained unchanged. Unfortunately it seems that there isn't a reliable way to change an environment's Python version without re-solving the environment, which brings back the unacceptable 20-40 minute wait times. If users really want to do an in-place upgrade, they can always use conda update --file with the desired environment*.yml file to initiate the re-solve but this is advanced usage and likely to cause issues.
Additionally:
The Miniconda installer version has been updated
The proper Miniconda installer is now used if the user has a mac with an M1 or M2 chip
If Miniconda required installation, the user is notified of a final setup step they must perform if they want to immediately begin using Conda
Security Improvements
This PR improves the handling of configuration file fields to avoid potential shell injection vulnerabilities.
The tinyRNA environment and its dependencies have been upgraded to use Python 3.10. This PR also improves the developer experience for managing dependencies and updating lockfiles.
Environment Management
The
update-lockfiles.sh
script has been replaced withenv.py
. The previous script only converted an environment.yml file (manually created) to lockfiles for osx and linux. The new script offers the following commands:new-env
: Automated creation of new environment.yml files for a specified version of Python and a template file. The file environment_template.yml has been provided. If additional dependencies or pinned specifications are required, they should be manually added to the template before passing it to the command. All unpinned specifications are solved with mamba before the new file is written.rebuild-locks
: Rebuilds the (OS-specific) lockfiles from the (OS-agnostic) environment file using conda-lock. Once complete, the new lockfiles are scanned for vulnerable Conda packages using jake. Lockfiles produced using this command will overwrite any existing lockfiles so that they are immediately available for use withsetup.sh
(i.e. unlike the environment.yml files produced above, their filenames do not indicate the version of Python)make-archive
: Produces a compressed archive of an existing Conda environment using conda-pack. The archive can be used to perform a fully offline installation of the environment on another machine, even if the requested packages don't exist in the target machine's local package cache. The archive is OS-specific, however.Installation Script Changes and Improvements
The installation script no longer supports in-place upgrades for existing tinyRNA environments. The environment, and any packages the user may have installed, must now be removed by the script during an update. This functionality had been introduced when dependency updates were frequent but the Python version remained unchanged. Unfortunately it seems that there isn't a reliable way to change an environment's Python version without re-solving the environment, which brings back the unacceptable 20-40 minute wait times. If users really want to do an in-place upgrade, they can always use
conda update --file
with the desired environment*.yml file to initiate the re-solve but this is advanced usage and likely to cause issues.Additionally:
Security Improvements
This PR improves the handling of configuration file fields to avoid potential shell injection vulnerabilities.
Closes #261