Use ReadTools wherever applicable

robmaz / distmap

Sequence alignment on Hadoop

0 stars 1 forks source link

Use ReadTools wherever applicable #4

Closed magicDGS closed 6 years ago

magicDGS commented 6 years ago

Use version >= 1.1.0 of https://github.com/magicDGS/ReadTools for:

[x] Upload the data
[ ] Trimming (should be done locally and uploaded afterwards)
[x] Download the data

magicDGS commented 6 years ago

The following line is quite important for the trimming part: https://github.com/robmaz/distmap/blob/de799f8938b79cb12b2d4b67335de9170ac6b4f7/bin/HadoopTrimming.pm#L202

ReadTools version >= 1.1.0 won't work there, and I believe that it is also requiring upload the jar file (which is quite big), so we should focus on a new trimming algorithm to perform locally, but without being transparent for the final user, just warn that it won't be done in the cluster...

magicDGS commented 6 years ago

Uploading/Downloading is already done in #2

robmaz commented 6 years ago

Mostly done with #54