hmatsu1226
commented
6 years ago Dear @rcannood https://github.com/rcannood and @zouter https://github.com/zouter
Thank you for your interest in SCOUP. I read the checklist and find the following modification and questions.
- The code_url is wrong. Please change the code_url: https://github.com/gcyuan/PySCUBA to code_url: https://github. com/hmatsu1226/SCOUP.
- Does the "end_n" in Input correspond to the number of lineage (nbranch)?
- I assume the normalized expression (such as log(TPM+1)) for input expression data.
- If possible, separate the pre-training algorithm (SP) and SCOUP to run SCOUP with different initial pseudo-time data (for example, the pseudo-time inferred by different algorithm).
If you have any questions or anything, please do not hesitate to contact me.
Your sincerely,
Matsumoto Hirotaka RIKEN Center for Biosystems Dynamics Research Bioinformatics Research Unit
2018-06-20 19:47 GMT+09:00 Wouter Saelens notifications@github.com:
Hello @hmatsu1226 https://github.com/hmatsu1226
This issue is for discussing the wrapper for your trajectory inference method, SCOUP, which we wrapped for our benchmarking study (10.1101/276907 https://doi.org/10.1101/276907). In our dynmethods https://github.com/dynverse/dynmethods framework, we collected some meta information about your method, and created a docker wrapper so that all methods can be easily run and compared. The code for this wrapper is located in a docker container https://github.com/dynverse/dynmethods/tree/master/containers/scoup. The way this container is structured is described in this vignette https://dynverse.github.io/dynwrap/articles/create_ti_method_docker.html .
We are creating this issue to ensure your method is being evaluated in the way it was designed for. The checklist below contains some important questions for you to have a look at.
Parameters, defined in definition.yml https://github.com/dynverse/dynmethods/tree/master/containers/scoup/definition.yml (more info https://dynverse.github.io/dynwrap/articles/create_ti_method_docker.html#parameters)
- Are all important parameters described in this file?
- For each parameter, is the proposed default value reasonable?
- For each parameter, is the proposed parameter space reasonable (e.g. lower and upper boundaries)?
- Is the description of the parameters correct and up-to-date?
Input, defined in definition.yml https://github.com/dynverse/dynmethods/tree/master/containers/scoup/definition.yml and loaded in run.R https://github.com/dynverse/dynmethods/tree/master/containers/scoup/run.R (more info https://dynverse.github.io/dynwrap/articles/create_ti_method_docker.html#input)
- Is the correct type of expression requested (raw counts or normalised expression)?
- Is all prior information (required or optional) requested?
- Would some other type of prior information help the method?
Output, defined in definition.yml https://github.com/dynverse/dynmethods/tree/master/containers/scoup/definition.yml and saved in run.R https://github.com/dynverse/dynmethods/tree/master/containers/scoup/run.R (more info https://dynverse.github.io/dynwrap/articles/create_ti_method_docker.html#output)
- Is the output correctly processed towards the common trajectory model https://github.com/dynverse/dynwrap#dynwrap? Would some other postprocessing method make more sense?
- Is all relevant output saved (dimensionality reduction, clustering/grouping, pseudotime, ...)
Wrapper script, see run.R https://github.com/dynverse/dynmethods/tree/master/containers/scoup/run.R (more info https://dynverse.github.io/dynwrap/articles/create_ti_method_docker.html#doing-trajectory-inference)
- This is a script that is executed upon starting the docker container. It will receive several input files as defined by definition.yml, and is expected to produce certain output files, also as defined by definition.yml.
- Is the script a correct representation of the general workflow a user is expected to follow when they want to apply your method to their data?
- Quality control, see the qc worksheet https://docs.google.com/spreadsheets/d/1GWWdndfTPtXPsCXK07jU03MmhIJHmv28E7YiMnQ_0Xo//edit#gid=482419835
- We also evaluated the implementation of a method based on a large check list of good software software development practices.
- Are the answers we wrote down for your method correct and up to date? Do you disagree with certain answers? (Feel free to leave a comment in the worksheet)
- You can improve the QC score of your method by implementing the required changes and letting us know. Do not gloss over this, as it is the easiest way to improve the overall ranking of your TI method in our study!
The most convenient way for you to test and adapt the wrapper is to install dyno https://github.com/dynverse/dyno, download and modify these files https://github.com/dynverse/dynmethods/tree/master/containers/scoup, and run your method on a dataset of interest or one of our synthetic toy datasets. This is further described in this vignette https://dynverse.github.io/dynwrap/articles/create_ti_method_docker.html. Once finished, we prefer that you fork the dynmethods repository https://github.com/dynverse/dynmethods, make the necessary changes, and send us a pull request. Alternatively, you can also send us the files and we will make the necessary changes.
If you have any further questions or remarks, feel free to reply to this issue.
Kind regards, @rcannood https://github.com/rcannood and @zouter https://github.com/zouter
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/dynverse/dynmethods/issues/48, or mute the thread https://github.com/notifications/unsubscribe-auth/APhazlYklZg-Kzn_UhMVWkPcHDkXNfKcks5t-igygaJpZM4Uu_g_ .
rcannood
Hello @hmatsu1226
This issue is for discussing the wrapper for your trajectory inference method, SCOUP, which we wrapped for our benchmarking study (10.1101/276907). In our dynmethods framework, we collected some meta information about your method, and created a docker wrapper so that all methods can be easily run and compared. The code for this wrapper is located in a docker container. The way this container is structured is described in this vignette.
We are creating this issue to ensure your method is being evaluated in the way it was designed for. The checklist below contains some important questions for you to have a look at.
definition.yml, and is expected to produce certain output files, also as defined bydefinition.yml.The most convenient way for you to test and adapt the wrapper is to install dyno, download and modify these files, and run your method on a dataset of interest or one of our synthetic toy datasets. This is further described in this vignette. Once finished, we prefer that you fork the dynmethods repository, make the necessary changes, and send us a pull request. Alternatively, you can also send us the files and we will make the necessary changes.
If you have any further questions or remarks, feel free to reply to this issue.
Kind regards, @zouter and @rcannood