Closed happyfox-robot closed 2 years ago
Ticket update by SciComp (scicomp@fredhutch.org)
Dear David Scalzo,
Your Ticket ID is #SC00007243 and Ticket Title is: DIHVA module creation
A copy of your original message is included below.
Sincerely,
Scientific Computing (SciComp)
SciComp@FredHutch.org
Ticket message:
Hi,
We would like this analysis pipeline available as a module.
If the Docker R/Shiny is workable that would be great.
If not, working at terminal will work with your guidance.
Thanks
David
Cohn lab
The DIHIVA pipeline is publicly available on GitHub and we created a docker image that can be used on Linux, Mac, and Windows.
Ticket update by Dan Tenenbaum (dtenenba@fredhutch.org)
Hello,
I am not clear exactly what you are asking for. If you just want to run this yourself, you can do it with the instructions provided at the link you sent. I'm happy to help with those if you have questions about them.
If you want this to be up 24/7 and available to anyone, please go to https://getshiny.fredhutch.org/ (VPN required if off campus) and request that.
We can't make a module of this because Docker is not allowed on the shared systems (rhino, gizmo, etc) because it requires root access and that is a security issue.
We do support Singularity which is like Docker but does not require root access. I am not sure if we can convert this container to run in Singularity but I can check into it.
Dan
Ticket update by David Scalzo (scalzod@fredhutch.org)
Hi Dan,
The developer (Victor, who Lillie formerly worked with at Rockefeller) told us we would be better off running this on a server. It’s how it’s run at Rockefeller.
Yes, it could be useful for other HIV researchers to have access to this.
Will the R/Shiny folks be able to handle the Docker issue? We will contact Victor to find out if Singularity will also work.
Thanks,
David Cohn lab
From: Dan Tenenbaum scicomp@fredhutch.org Date: Thursday, March 24, 2022 at 10:23 AM To: Scalzo, David scalzod@fredhutch.org Cc: Cohn, Lillian B lcohn@fredhutch.org, Dey, John F jfdey@fredhutch.org Subject: Re: #SC00007243: : DIHVA module creation New Reply for Ticket #SC00007243
Ticket Title: : DIHVA module creation
Hello,
I am not clear exactly what you are asking for. If you just want to run this yourself, you can do it with the instructions provided at the link you sent. I'm happy to help with those if you have questions about them.
If you want this to be up 24/7 and available to anyone, please go to https://getshiny.fredhutch.org/ (VPN required if off campus) and request that.
We can't make a module of this because Docker is not allowed on the shared systems (rhino, gizmo, etc) because it requires root access and that is a security issue.
We do support Singularity which is like Docker but does not require root access. I am not sure if we can convert this container to run in Singularity but I can check into it.
Dan
Dan Tenenbaum Cloud Systems Engineer, Scientific Computing Fred Hutch Pronouns: He/him Email: dtenenba@fredhutch.org
Thank you,
Scientific Computing (SciComp) SciComp@FredHutch.org User Self-Service Portal: https://fredhutch.happyfox.com/saml/login/https://urldefense.proofpoint.com/v2/url?u=https-3A__fredhutch.happyfox.com_saml_login_&d=DwMDaQ&c=eRAMFD45gAfqt84VtBcfhfEazhEXT91ASHynm_9f1N0&r=GO1B7lbxFhbz62_4-qxrS34YhWYz26EIwt_QIcAjrTo&m=No2fzgjUYtEjIQKmpbgOxyi6tvQfNXuvQgFAmllurzeDHU5cYmps4Ond2mn4YPIP&s=iJiA1kQRjApov6bSoraZcdtQQuFW6MT3SQnMSBpMR60&e=
Ticket update by Dan Tenenbaum (dtenenba@fredhutch.org)
Sorry, I should have been more clear. If you go with the option of having it up 24/7 for multiple people to access, then it will use Docker, not singularity.
Dan
Ticket update by David Scalzo (scalzod@fredhutch.org)
Which means it is not possible to have it running on a server?
What if it’s only accessible to our group?
Are the R/Shiny folks able to handle this?
David Cohn lab
From: Dan Tenenbaum scicomp@fredhutch.org Date: Thursday, March 24, 2022 at 10:51 AM To: Scalzo, David scalzod@fredhutch.org Cc: Cohn, Lillian B lcohn@fredhutch.org, Dey, John F jfdey@fredhutch.org Subject: Re: #SC00007243: : : DIHVA module creation New Reply for Ticket #SC00007243
Ticket Title: : : DIHVA module creation
Sorry, I should have been more clear. If you go with the option of having it up 24/7 for multiple people to access, then it will use Docker, not singularity.
Dan
Dan Tenenbaum Cloud Systems Engineer, Scientific Computing Fred Hutch Pronouns: He/him Email: dtenenba@fredhutch.org
Thank you,
Scientific Computing (SciComp) SciComp@FredHutch.org User Self-Service Portal: https://fredhutch.happyfox.com/saml/login/https://urldefense.proofpoint.com/v2/url?u=https-3A__fredhutch.happyfox.com_saml_login_&d=DwMDaQ&c=eRAMFD45gAfqt84VtBcfhfEazhEXT91ASHynm_9f1N0&r=GO1B7lbxFhbz62_4-qxrS34YhWYz26EIwt_QIcAjrTo&m=EjoNsPEF-CVgQPsZzSq2gYi6Yxp8FQB5Tj4n8gNtibk5D2zp4z66h7FDsVQCOLx3&s=2QtHuMSEQWWIvBa8WNc1RbKQW3lpTfp0hJrTfZKoiy4&e=
Ticket update by Dan Tenenbaum (dtenenba@fredhutch.org)
Hi,
I am the "R/Shiny folks".
it would be running on a server. it can be accessible to anyone you want, either inside the Fred Hutch network only, or on the public internet. If you don't want everyone (either on campus or worldwide) to have access to it, we can put a username and password on it to restrict access.
Dan
Ticket update by David Scalzo (scalzod@fredhutch.org)
I see, and this gets around the Docker issue?
From: Dan Tenenbaum scicomp@fredhutch.org Date: Thursday, March 24, 2022 at 10:57 AM To: Scalzo, David scalzod@fredhutch.org Cc: Cohn, Lillian B lcohn@fredhutch.org, Dey, John F jfdey@fredhutch.org Subject: Re: #SC00007243: : : : DIHVA module creation New Reply for Ticket #SC00007243
Ticket Title: : : : DIHVA module creation
Hi,
I am the "R/Shiny folks".
it would be running on a server. it can be accessible to anyone you want, either inside the Fred Hutch network only, or on the public internet. If you don't want everyone (either on campus or worldwide) to have access to it, we can put a username and password on it to restrict access.
Dan
Dan Tenenbaum Cloud Systems Engineer, Scientific Computing Fred Hutch Pronouns: He/him Email: dtenenba@fredhutch.org
Thank you,
Scientific Computing (SciComp) SciComp@FredHutch.org User Self-Service Portal: https://fredhutch.happyfox.com/saml/login/https://urldefense.proofpoint.com/v2/url?u=https-3A__fredhutch.happyfox.com_saml_login_&d=DwMDaQ&c=eRAMFD45gAfqt84VtBcfhfEazhEXT91ASHynm_9f1N0&r=GO1B7lbxFhbz62_4-qxrS34YhWYz26EIwt_QIcAjrTo&m=9ictR98e5dr1zZ_Ify0ueDkoBPruuvNU_doD29HDUGiGkwAibdE5-5sV_hYqt0hJ&s=u7zqEV68-Y5NkmFVokVe9ExJLZ_BgrAkmIX7wmeCAaM&e=
Ticket update by Dan Tenenbaum (dtenenba@fredhutch.org)
Yes.
Dan
Ticket update by David Scalzo (scalzod@fredhutch.org)
Hi Dan,
Can you give me advice on how to fill out the shiny form?
I’m not submitting a project. I’m asking for an application to be made available for us to use.
David Cohn lab From: Dan Tenenbaum scicomp@fredhutch.org Date: Thursday, March 24, 2022 at 11:03 AM To: Scalzo, David scalzod@fredhutch.org Subject: Re: #SC00007243: : : : : DIHVA module creation New Reply for Ticket #SC00007243
Ticket Title: : : : : DIHVA module creation
Yes. Dan
Dan Tenenbaum Cloud Systems Engineer, Scientific Computing Fred Hutch Pronouns: He/him Email: dtenenba@fredhutch.org
Thank you,
Scientific Computing (SciComp) SciComp@FredHutch.org User Self-Service Portal: https://fredhutch.happyfox.com/saml/login/https://urldefense.proofpoint.com/v2/url?u=https-3A__fredhutch.happyfox.com_saml_login_&d=DwMDaQ&c=eRAMFD45gAfqt84VtBcfhfEazhEXT91ASHynm_9f1N0&r=GO1B7lbxFhbz62_4-qxrS34YhWYz26EIwt_QIcAjrTo&m=49BXu33VC7KucJ2ew4VYaNIqsxBH0Hbhux7XbfWu4jAcTxBFoR8W9FBp_EuhRUiO&s=VFZmmECgNaRnN94U-YwHeIPjm6Mn0NiVA_gvGv3s7JM&e=
Ticket update by Dan Tenenbaum (dtenenba@fredhutch.org)
it's the same thing.
We can probably skip having you fill out that form, I just need answers to the following questions:
What do you want the site to be called? something.fredhutch.org
Do you want it on the public internet or just on the fred hutch network (VPN required if not on campus)?
Do you want access protected by a username and password? if so,. what is the username and password? (should not be an actual user's hutchnet ID and password).
can you affirm that this app will not be handling PHI, including the files that get uploaded to it?
Thanks
Dan
Ticket update by David Scalzo (scalzod@fredhutch.org)
Okay.
DIHIVA.fredhutch.org
Hutch network.
No password.
By PHI you mean patient/donor data? I can affirm that any such data DIVIHA will handle is de-identified.
Best,
David Cohn lab
From: Dan Tenenbaum scicomp@fredhutch.org Date: Thursday, March 24, 2022 at 11:16 AM To: Scalzo, David scalzod@fredhutch.org Cc: Cohn, Lillian B lcohn@fredhutch.org Subject: Re: #SC00007243: : : : : : DIHVA module creation New Reply for Ticket #SC00007243
Ticket Title: : : : : : DIHVA module creation
it's the same thing.
We can probably skip having you fill out that form, I just need answers to the following questions:
What do you want the site to be called? something.fredhutch.org
Do you want it on the public internet or just on the fred hutch network (VPN required if not on campus)?
Do you want access protected by a username and password? if so,. what is the username and password? (should not be an actual user's hutchnet ID and password).
can you affirm that this app will not be handling PHI, including the files that get uploaded to it?
Thanks Dan
Dan Tenenbaum Cloud Systems Engineer, Scientific Computing Fred Hutch Pronouns: He/him Email: dtenenba@fredhutch.org
Thank you,
Scientific Computing (SciComp) SciComp@FredHutch.org User Self-Service Portal: https://fredhutch.happyfox.com/saml/login/https://urldefense.proofpoint.com/v2/url?u=https-3A__fredhutch.happyfox.com_saml_login_&d=DwMDaQ&c=eRAMFD45gAfqt84VtBcfhfEazhEXT91ASHynm_9f1N0&r=GO1B7lbxFhbz62_4-qxrS34YhWYz26EIwt_QIcAjrTo&m=8WXzbSCp5g3kncjjxXTb3QfX8nhiisGI00qblAZhFQWn-HAeB3s9WZ01g2BrB2wC&s=U_NBcZfQlQjwNVjZLldoIrsq5Yc8JStdnizaolKfWDY&e=
Ticket update by Dan Tenenbaum (dtenenba@fredhutch.org)
Great.
One more thing - can you point me to a couple of files that can be used to test this app? Files that I could upload and get results from. If they are too large to email, maybe send them in onedrive or share them some other way?
Thanks,
Dan
Ticket update by David Scalzo (scalzod@fredhutch.org)
https://www.dropbox.com/sh/jbzy6s0frfpof36/AADYCqmLRxWrXa2e10KsRY9ia?dl=0
It’s at the very bottom of the readme file. The readme file also includes examples of the expected results.
Thanks,
David Cohn lab
From: Dan Tenenbaum scicomp@fredhutch.org Date: Thursday, March 24, 2022 at 11:37 AM To: Scalzo, David scalzod@fredhutch.org Cc: Cohn, Lillian B lcohn@fredhutch.org Subject: Re: #SC00007243: : : : : : : DIHVA module creation New Reply for Ticket #SC00007243
Ticket Title: : : : : : : DIHVA module creation
Great.
One more thing - can you point me to a couple of files that can be used to test this app? Files that I could upload and get results from. If they are too large to email, maybe send them in onedrive or share them some other way?
Thanks, Dan
Dan Tenenbaum Cloud Systems Engineer, Scientific Computing Fred Hutch Pronouns: He/him Email: dtenenba@fredhutch.org
Thank you,
Scientific Computing (SciComp) SciComp@FredHutch.org User Self-Service Portal: https://fredhutch.happyfox.com/saml/login/https://urldefense.proofpoint.com/v2/url?u=https-3A__fredhutch.happyfox.com_saml_login_&d=DwMDaQ&c=eRAMFD45gAfqt84VtBcfhfEazhEXT91ASHynm_9f1N0&r=GO1B7lbxFhbz62_4-qxrS34YhWYz26EIwt_QIcAjrTo&m=GrjUIUVzmM6TIWAN4-gw6Y8uxBqlQsWt15mDaM2NDlprRFxSH2cn82PiNrMXlHWM&s=K1ytqWaOOxNFGRwHD7y1pIqk74nv1uyy0KWtaLY1b8c&e=
Ticket update by Dan Tenenbaum (dtenenba@fredhutch.org)
Hello,
This is now running at https://dihiva.fredhutch.org/ .
Sorry it took a few days - we had some trouble provisioning a virtual machine.
Let me know if you have further questions or need any more assistance.
Dan
Ticket update by Dan Tenenbaum (dtenenba@fredhutch.org)
PS. You need to be on campus or using VPN for that link to work.
Ticket update by David Scalzo (scalzod@fredhutch.org)
Dan,
Thanks, from all of us!
I’ll run some datasets next week and let you know if any questions come up.
Best,
David Cohn lab FHCC
From: Dan Tenenbaum scicomp@fredhutch.org Date: Friday, April 1, 2022 at 13:43 To: Scalzo, David scalzod@fredhutch.org Cc: Cohn, Lillian B lcohn@fredhutch.org Subject: Re: #SC00007243: : : : : : : : DIHVA module creation New Reply for Ticket #SC00007243
Ticket Title: : : : : : : : DIHVA module creation
PS. You need to be on campus or using VPN for that link to work.
Dan Tenenbaum Cloud Systems Engineer, Scientific Computing Fred Hutch Pronouns: He/him Email: dtenenba@fredhutch.org
Thank you,
Scientific Computing (SciComp) SciComp@FredHutch.org User Self-Service Portal: https://fredhutch.happyfox.com/saml/login/https://urldefense.proofpoint.com/v2/url?u=https-3A__fredhutch.happyfox.com_saml_login_&d=DwMDaQ&c=eRAMFD45gAfqt84VtBcfhfEazhEXT91ASHynm_9f1N0&r=GO1B7lbxFhbz62_4-qxrS34YhWYz26EIwt_QIcAjrTo&m=4em-gI0ATM3b7kyGabjqpp7VbLLe4IA1O86YwaIJyy3QobW2CJPSfdjDH9jQXThx&s=kjAkgzDNAdFSOtGcldaKaZytz667mkwB3JLQOQLPTuo&e=
Ticket update by Lillian Cohn (lcohn@fredhutch.org)
Thank you so much, Dan! Looking forward to getting our experiments and analysis going using DIHIVA. We greatly appreciate your help with this!
Lillian
From: Scalzo, David scalzod@fredhutch.org Date: Friday, April 1, 2022 at 8:19 PM To: scicomp scicomp@fhcrc.org Cc: Cohn, Lillian B lcohn@fredhutch.org Subject: Re: #SC00007243: : : : : : : : DIHVA module creation Dan,
Thanks, from all of us!
I’ll run some datasets next week and let you know if any questions come up.
Best,
David Cohn lab FHCC
From: Dan Tenenbaum scicomp@fredhutch.org Date: Friday, April 1, 2022 at 13:43 To: Scalzo, David scalzod@fredhutch.org Cc: Cohn, Lillian B lcohn@fredhutch.org Subject: Re: #SC00007243: : : : : : : : DIHVA module creation New Reply for Ticket #SC00007243
Ticket Title: : : : : : : : DIHVA module creation
PS. You need to be on campus or using VPN for that link to work.
Dan Tenenbaum Cloud Systems Engineer, Scientific Computing Fred Hutch Pronouns: He/him Email: dtenenba@fredhutch.org
Thank you,
Scientific Computing (SciComp) SciComp@FredHutch.org User Self-Service Portal: https://fredhutch.happyfox.com/saml/login/https://urldefense.proofpoint.com/v2/url?u=https-3A__fredhutch.happyfox.com_saml_login_&d=DwMDaQ&c=eRAMFD45gAfqt84VtBcfhfEazhEXT91ASHynm_9f1N0&r=GO1B7lbxFhbz62_4-qxrS34YhWYz26EIwt_QIcAjrTo&m=4em-gI0ATM3b7kyGabjqpp7VbLLe4IA1O86YwaIJyy3QobW2CJPSfdjDH9jQXThx&s=kjAkgzDNAdFSOtGcldaKaZytz667mkwB3JLQOQLPTuo&e=
Ticket update by David Scalzo (scalzod@fredhutch.org)
Hi Dan,
I finally had time today to run some tests.
Where are the results ending up? I’m expecting something like what’s in this zip file.
Nested directories containing the output from each step in the analysis. When I run the first two demo files from the website I don’t see where the result ended up.
Thanks,
David
From: SciComp scicomp@fredhutch.org Date: Wednesday, April 6, 2022 at 14:14 To: Scalzo, David scalzod@fredhutch.org Subject: Ticket #SC00007243 is Closed - Responding w/in 3 days Reopens Ticket Ticket #SC00007243
Ticket Title: : : : : : : : : DIHVA module creation
This ticket has been resolved and Closed by Scientific Computing.
If you feel there are additional steps needed to fully resolve your issue, please respond to this email and this ticket to be reopened within 3 days.
This ticket is reopened if you respond with a Thank You.
Any response after 3 days will NOT REOPEN TICKET. It will be forwarded to the last agent responding to this ticket.
Scientific Computing (SciComp) SciComp@FredHutch.org User Self-Service Portal: https://fredhutch.happyfox.com/saml/login/https://urldefense.proofpoint.com/v2/url?u=https-3A__fredhutch.happyfox.com_saml_login_&d=DwMDaQ&c=eRAMFD45gAfqt84VtBcfhfEazhEXT91ASHynm_9f1N0&r=GO1B7lbxFhbz62_4-qxrS34YhWYz26EIwt_QIcAjrTo&m=8cURnlcXO0Zy3gx2jOVRruNj-kdoeMnPNJqAU3dynmtasdKVfurv6K2IfV7-eURD&s=GZ5y9o1MY-29rcSjObqRYfbEQx1Uz4U8YFfhGPq-w28&e=
Ticket update by Dan Tenenbaum (dtenenba@fredhutch.org)
Hi David,
I think it depends on which files you upload. When I uploaded the first two files on that dropbox site, my result was a zip file with a single HTML file in it.
Just now I submitted the following two files (towards the end of the listing on the dropbox site):
demonstration-file-wk12-4-C13_S243_L001_R1_001-1.fastq.gz
demonstration-file-wk12-4-C13_S243_L001_R2_001-1.fastq.gz
And this time I got a zip file that had the following folders in it:
I'm guessing that if I had submitted a different pair of files (or larger "real" data) that I would have more folders/steps in my output zip file?
Hope that helps,
Dan
Ticket update by David Scalzo (scalzod@fredhutch.org)
Hi Dan,
Yes, you are correct.
Depending on the sequencing results Diviha may or may not produce results for all of the 8 steps.
We’re working on our own library preps for sequencing. I may have more questions once we run some of our own data.
Again, thanks for setting this up. It will help us a lot with our research.
Best,
David
From: Dan Tenenbaum scicomp@fredhutch.org Date: Wednesday, April 6, 2022 at 15:40 To: Scalzo, David scalzod@fredhutch.org Subject: Re: #SC00007243: Ticket is Closed - Responding w/in 3 days Reopens Ticket New Reply for Ticket #SC00007243
Ticket Title: Ticket is Closed - Responding w/in 3 days Reopens Ticket
Hi David,
I think it depends on which files you upload. When I uploaded the first two files on that dropbox site, my result was a zip file with a single HTML file in it.
Just now I submitted the following two files (towards the end of the listing on the dropbox site):
demonstration-file-wk12-4-C13_S243_L001_R1_001-1.fastq.gz demonstration-file-wk12-4-C13_S243_L001_R2_001-1.fastq.gz
And this time I got a zip file that had the following folders in it:
[cid:22575] I'm guessing that if I had submitted a different pair of files (or larger "real" data) that I would have more folders/steps in my output zip file?
Hope that helps, Dan
Dan Tenenbaum Cloud Systems Engineer, Scientific Computing Fred Hutch Pronouns: He/him Email: dtenenba@fredhutch.org
Thank you,
Scientific Computing (SciComp) SciComp@FredHutch.org User Self-Service Portal: https://fredhutch.happyfox.com/saml/login/https://urldefense.proofpoint.com/v2/url?u=https-3A__fredhutch.happyfox.com_saml_login_&d=DwMDaQ&c=eRAMFD45gAfqt84VtBcfhfEazhEXT91ASHynm_9f1N0&r=GO1B7lbxFhbz62_4-qxrS34YhWYz26EIwt_QIcAjrTo&m=zbwJNuaeDgJ9YqX1tAkyYKZQs1MubRTQjy_Exaamf3TMJFm3jkYBXQCWdbMjFcqe&s=tXoqdEul3BwwBP_QYFEWsmCdrhDUfDE4jeGLZSKZArs&e=
Ticket update by Dan Tenenbaum (dtenenba@fredhutch.org)
OK, feel free to ask any questions. I don't have any special expertise with DIHIVA so if the question is specific to that, I might have to pass it along to them. But if something in the setup is not working right I can definitely take a look.
I will close this ticket now but just send another email to scicomp@fredhutch.org if anything else comes up.
Thanks,
Dan
closing issue
We would like this analysis pipeline available as a module.
The DIHIVA pipeline is publicly available on GitHub and we created a docker image that can be used on Linux, Mac, and Windows. https://github.com/stratust/DIHIVAhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_stratust_DIHIVA&d=DwMFaQ&c=eRAMFD45gAfqt84VtBcfhQ&r=oMVAMb2E0frjc0yVw6uft-LwUARPPbYpzbKjEZGRF3E&m=Y75RAkl1cxGYTemWyJQmtTGWiFDD5rtNxiyxS_1oW0A&s=4tvpO8HGtAwjVGYzPqYm_R0FKAeo0t66lr37AMJY5ZU&e=
Ticket Custom Fields: SciComp Ticket Class: