Closed cuttlefishh closed 9 years ago
Minor, but we should change the script to use spaces instead of tabs for indentation.
On (May-15-15|18:31), Luke Thompson wrote:
All new PCoAs and legends for bottom half of Module 2 report, including latex and matplotlib code. Double-check that pipeline-generated macros files still work (I had to change the dummy macros_gut.tex file but I think this was due to changes already in the repo). Still need to tweak a few pieces of the latex and python script. Also need to add the actual rendering of the PCoAs via the pipeline, making sure the right options and file paths are invoked. You can view, comment on, or merge this pull request online at:
https://github.com/biocore/American-Gut/pull/150
-- Commit Summary --
- All new PCoAs and legends for bottom half of Module 2 report, including latex and matplotlib code
-- File Changes --
M latex/macros_gut.tex (12) M latex/pdfs-gut/figure1.pdf (0) A latex/pdfs-gut/figure1_legend.ai (416) M latex/pdfs-gut/figure1_legend.pdf (0) M latex/pdfs-gut/figure2.pdf (0) A latex/pdfs-gut/figure2_legend.ai (365) M latex/pdfs-gut/figure2_legend.pdf (0) M latex/pdfs-gut/figure3.pdf (0) A latex/pdfs-gut/figure3_legend.ai (451) M latex/pdfs-gut/figure3_legend.pdf (0) A latex/pdfs-gut/figure3_legend_double_gradient.ai (451) A latex/pdfs-gut/figure3_legend_double_gradient.pdf (0) M latex/template_gut.tex (33) A scripts/mod2_pcoa.py (276)
-- Patch Links --
https://github.com/biocore/American-Gut/pull/150.patch https://github.com/biocore/American-Gut/pull/150.diff
Reply to this email directly or view it on GitHub: https://github.com/biocore/American-Gut/pull/150
Cool. I will include spaces (not tabs) when I push the next set of edits.
On May 17, 2015, at 1:25 PM, Yoshiki Vázquez Baeza notifications@github.com wrote:
Minor, but we should change the script to use spaces instead of tabs for indentation.
On (May-15-15|18:31), Luke Thompson wrote:
All new PCoAs and legends for bottom half of Module 2 report, including latex and matplotlib code. Double-check that pipeline-generated macros files still work (I had to change the dummy macros_gut.tex file but I think this was due to changes already in the repo). Still need to tweak a few pieces of the latex and python script. Also need to add the actual rendering of the PCoAs via the pipeline, making sure the right options and file paths are invoked. You can view, comment on, or merge this pull request online at:
https://github.com/biocore/American-Gut/pull/150
-- Commit Summary --
- All new PCoAs and legends for bottom half of Module 2 report, including latex and matplotlib code
-- File Changes --
M latex/macros_gut.tex (12) M latex/pdfs-gut/figure1.pdf (0) A latex/pdfs-gut/figure1_legend.ai (416) M latex/pdfs-gut/figure1_legend.pdf (0) M latex/pdfs-gut/figure2.pdf (0) A latex/pdfs-gut/figure2_legend.ai (365) M latex/pdfs-gut/figure2_legend.pdf (0) M latex/pdfs-gut/figure3.pdf (0) A latex/pdfs-gut/figure3_legend.ai (451) M latex/pdfs-gut/figure3_legend.pdf (0) A latex/pdfs-gut/figure3_legend_double_gradient.ai (451) A latex/pdfs-gut/figure3_legend_double_gradient.pdf (0) M latex/template_gut.tex (33) A scripts/mod2_pcoa.py (276)
-- Patch Links --
https://github.com/biocore/American-Gut/pull/150.patch https://github.com/biocore/American-Gut/pull/150.diff
Reply to this email directly or view it on GitHub: https://github.com/biocore/American-Gut/pull/150 — Reply to this email directly or view it on GitHub https://github.com/biocore/American-Gut/pull/150#issuecomment-102848142.
The spaces/tabs thing is required, it plays hell with the codebase if its not consistent. There are a large number of pep-8 failures btw
Able to address the flake8 issues?
I might need some assistance on the pep-8/flake8 issues. Maybe @ElDeveloper can help this morning.
I can help you out!
On (May-18-15| 8:31), Luke Thompson wrote:
I might need some assistance on the pep-8/flake8 issues. Maybe @ElDeveloper can help this morning.
Reply to this email directly or view it on GitHub: https://github.com/biocore/American-Gut/pull/150#issuecomment-103105345
I'm working on it. Didn't know what flake8 was, but Zech told me, I pip installed that shiznit, and am working through it now. Cool way to clean up code!
OK, the latest updates should fix all known issues:
Thanks to @ElDeveloper and @amnona for assistance and suggestions. @JWDebelius suggested to use alpha diversity instead of firmicutes for Figure 3, but that may have to wait (couldn't find it in any mapping file).
two flake8 issues still FYI
This looks great though. Getting the processing notebook setup to support it now.
I don't get any flake8 issues when I run it on my laptop. Is there some version of flake8 that github runs?
Should be using the latest version as no version is specified. Can you try doing:
pip install -U flake8
...and then rerun?
I ran that update, updated from 2.4.0 to 2.4.1, but still no issues identified. Are you just running with default options?
On May 19, 2015, at 9:25 AM, Daniel McDonald notifications@github.com wrote:
Should be using the latest version as no version is specified. Can you try doing:
pip install -U flake8 ...and then rerun?
— Reply to this email directly or view it on GitHub https://github.com/biocore/American-Gut/pull/150#issuecomment-103578436.
Just a note, I've seen this behavior, when I can't reproduce the flake8 errors locally. I think it's because the flake8 version installed with conda is different from the one in pip, but this is just an idea, no tests....
@cuttlefishh, just look at the travis output in the PR. It'll show the reason the build fails
OK, below is the line (used twice) creating the error. The error is "line break before binary operator". If I put it on the same line, it's more than 80 characters. Not sure where I can break it legally. Other option is I'm sure I can make the line shorter using a logical "or" as in "Malawi or Venezuela", but I am still new to Python logical structures. Any suggestions from the experts?
if (mf.loc[sample]['COUNTRY'] != 'Malawi') & (mf.loc[sample]['COUNTRY']
!= 'Venezuela'):
On May 19, 2015, at 10:15 AM, Daniel McDonald notifications@github.com wrote:
@cuttlefishh https://github.com/cuttlefishh, just look at the travis output in the PR. It'll show the reason the build fails
— Reply to this email directly or view it on GitHub https://github.com/biocore/American-Gut/pull/150#issuecomment-103598759.
if ((mf.loc[sample]['COUNTRY'] != 'Malawi') &
(mf.loc[sample]['COUNTRY'] != 'Venezuela')):
Thanks Jose, but that gives me an "invalid syntax" error.
On May 19, 2015, at 10:25 AM, josenavas notifications@github.com wrote:
if (mf.loc[sample]['COUNTRY'] != 'Malawi') & (mf.loc[sample]['COUNTRY'] != 'Venezuela'): — Reply to this email directly or view it on GitHub https://github.com/biocore/American-Gut/pull/150#issuecomment-103601974.
This does not give me an error, but still don't know how it will go on github.
if (mf.loc[sample]['COUNTRY'] != 'Malawi') & (
mf.loc[sample]['COUNTRY'] != 'Venezuela'):
On May 19, 2015, at 10:28 AM, Luke Thompson lukethompson@gmail.com wrote:
Thanks Jose, but that gives me an "invalid syntax" error.
On May 19, 2015, at 10:25 AM, josenavas <notifications@github.com mailto:notifications@github.com> wrote:
if (mf.loc[sample]['COUNTRY'] != 'Malawi') & (mf.loc[sample]['COUNTRY'] != 'Venezuela'): — Reply to this email directly or view it on GitHub https://github.com/biocore/American-Gut/pull/150#issuecomment-103601974.
That should make it. Note that I've updated my comment just after I send it, cause I saw the syntax error (I just wrapped everything in ()
, but your current solution shouldn't be an issue.
Cool, thanks! Still learning :)
On May 19, 2015, at 10:37 AM, josenavas notifications@github.com wrote:
That should make it. Note that I've updated my comment just after I send it, cause I saw the syntax error (I just wrapped everything in (), but your current solution shouldn't be an issue.
— Reply to this email directly or view it on GitHub https://github.com/biocore/American-Gut/pull/150#issuecomment-103609202.
Ah, damn. Merged this prematurely. When looking at integrating, there are two things missing. First is that it needs an output directory. Second, it needs the ability to specify a prefix for the output files. I'll issue a separate PR on that shortly though.
Yeah, I discussed that with @eldeveloper but we decided you would figure it out, as we didn't know how it would fit in with the rest of the processing pipeline. Thanks!
On May 19, 2015, at 3:47 PM, Daniel McDonald notifications@github.com wrote:
Ah, damn. Merged this prematurely. When looking at integrating, there are two things missing. First is that it needs an output directory. Second, it needs the ability to specify a prefix for the output files. I'll issue a separate PR on that shortly though.
— Reply to this email directly or view it on GitHub.
Please describe incomplete PRs as a work in progress and explicitly enumerate what needs to be done in these instances.
What version of skbio did you write this against? The methods used do not exist.
scikit-bio==0.2.3
How difficult would it be to specify what samples are printed when called? Issue is that this script is slow and cannot be parallelized in its current form (at least, specifying a reduced mapping file causes it to break). I'm currently projecting 16 hours to produce the results for figure 1.
nm, ill adjust it
Yeah, we realized that it wasn't necessary to generate reports for HMP and GG, but didn't institute any way to filter those out. Sounds like you got it working.
On May 20, 2015, at 2:05 PM, Daniel McDonald notifications@github.com wrote:
nm, ill adjust it
— Reply to this email directly or view it on GitHub https://github.com/biocore/American-Gut/pull/150#issuecomment-104039065.
No, still can't get the code to actually run yet...
On Wed, May 20, 2015 at 3:14 PM, Luke Thompson notifications@github.com wrote:
Yeah, we realized that it wasn't necessary to generate reports for HMP and GG, but didn't institute any way to filter those out. Sounds like you got it working.
On May 20, 2015, at 2:05 PM, Daniel McDonald notifications@github.com wrote:
nm, ill adjust it
— Reply to this email directly or view it on GitHub < https://github.com/biocore/American-Gut/pull/150#issuecomment-104039065>.
— Reply to this email directly or view it on GitHub https://github.com/biocore/American-Gut/pull/150#issuecomment-104041042.
If speed of Figure 1 is the slowest part, you can speed that up by changing "50" in lines 60 and 63 to "20". That will slice each category into 20 slices instead of 50, so there are fewer sets to plot.
On May 20, 2015, at 2:15 PM, Daniel McDonald notifications@github.com wrote:
No, still can't get the code to actually run yet...
On Wed, May 20, 2015 at 3:14 PM, Luke Thompson notifications@github.com wrote:
Yeah, we realized that it wasn't necessary to generate reports for HMP and GG, but didn't institute any way to filter those out. Sounds like you got it working.
On May 20, 2015, at 2:05 PM, Daniel McDonald notifications@github.com wrote:
nm, ill adjust it
— Reply to this email directly or view it on GitHub < https://github.com/biocore/American-Gut/pull/150#issuecomment-104039065>.
— Reply to this email directly or view it on GitHub https://github.com/biocore/American-Gut/pull/150#issuecomment-104041042.
— Reply to this email directly or view it on GitHub https://github.com/biocore/American-Gut/pull/150#issuecomment-104041289.
If that decreases the runtime by 10x then it would be acceptable, what's your take there?
On Wed, May 20, 2015 at 3:20 PM, Luke Thompson notifications@github.com wrote:
If speed of Figure 1 is the slowest part, you can speed that up by changing "50" in lines 60 and 63 to "20". That will slice each category into 20 slices instead of 50, so there are fewer sets to plot.
On May 20, 2015, at 2:15 PM, Daniel McDonald notifications@github.com wrote:
No, still can't get the code to actually run yet...
On Wed, May 20, 2015 at 3:14 PM, Luke Thompson <notifications@github.com
wrote:
Yeah, we realized that it wasn't necessary to generate reports for HMP and GG, but didn't institute any way to filter those out. Sounds like you got it working.
On May 20, 2015, at 2:05 PM, Daniel McDonald < notifications@github.com> wrote:
nm, ill adjust it
— Reply to this email directly or view it on GitHub <
https://github.com/biocore/American-Gut/pull/150#issuecomment-104039065>.
— Reply to this email directly or view it on GitHub < https://github.com/biocore/American-Gut/pull/150#issuecomment-104041042>.
— Reply to this email directly or view it on GitHub < https://github.com/biocore/American-Gut/pull/150#issuecomment-104041289>.
— Reply to this email directly or view it on GitHub https://github.com/biocore/American-Gut/pull/150#issuecomment-104042759.
Let me test that quickly on my laptop (I'm assuming it will scale on pando/compy). We can reduce the number of slices even lower, but as we approach 1 we start to get points from one group (e.g. HMP fecal) covering up those from another group (e.g. AG fecal).
On May 20, 2015, at 2:21 PM, Daniel McDonald notifications@github.com wrote:
If that decreases the runtime by 10x then it would be acceptable, what's your take there?
On Wed, May 20, 2015 at 3:20 PM, Luke Thompson notifications@github.com wrote:
If speed of Figure 1 is the slowest part, you can speed that up by changing "50" in lines 60 and 63 to "20". That will slice each category into 20 slices instead of 50, so there are fewer sets to plot.
On May 20, 2015, at 2:15 PM, Daniel McDonald notifications@github.com wrote:
No, still can't get the code to actually run yet...
On Wed, May 20, 2015 at 3:14 PM, Luke Thompson <notifications@github.com
wrote:
Yeah, we realized that it wasn't necessary to generate reports for HMP and GG, but didn't institute any way to filter those out. Sounds like you got it working.
On May 20, 2015, at 2:05 PM, Daniel McDonald < notifications@github.com> wrote:
nm, ill adjust it
— Reply to this email directly or view it on GitHub <
https://github.com/biocore/American-Gut/pull/150#issuecomment-104039065>.
— Reply to this email directly or view it on GitHub < https://github.com/biocore/American-Gut/pull/150#issuecomment-104041042>.
— Reply to this email directly or view it on GitHub < https://github.com/biocore/American-Gut/pull/150#issuecomment-104041289>.
— Reply to this email directly or view it on GitHub https://github.com/biocore/American-Gut/pull/150#issuecomment-104042759.
— Reply to this email directly or view it on GitHub https://github.com/biocore/American-Gut/pull/150#issuecomment-104043086.
If we set it to 10 slices, it takes ~1.5 seconds per plot on my laptop. Compare that to ~3 seconds per plot for 50 slices. It still looks OK but not quite as well intercalated, and there's more plot-to-plot variation. That's a 2x gain, not 10x. Is that enough to be worth it?
On May 20, 2015, at 2:24 PM, Luke Thompson lukethompson@gmail.com wrote:
Let me test that quickly on my laptop (I'm assuming it will scale on pando/compy). We can reduce the number of slices even lower, but as we approach 1 we start to get points from one group (e.g. HMP fecal) covering up those from another group (e.g. AG fecal).
On May 20, 2015, at 2:21 PM, Daniel McDonald <notifications@github.com mailto:notifications@github.com> wrote:
If that decreases the runtime by 10x then it would be acceptable, what's your take there?
On Wed, May 20, 2015 at 3:20 PM, Luke Thompson <notifications@github.com mailto:notifications@github.com> wrote:
If speed of Figure 1 is the slowest part, you can speed that up by changing "50" in lines 60 and 63 to "20". That will slice each category into 20 slices instead of 50, so there are fewer sets to plot.
On May 20, 2015, at 2:15 PM, Daniel McDonald <notifications@github.com mailto:notifications@github.com> wrote:
No, still can't get the code to actually run yet...
On Wed, May 20, 2015 at 3:14 PM, Luke Thompson <notifications@github.com mailto:notifications@github.com
wrote:
Yeah, we realized that it wasn't necessary to generate reports for HMP and GG, but didn't institute any way to filter those out. Sounds like you got it working.
On May 20, 2015, at 2:05 PM, Daniel McDonald < notifications@github.com mailto:notifications@github.com> wrote:
nm, ill adjust it
— Reply to this email directly or view it on GitHub <
https://github.com/biocore/American-Gut/pull/150#issuecomment-104039065 https://github.com/biocore/American-Gut/pull/150#issuecomment-104039065>.
— Reply to this email directly or view it on GitHub < https://github.com/biocore/American-Gut/pull/150#issuecomment-104041042 https://github.com/biocore/American-Gut/pull/150#issuecomment-104041042>.
— Reply to this email directly or view it on GitHub < https://github.com/biocore/American-Gut/pull/150#issuecomment-104041289 https://github.com/biocore/American-Gut/pull/150#issuecomment-104041289>.
— Reply to this email directly or view it on GitHub <https://github.com/biocore/American-Gut/pull/150#issuecomment-104042759 https://github.com/biocore/American-Gut/pull/150#issuecomment-104042759>.
— Reply to this email directly or view it on GitHub https://github.com/biocore/American-Gut/pull/150#issuecomment-104043086.
Drops it to ~8 hours by present estimates. Figured out the juggling necessary to specify samples though, issuing a PR momentarily
On Wed, May 20, 2015 at 3:32 PM, Luke Thompson notifications@github.com wrote:
If we set it to 10 slices, it takes ~1.5 seconds per plot on my laptop. Compare that to ~3 seconds per plot for 50 slices. It still looks OK but not quite as well intercalated, and there's more plot-to-plot variation. That's a 2x gain, not 10x. Is that enough to be worth it?
On May 20, 2015, at 2:24 PM, Luke Thompson lukethompson@gmail.com wrote:
Let me test that quickly on my laptop (I'm assuming it will scale on pando/compy). We can reduce the number of slices even lower, but as we approach 1 we start to get points from one group (e.g. HMP fecal) covering up those from another group (e.g. AG fecal).
On May 20, 2015, at 2:21 PM, Daniel McDonald <notifications@github.com mailto:notifications@github.com> wrote:
If that decreases the runtime by 10x then it would be acceptable, what's your take there?
On Wed, May 20, 2015 at 3:20 PM, Luke Thompson < notifications@github.com mailto:notifications@github.com> wrote:
If speed of Figure 1 is the slowest part, you can speed that up by changing "50" in lines 60 and 63 to "20". That will slice each category into 20 slices instead of 50, so there are fewer sets to plot.
On May 20, 2015, at 2:15 PM, Daniel McDonald < notifications@github.com mailto:notifications@github.com> wrote:
No, still can't get the code to actually run yet...
On Wed, May 20, 2015 at 3:14 PM, Luke Thompson < notifications@github.com mailto:notifications@github.com
wrote:
Yeah, we realized that it wasn't necessary to generate reports for HMP and GG, but didn't institute any way to filter those out. Sounds like you got it working.
On May 20, 2015, at 2:05 PM, Daniel McDonald < notifications@github.com mailto:notifications@github.com> wrote:
nm, ill adjust it
— Reply to this email directly or view it on GitHub <
https://github.com/biocore/American-Gut/pull/150#issuecomment-104039065 < https://github.com/biocore/American-Gut/pull/150#issuecomment-104039065>>.
— Reply to this email directly or view it on GitHub <
https://github.com/biocore/American-Gut/pull/150#issuecomment-104041042 < https://github.com/biocore/American-Gut/pull/150#issuecomment-104041042>>.
— Reply to this email directly or view it on GitHub <
https://github.com/biocore/American-Gut/pull/150#issuecomment-104041289 < https://github.com/biocore/American-Gut/pull/150#issuecomment-104041289>>.
— Reply to this email directly or view it on GitHub < https://github.com/biocore/American-Gut/pull/150#issuecomment-104042759 < https://github.com/biocore/American-Gut/pull/150#issuecomment-104042759>>.
— Reply to this email directly or view it on GitHub < https://github.com/biocore/American-Gut/pull/150#issuecomment-104043086>.
— Reply to this email directly or view it on GitHub https://github.com/biocore/American-Gut/pull/150#issuecomment-104045555.
The challenge with using a reduced mapping file, I think, is the script assumes the sample IDs are the same (and in the same order) in the mapping file and PC file.
On May 20, 2015, at 2:32 PM, Luke Thompson lukethompson@gmail.com wrote:
If we set it to 10 slices, it takes ~1.5 seconds per plot on my laptop. Compare that to ~3 seconds per plot for 50 slices. It still looks OK but not quite as well intercalated, and there's more plot-to-plot variation. That's a 2x gain, not 10x. Is that enough to be worth it?
On May 20, 2015, at 2:24 PM, Luke Thompson <lukethompson@gmail.com mailto:lukethompson@gmail.com> wrote:
Let me test that quickly on my laptop (I'm assuming it will scale on pando/compy). We can reduce the number of slices even lower, but as we approach 1 we start to get points from one group (e.g. HMP fecal) covering up those from another group (e.g. AG fecal).
On May 20, 2015, at 2:21 PM, Daniel McDonald <notifications@github.com mailto:notifications@github.com> wrote:
If that decreases the runtime by 10x then it would be acceptable, what's your take there?
On Wed, May 20, 2015 at 3:20 PM, Luke Thompson <notifications@github.com mailto:notifications@github.com> wrote:
If speed of Figure 1 is the slowest part, you can speed that up by changing "50" in lines 60 and 63 to "20". That will slice each category into 20 slices instead of 50, so there are fewer sets to plot.
On May 20, 2015, at 2:15 PM, Daniel McDonald <notifications@github.com mailto:notifications@github.com> wrote:
No, still can't get the code to actually run yet...
On Wed, May 20, 2015 at 3:14 PM, Luke Thompson <notifications@github.com mailto:notifications@github.com
wrote:
Yeah, we realized that it wasn't necessary to generate reports for HMP and GG, but didn't institute any way to filter those out. Sounds like you got it working.
On May 20, 2015, at 2:05 PM, Daniel McDonald < notifications@github.com mailto:notifications@github.com> wrote:
nm, ill adjust it
— Reply to this email directly or view it on GitHub <
https://github.com/biocore/American-Gut/pull/150#issuecomment-104039065 https://github.com/biocore/American-Gut/pull/150#issuecomment-104039065>.
— Reply to this email directly or view it on GitHub < https://github.com/biocore/American-Gut/pull/150#issuecomment-104041042 https://github.com/biocore/American-Gut/pull/150#issuecomment-104041042>.
— Reply to this email directly or view it on GitHub < https://github.com/biocore/American-Gut/pull/150#issuecomment-104041289 https://github.com/biocore/American-Gut/pull/150#issuecomment-104041289>.
— Reply to this email directly or view it on GitHub <https://github.com/biocore/American-Gut/pull/150#issuecomment-104042759 https://github.com/biocore/American-Gut/pull/150#issuecomment-104042759>.
— Reply to this email directly or view it on GitHub https://github.com/biocore/American-Gut/pull/150#issuecomment-104043086.
Great. In that case, I would prefer to keep it at 50 slices because it looks much better. Or compromise and set it to 25.
On May 20, 2015, at 2:37 PM, Daniel McDonald notifications@github.com wrote:
Drops it to ~8 hours by present estimates. Figured out the juggling necessary to specify samples though, issuing a PR momentarily
On Wed, May 20, 2015 at 3:32 PM, Luke Thompson notifications@github.com wrote:
If we set it to 10 slices, it takes ~1.5 seconds per plot on my laptop. Compare that to ~3 seconds per plot for 50 slices. It still looks OK but not quite as well intercalated, and there's more plot-to-plot variation. That's a 2x gain, not 10x. Is that enough to be worth it?
On May 20, 2015, at 2:24 PM, Luke Thompson lukethompson@gmail.com wrote:
Let me test that quickly on my laptop (I'm assuming it will scale on pando/compy). We can reduce the number of slices even lower, but as we approach 1 we start to get points from one group (e.g. HMP fecal) covering up those from another group (e.g. AG fecal).
On May 20, 2015, at 2:21 PM, Daniel McDonald <notifications@github.com mailto:notifications@github.com> wrote:
If that decreases the runtime by 10x then it would be acceptable, what's your take there?
On Wed, May 20, 2015 at 3:20 PM, Luke Thompson < notifications@github.com mailto:notifications@github.com> wrote:
If speed of Figure 1 is the slowest part, you can speed that up by changing "50" in lines 60 and 63 to "20". That will slice each category into 20 slices instead of 50, so there are fewer sets to plot.
On May 20, 2015, at 2:15 PM, Daniel McDonald < notifications@github.com mailto:notifications@github.com> wrote:
No, still can't get the code to actually run yet...
On Wed, May 20, 2015 at 3:14 PM, Luke Thompson < notifications@github.com mailto:notifications@github.com
wrote:
Yeah, we realized that it wasn't necessary to generate reports for HMP and GG, but didn't institute any way to filter those out. Sounds like you got it working.
On May 20, 2015, at 2:05 PM, Daniel McDonald < notifications@github.com mailto:notifications@github.com> wrote:
nm, ill adjust it
— Reply to this email directly or view it on GitHub <
https://github.com/biocore/American-Gut/pull/150#issuecomment-104039065 < https://github.com/biocore/American-Gut/pull/150#issuecomment-104039065>>.
— Reply to this email directly or view it on GitHub <
https://github.com/biocore/American-Gut/pull/150#issuecomment-104041042 < https://github.com/biocore/American-Gut/pull/150#issuecomment-104041042>>.
— Reply to this email directly or view it on GitHub <
https://github.com/biocore/American-Gut/pull/150#issuecomment-104041289 < https://github.com/biocore/American-Gut/pull/150#issuecomment-104041289>>.
— Reply to this email directly or view it on GitHub < https://github.com/biocore/American-Gut/pull/150#issuecomment-104042759 < https://github.com/biocore/American-Gut/pull/150#issuecomment-104042759>>.
— Reply to this email directly or view it on GitHub < https://github.com/biocore/American-Gut/pull/150#issuecomment-104043086>.
— Reply to this email directly or view it on GitHub https://github.com/biocore/American-Gut/pull/150#issuecomment-104045555.
— Reply to this email directly or view it on GitHub https://github.com/biocore/American-Gut/pull/150#issuecomment-104046883.
If we’re talking about a 1 day difference, it’s worth it to make them nicer as this is already so much later than we wanted…
On May 20, 2015, at 2:39 PM, Luke Thompson notifications@github.com wrote:
Great. In that case, I would prefer to keep it at 50 slices because it looks much better. Or compromise and set it to 25.
On May 20, 2015, at 2:37 PM, Daniel McDonald notifications@github.com wrote:
Drops it to ~8 hours by present estimates. Figured out the juggling necessary to specify samples though, issuing a PR momentarily
On Wed, May 20, 2015 at 3:32 PM, Luke Thompson notifications@github.com wrote:
If we set it to 10 slices, it takes ~1.5 seconds per plot on my laptop. Compare that to ~3 seconds per plot for 50 slices. It still looks OK but not quite as well intercalated, and there's more plot-to-plot variation. That's a 2x gain, not 10x. Is that enough to be worth it?
On May 20, 2015, at 2:24 PM, Luke Thompson lukethompson@gmail.com wrote:
Let me test that quickly on my laptop (I'm assuming it will scale on pando/compy). We can reduce the number of slices even lower, but as we approach 1 we start to get points from one group (e.g. HMP fecal) covering up those from another group (e.g. AG fecal).
On May 20, 2015, at 2:21 PM, Daniel McDonald <notifications@github.com mailto:notifications@github.com> wrote:
If that decreases the runtime by 10x then it would be acceptable, what's your take there?
On Wed, May 20, 2015 at 3:20 PM, Luke Thompson < notifications@github.com mailto:notifications@github.com> wrote:
If speed of Figure 1 is the slowest part, you can speed that up by changing "50" in lines 60 and 63 to "20". That will slice each category into 20 slices instead of 50, so there are fewer sets to plot.
On May 20, 2015, at 2:15 PM, Daniel McDonald < notifications@github.com mailto:notifications@github.com> wrote:
No, still can't get the code to actually run yet...
On Wed, May 20, 2015 at 3:14 PM, Luke Thompson < notifications@github.com mailto:notifications@github.com
wrote:
Yeah, we realized that it wasn't necessary to generate reports for HMP and GG, but didn't institute any way to filter those out. Sounds like you got it working.
On May 20, 2015, at 2:05 PM, Daniel McDonald < notifications@github.com mailto:notifications@github.com> wrote:
nm, ill adjust it
— Reply to this email directly or view it on GitHub <
https://github.com/biocore/American-Gut/pull/150#issuecomment-104039065 < https://github.com/biocore/American-Gut/pull/150#issuecomment-104039065>>.
— Reply to this email directly or view it on GitHub <
https://github.com/biocore/American-Gut/pull/150#issuecomment-104041042 < https://github.com/biocore/American-Gut/pull/150#issuecomment-104041042>>.
— Reply to this email directly or view it on GitHub <
https://github.com/biocore/American-Gut/pull/150#issuecomment-104041289 < https://github.com/biocore/American-Gut/pull/150#issuecomment-104041289>>.
— Reply to this email directly or view it on GitHub < https://github.com/biocore/American-Gut/pull/150#issuecomment-104042759 < https://github.com/biocore/American-Gut/pull/150#issuecomment-104042759>>.
— Reply to this email directly or view it on GitHub < https://github.com/biocore/American-Gut/pull/150#issuecomment-104043086>.
— Reply to this email directly or view it on GitHub https://github.com/biocore/American-Gut/pull/150#issuecomment-104045555.
— Reply to this email directly or view it on GitHub https://github.com/biocore/American-Gut/pull/150#issuecomment-104046883.
— Reply to this email directly or view it on GitHub https://github.com/biocore/American-Gut/pull/150#issuecomment-104047196.
Just issued #152 which allows this to be done in parallel. Walltime is now managable
Looks good. One nice thing about passing sample IDs as a parameter is we can quickly test known samples as a sanity check. Probably worth doing before we deploy for the whole AG.
On May 20, 2015, at 2:46 PM, Daniel McDonald notifications@github.com wrote:
Just issued #152 https://github.com/biocore/American-Gut/pull/152 which allows this to be done in parallel. Walltime is now managable
— Reply to this email directly or view it on GitHub https://github.com/biocore/American-Gut/pull/150#issuecomment-104048467.
That is a definite benefit as well
...hooking it up now though
All new PCoAs and legends for bottom half of Module 2 report, including latex and matplotlib code. Double-check that pipeline-generated macros files still work (I had to change the dummy macros_gut.tex file but I think this was due to changes already in the repo). Still need to tweak a few pieces of the latex and python script. Also need to add the actual rendering of the PCoAs via the pipeline, making sure the right options and file paths are invoked.