Update emperor

[antgonza]

This closes #40, resolves #35, solves #12, and closes #6

[ElDeveloper]

I'll be taking a look at this PR, however I think I won't be able to do it for the next few hours as I'm re-downloading the repository and I am on a very slow internet connection.

Just wanted to give a heads up to all the other devs.

El 05/08/2013, a las 10:36, Antonio Gonzalez notifications@github.com escribió:

This closes #40, resolves #35, solves #12, and closes #6

You can merge this Pull Request by running

git pull https://github.com/antgonza/evident update_emperor Or view, comment on, or merge it at:

https://github.com/qiime/evident/pull/44

Commit Summary

This closes #40, resolves #35, solves #12, and closes #6 removing files Removing tests File Changes

A UPDATE_EMPEROR.md (9) D evident/evident.py (127) M evident/pcoa.py (157) D tests/test_evident.py (344) D tests/test_pcoa.py (159) D www/css/images/.svn/entries (470) D www/css/images/.svn/prop-base/ui-bg_flat_0_aaaaaa_40x100.png.svn-base (9) D www/css/images/.svn/prop-base/ui-bg_flat_75_ffffff_40x100.png.svn-base (9) D www/css/images/.svn/prop-base/ui-bg_glass_55_fbf9ee_1x400.png.svn-base (9) D www/css/images/.svn/prop-base/ui-bg_glass_65_ffffff_1x400.png.svn-base (9) D www/css/images/.svn/prop-base/ui-bg_glass_75_dadada_1x400.png.svn-base (9) D www/css/images/.svn/prop-base/ui-bg_glass_75_e6e6e6_1x400.png.svn-base (9) D www/css/images/.svn/prop-base/ui-bg_glass_95_fef1ec_1x400.png.svn-base (9) D www/css/images/.svn/prop-base/ui-bg_highlight-soft_75_cccccc_1x100.png.svn-base (9) D www/css/images/.svn/prop-base/ui-icons_222222_256x240.png.svn-base (9) D www/css/images/.svn/prop-base/ui-icons_2e83ff_256x240.png.svn-base (9) D www/css/images/.svn/prop-base/ui-icons_454545_256x240.png.svn-base (9) D www/css/images/.svn/prop-base/ui-icons_888888_256x240.png.svn-base (9) D www/css/images/.svn/prop-base/ui-icons_cd0a0a_256x240.png.svn-base (9) D www/css/images/.svn/text-base/ui-bg_flat_0_aaaaaa_40x100.png.svn-base (0) D www/css/images/.svn/text-base/ui-bg_flat_75_ffffff_40x100.png.svn-base (0) D www/css/images/.svn/text-base/ui-bg_glass_55_fbf9ee_1x400.png.svn-base (0) D www/css/images/.svn/text-base/ui-bg_glass_65_ffffff_1x400.png.svn-base (0) D www/css/images/.svn/text-base/ui-bg_glass_75_dadada_1x400.png.svn-base (0) D www/css/images/.svn/text-base/ui-bg_glass_75_e6e6e6_1x400.png.svn-base (0) D www/css/images/.svn/text-base/ui-bg_glass_95_fef1ec_1x400.png.svn-base (0) D www/css/images/.svn/text-base/ui-bg_highlight-soft_75_cccccc_1x100.png.svn-base (0) D www/css/images/.svn/text-base/ui-icons_222222_256x240.png.svn-base (0) D www/css/images/.svn/text-base/ui-icons_2e83ff_256x240.png.svn-base (0) D www/css/images/.svn/text-base/ui-icons_454545_256x240.png.svn-base (0) D www/css/images/.svn/text-base/ui-icons_888888_256x240.png.svn-base (0) D www/css/images/.svn/text-base/ui-icons_cd0a0a_256x240.png.svn-base (0) D www/emperor/css/.svn/entries (62) D www/emperor/css/.svn/text-base/emperor.css.svn-base (210) D www/emperor/emperor.html (115) D www/emperor/emperor_head.html (21) D www/emperor/img/.svn/entries (130) D www/emperor/img/.svn/prop-base/pause.png.svn-base (5) D www/emperor/img/.svn/prop-base/play.png.svn-base (5) D www/emperor/img/.svn/prop-base/reset.png.svn-base (5) D www/emperor/img/.svn/text-base/pause.png.svn-base (0) D www/emperor/img/.svn/text-base/play.png.svn-base (0) D www/emperor/img/.svn/text-base/reset.png.svn-base (0) D www/emperor/js/.svn/entries (62) D www/emperor/js/.svn/text-base/emperor.js.svn-base (951) D www/emperor/js/emperor.js (1016) D www/emperor/lib.psp (18) D www/emperor/support_files/.svn/entries (96) D www/emperor/support_files/.svn/text-base/webgl_pcoa_footer.txt.svn-base (253) D www/emperor/support_files/.svn/text-base/webgl_pcoa_header.txt.svn-base (589) D www/emperor/support_files/webgl_pcoa_footer.txt (253) D www/emperor/support_files/webgl_pcoa_header.txt (589) R www/emperor_required_resources/css/colorPicker.css (0) A www/emperor_required_resources/css/d3.parcoords.css (40) R www/emperor_required_resources/css/images/ui-bg_flat_0_aaaaaa_40x100.png (0) R www/emperor_required_resources/css/images/ui-bg_flat_75_ffffff_40x100.png (0) R www/emperor_required_resources/css/images/ui-bg_glass_55_fbf9ee_1x400.png (0) R www/emperor_required_resources/css/images/ui-bg_glass_65_ffffff_1x400.png (0) R www/emperor_required_resources/css/images/ui-bg_glass_75_dadada_1x400.png (0) R www/emperor_required_resources/css/images/ui-bg_glass_75_e6e6e6_1x400.png (0) R www/emperor_required_resources/css/images/ui-bg_glass_95_fef1ec_1x400.png (0) R www/emperor_required_resources/css/images/ui-bg_highlight-soft_75_cccccc_1x100.png (0) R www/emperor_required_resources/css/images/ui-icons_222222_256x240.png (0) R www/emperor_required_resources/css/images/ui-icons_2e83ff_256x240.png (0) R www/emperor_required_resources/css/images/ui-icons_454545_256x240.png (0) R www/emperor_required_resources/css/images/ui-icons_888888_256x240.png (0) R www/emperor_required_resources/css/images/ui-icons_cd0a0a_256x240.png (0) A www/emperor_required_resources/css/jquery-ui-1.8.16.custom.css (568) A www/emperor_required_resources/css/jquery-ui2.css (567) A www/emperor_required_resources/css/spectrum.css (408) R www/emperor_required_resources/emperor/css/emperor.css (48) A www/emperor_required_resources/emperor/js/emperor.js (2229) A www/emperor_required_resources/img/emperor.png (0) A www/emperor_required_resources/img/favicon.ico (0) R www/emperor_required_resources/img/pause.png (0) R www/emperor_required_resources/img/play.png (0) R www/emperor_required_resources/img/reset.png (0) A www/emperor_required_resources/js/FileSaver.min.js (2) A www/emperor_required_resources/js/THREEx.screenshot.js (130) A www/emperor_required_resources/js/Three.js (37637) A www/emperor_required_resources/js/d3.parcoords.js (568) A www/emperor_required_resources/js/d3.v3.min.js (5) R www/emperor_required_resources/js/jquery-1.6.2.min.js (0) A www/emperor_required_resources/js/jquery-1.7.1.min.js (4) A www/emperor_required_resources/js/jquery-ui-1.8.17.custom.min.js (356) R www/emperor_required_resources/js/jquery.colorPicker.js (0) R www/emperor_required_resources/js/js/AudioObject.js (0) R www/emperor_required_resources/js/js/Car.js (0) R www/emperor_required_resources/js/js/DAT.GUI.min.js (0) R www/emperor_required_resources/js/js/Detector.js (0) R www/emperor_required_resources/js/js/ImprovedNoise.js (0) R www/emperor_required_resources/js/js/PRNG.js (0) R www/emperor_required_resources/js/js/RequestAnimationFrame.js (0) R www/emperor_required_resources/js/js/ShaderExtras.js (0) R www/emperor_required_resources/js/js/ShaderSkin.js (0) R www/emperor_required_resources/js/js/ShaderTerrain.js (0) R www/emperor_required_resources/js/js/SimplexNoise.js (0) R www/emperor_required_resources/js/js/Sparks.js (0) R www/emperor_required_resources/js/js/Stats.js (0) R www/emperor_required_resources/js/js/Tween.js (0) R www/emperor_required_resources/js/js/ctm/CTMLoader.js (0) R www/emperor_required_resources/js/js/ctm/CTMWorker.js (0) R www/emperor_required_resources/js/js/ctm/ctm.js (0) R www/emperor_required_resources/js/js/ctm/license/OpenCTM.txt (0) R www/emperor_required_resources/js/js/ctm/license/js-lzma.txt (0) R www/emperor_required_resources/js/js/ctm/license/js-openctm.txt (0) R www/emperor_required_resources/js/js/ctm/lzma.js (0) R www/emperor_required_resources/js/js/postprocessing/BloomPass.js (0) R www/emperor_required_resources/js/js/postprocessing/DotScreenPass.js (0) R www/emperor_required_resources/js/js/postprocessing/EffectComposer.js (0) R www/emperor_required_resources/js/js/postprocessing/FilmPass.js (0) R www/emperor_required_resources/js/js/postprocessing/MaskPass.js (0) R www/emperor_required_resources/js/js/postprocessing/RenderPass.js (0) R www/emperor_required_resources/js/js/postprocessing/SavePass.js (0) R www/emperor_required_resources/js/js/postprocessing/ShaderPass.js (0) R www/emperor_required_resources/js/js/postprocessing/TexturePass.js (0) A www/emperor_required_resources/js/specifications.txt (1) A www/emperor_required_resources/js/spectrum.js (1566) M www/handler.psp (37) M www/index.html (1) D www/js/Three.js (34389) D www/js/jquery-1.7.min.js (4) D www/js/jquery-ui-1.8.16.custom.min.js (791) D www/js/js/.svn/entries (510) D www/js/js/.svn/text-base/AudioObject.js.svn-base (172) D www/js/js/.svn/text-base/Car.js.svn-base (364) D www/js/js/.svn/text-base/DAT.GUI.min.js.svn-base (53) D www/js/js/.svn/text-base/Detector.js.svn-base (58) D www/js/js/.svn/text-base/ImprovedNoise.js.svn-base (71) D www/js/js/.svn/text-base/PRNG.js.svn-base (11) D www/js/js/.svn/text-base/RequestAnimationFrame.js.svn-base (22) D www/js/js/.svn/text-base/ShaderExtras.js.svn-base (1779) D www/js/js/.svn/text-base/ShaderSkin.js.svn-base (752) D www/js/js/.svn/text-base/ShaderTerrain.js.svn-base (306) D www/js/js/.svn/text-base/SimplexNoise.js.svn-base (316) D www/js/js/.svn/text-base/Sparks.js.svn-base (832) D www/js/js/.svn/text-base/Stats.js.svn-base (8) D www/js/js/.svn/text-base/Tween.js.svn-base (13) D www/js/js/ctm/.svn/entries (167) D www/js/js/ctm/.svn/text-base/CTMLoader.js.svn-base (821) D www/js/js/ctm/.svn/text-base/CTMWorker.js.svn-base (19) D www/js/js/ctm/.svn/text-base/ctm.js.svn-base (626) D www/js/js/ctm/.svn/text-base/lzma.js.svn-base (510) D www/js/js/ctm/license/.svn/entries (130) D www/js/js/ctm/license/.svn/text-base/OpenCTM.txt.svn-base (20) D www/js/js/ctm/license/.svn/text-base/js-lzma.txt.svn-base (19) D www/js/js/ctm/license/.svn/text-base/js-openctm.txt.svn-base (19) D www/js/js/postprocessing/.svn/entries (334) D www/js/js/postprocessing/.svn/text-base/BloomPass.js.svn-base (100) D www/js/js/postprocessing/.svn/text-base/DotScreenPass.js.svn-base (52) D www/js/js/postprocessing/.svn/text-base/EffectComposer.js.svn-base (136) D www/js/js/postprocessing/.svn/text-base/FilmPass.js.svn-base (51) D www/js/js/postprocessing/.svn/text-base/MaskPass.js.svn-base (86) D www/js/js/postprocessing/.svn/text-base/RenderPass.js.svn-base (51) D www/js/js/postprocessing/.svn/text-base/SavePass.js.svn-base (52) D www/js/js/postprocessing/.svn/text-base/ShaderPass.js.svn-base (51) D www/js/js/postprocessing/.svn/text-base/TexturePass.js.svn-base (37) Patch Links:

https://github.com/qiime/evident/pull/44.patch https://github.com/qiime/evident/pull/44.diff

[ElDeveloper]

This is looking great so far, amazed by the fact that it wasn't that difficult to integrate Emperor. So far just a couple things that I've noticed:

• Trying to visualize Fierer et al with 228 sequences per sample, 3/3 subjects, 9/31 samples per subject and 3/10 iterations throws the following error:

MOD_PYTHON ERROR

ProcessId:      3646
Interpreter:    'evident'

ServerName:     'evident'
DocumentRoot:   '/var/www'

URI:            '/~evident/qiime_software/evident/evident/www/handler.psp'
Location:       None
Directory:      None
Filename:       '/home/evident/shared/qiime_software/evident/evident/www/handler.psp'
PathInfo:       ''

Phase:          'PythonHandler'
Handler:        'mod_python.psp'

Traceback (most recent call last):

  File "/usr/lib/python2.7/dist-packages/mod_python/importer.py", line 1537, in HandlerDispatch
    default=default_handler, arg=req, silent=hlist.silent)

  File "/usr/lib/python2.7/dist-packages/mod_python/importer.py", line 1229, in _process_target
    result = _execute_target(config, req, object, arg)

  File "/usr/lib/python2.7/dist-packages/mod_python/importer.py", line 1128, in _execute_target
    result = object(arg)

  File "/usr/lib/python2.7/dist-packages/mod_python/psp.py", line 337, in handler
    p.run()

  File "/usr/lib/python2.7/dist-packages/mod_python/psp.py", line 243, in run
    exec code in global_scope

  File "/home/evident/shared/qiime_software/evident/evident/www/handler.psp", line 101, in <module>
    tree_object=tree_object)

TypeError: generate_pcoa_cloud_from_point_in_omega() got an unexpected keyword argument 'map_headers'

• The Demo of the Costello, Turbaugh datasets throws the following error:

MOD_PYTHON ERROR

ProcessId:      3646
Interpreter:    'evident'

ServerName:     'evident'
DocumentRoot:   '/var/www'

URI:            '/~evident/qiime_software/evident/evident/www/handler.psp'
Location:       None
Directory:      None
Filename:       '/home/evident/shared/qiime_software/evident/evident/www/handler.psp'
PathInfo:       ''

Phase:          'PythonHandler'
Handler:        'mod_python.psp'

Traceback (most recent call last):

  File "/usr/lib/python2.7/dist-packages/mod_python/importer.py", line 1537, in HandlerDispatch
    default=default_handler, arg=req, silent=hlist.silent)

  File "/usr/lib/python2.7/dist-packages/mod_python/importer.py", line 1229, in _process_target
    result = _execute_target(config, req, object, arg)

  File "/usr/lib/python2.7/dist-packages/mod_python/importer.py", line 1128, in _execute_target
    result = object(arg)

  File "/usr/lib/python2.7/dist-packages/mod_python/psp.py", line 337, in handler
    p.run()

  File "/usr/lib/python2.7/dist-packages/mod_python/psp.py", line 243, in run
    exec code in global_scope

  File "/home/evident/shared/qiime_software/evident/evident/www/handler.psp", line 131, in <module>
    map_headers, map_data)

TypeError: make_pcoa_plot() takes exactly 3 arguments (6 given)

• The HMPV13 demo colors first by description, might be worth to remove that column to improve the loading time, we should be able to use the body site.

Also, there are far too many files in this changeset to efficiently look at them all. Can you point at the ones that I should keep an eye open? I've noticed UPDATE_EMPEROR.md and evident/pcoa.py any others I should check?

Thanks!

[ElDeveloper]

Ok, just followed @antgonza's offline suggestions of: restarting apache and cleaning up all .pyc files and it seems to be working fine now.

However I'm now seeing this problem where the PCoA and Parallel buttons are hidden.

[ElDeveloper]

I think it would probably make sense to use a git-submodule for Emperor. What does the other devs think?

A quote from kernel.org:

Submodules allow foreign repositories to be embedded within a dedicated subdirectory of the source tree, always pointed at a particular commit.

[wdwvt1]

@eldeveloper and all-knowing wikipedia explained this concept to me and it seems like a great way to go.

On Thu, Aug 8, 2013 at 6:33 PM, Yoshiki Vázquez Baeza < notifications@github.com> wrote:

I think it would probably make sense to use a git-submodule for Emperor. What does the other devs think?

A quote from kernel.orghttps://www.kernel.org/pub/software/scm/git/docs/git-submodule.html :

Submodules allow foreign repositories to be embedded within a dedicated subdirectory of the source tree, always pointed at a particular commit.

— Reply to this email directly or view it on GitHubhttps://github.com/qiime/evident/pull/44#issuecomment-22368502 .

[antgonza]

In my opinion it doesn't: the repository is fully copied, you can not just submodule a folder, thus we will need to change all the code to point to point to the new folders and host, twice, all the files.

[ElDeveloper]

Hello,

Ah, I see what you are saying. I most certainly think you are right but cluttering the repository with changes from another repository sounds like a bad idea and kind of hard to maintain.

I think it would probably be worth to try and make it work like that for future implementations and maybe deal with this as it is right now?

An alternative is if Evident writes the header instead of the constant EMPEOR_HTML_HEADER, then we wouldn't have to change the paths and it would basically just point to that submodule.

Thoughts?

El 08/08/2013, a las 22:31, Antonio Gonzalez notifications@github.com escribió:

In my opinion it doesn't: the repository is fully copied, you can not just submodule a folder, thus we will need to change all the code to point to point to the new folders and host, twice, all the files. — Reply to this email directly or view it on GitHub.

[antgonza]

Not sure why you think this is messy? We are only copying one folder. The rest of the files come from installing Emperor. In fact, we could even set up a global variable to let the system know where Evident is and avoid having to copy that folder but this can be problematic as for the way is set up the web libraries should be static. Is the idea to avoid having to install Emperor and just have it as its? Not sure if this will work because you still will need to set up the paths ...

BTW the main files changed are: UPDATE_EMPEROR.md evident/pcoa.py www/handler.psp www/index.html

Deleted as we are now importing these functionalities from Emperor: tests/test_evident.py tests/test_pcoa.py

[ElDeveloper]

I think it's messy because everytime you update the resources, for emperor, you will clutter the repo with things that are not relevant to itself. Whilst if you have a submodule, you should be able to easily update at what version you want to point.

I agree with you that the global variable would be problematic.

No, my idea is not to avoid installing emperor, emperor will have to be installed in the system. I just think that if we had this a submodule, it would become easier and cleaner to change versions of emperor.

Thanks for pointing those files out, taking a look at them right away.

El 09/08/2013, a las 11:26, Antonio Gonzalez notifications@github.com escribió:

Not sure why you think this is messy? We are only copying one folder. The rest of the files come from installing Emperor. In fact, we could even set up a global variable to let the system know where Evident is and avoid having to copy that folder but this can be problematic as for the way is set up the web libraries should be static. Is the idea to avoid having to install Emperor and just have it as its? Not sure if this will work because you still will need to set up the paths ...

BTW the main files changed are: UPDATE_EMPEROR.md evident/pcoa.py www/handler.psp www/index.html

Deleted as we are now importing these functionalities from Emperor: tests/test_evident.py tests/test_pcoa.py — Reply to this email directly or view it on GitHub.

[ElDeveloper]

I think everything is looking good besides that minor comment, which is a question really.

[antgonza]

So what will be the plan with this? Add an issue to make it work with submodules? BTW I looked into this option for this pull request but it didn't make sense to me based on the duplication of code/scripts, and having those copied files exposed in the web as they have to be static (which will mean that the full submodule will have to be exposed). However, I agree that will make it cleaner. What to do in this conundrum?

[ElDeveloper]

I've opened a new issue for the usage of submodules #45.

[ElDeveloper]

Should we prepare a new virtual machine with this build?

[antgonza]

That's a good idea but ideally we should create a list of commands or a GUI to auto update the machine.

[ElDeveloper]

Agreed, perhaps having a script that everytime the virtual machine boots, it tries to pull the latest version of the repository? I think VirtualBox offers a way to automate the creation of a virtual machine, but I don't know how we would go as to configure the machine itself.

El 13/08/2013, a las 09:40, Antonio Gonzalez notifications@github.com escribió:

That's a good idea but ideally we should create a list of commands or a GUI to auto update the machine. — Reply to this email directly or view it on GitHub.

[antgonza]

At boot is an option but most of the time the Virtual Machine has no internet the first time it boots so maybe having a command or a GUI is the way to go.