Test OSX .app file

[ethanwhite]

I'm in the process of trying to get the EcoData Retriever fully functional on OSX (since so many of the awesome ecological informaticsy people I know use Macs). As I've mentioned elsewhere it looks like building from source now works, at least when using homebrew (http://ecodataretriever.org/getting_started.html).

What I'm working on now is getting the .app working so that you don't need to be comfortable in the shell (and have XCode installed) to use the Retriever. I have a version that is working on the machine that I built it on (our only Mac) and was wondering if some kind Mac folks like @karthik, @sckott, @emhart, @sarahsupp and @dfalster might have a few minutes to give it a trial run.

The file is here: https://www.dropbox.com/s/26b1pj91mqucc0l/retriever.zip

Basically I'm just looking for folks to unzip it, double click on it, and see if:

1. It opens at all.
2. You can install things successfully when setting the database management system to CSV and sqlite (these don't have any external dependencies).
3. If you have either MySQL or PostgreSQL installed if it works with them. (MySQL is a bit fragile at the moment. It is currently working for most datasets, but not all, so just try a few if you get errors).
4. Report back.

Thanks in advance. And, yes, I wrote this issue... on a Mac.

[emhart]

Worked great for me. I used both the CSV option and the sqlite with no trouble. DB's look good and functioned properly. The only issue was that opening it requires that you bypass some security setting because it's not from an "identified developer". Other than that works like a charm for me. FYI I'm running OSX Mavericks.

[karthik]

Everything worked great for me.

1. Yes it opens
2. Yep
3. MySQL worked fine for the datasets that I tried. I have Postgres on a different box so I'll try that out shortly. Switching connections sometimes crashes everything.
4. done!

Nice with Ethan!

[sckott]

If Mac users don't already allow external apps to be opened, they probably need to do that.

• Installation: Worked great.
• SQLite connection: worked great
• CSV connection: worked great
• Postgres: worked well - using Postgres.app
• didn't try MySQL

I wonder if it could be clearer where data ends up though...or maybe I missed it somehow.

[emhart]

@sckott The data ends up in the directory you ran the retriever in.

[sckott]

ah, woops :frowning:

[ethanwhite]

Thanks so much to all of you for taking a look at this so quickly! I'm really excited that this is finally working! Expect a new release within the next week.

[sarahsupp]

The retriever file is not working for me. I am using Mac OSX 10.7.5. When I download it from the dropbox link, it seems to download and unzip automatically, but when I double click on the icon, a box pops up that says "retriever error" and gives me two options to click on: open console or terminate.

I'm not sure if I have a setting that is not allowing me to use the retriever app, or how to change it.

Sarah

On Thu, Jan 2, 2014 at 3:14 PM, Ethan White notifications@github.comwrote:

I'm in the process of trying to get the EcoData Retriever fully functional on OSX (since so many of the awesome ecological informaticsy people I know use Macs). As I've mentioned elsewhere it looks like building from source now works, at least when using homebrew ( http://ecodataretriever.org/getting_started.html).

What I'm working on now is getting the .app working so that you don't need to be comfortable in the shell (and have XCode installed) to use the Retriever. I have a version that is working on the machine that I built it on (our only Mac) and was wondering if some kind Mac folks like @karthikhttps://github.com/karthik, @sckott https://github.com/sckott, @emhart https://github.com/emhart, @sarahsupp https://github.com/sarahsupp and @dfalsterhttps://github.com/dfalstermight have a few minutes to give it a trial run.

The file is here: https://www.dropbox.com/s/26b1pj91mqucc0l/retriever.zip

Basically I'm just looking for folks to unzip it, double click on it, and see if:

1. It opens at all.
2. You can install things successfully when setting the database management system to CSV and sqlite (these don't have any external dependencies).
3. If you have either MySQL or PostgreSQL installed if it works with them. (MySQL is a bit fragile at the moment. It is currently working for most datasets, but not all, so just try a few if you get errors).
4. Report back.

Thanks in advance. And, yes, I wrote this issue... on a Mac.

— Reply to this email directly or view it on GitHubhttps://github.com/weecology/retriever/issues/141 .

Sarah R. Supp Postdoctoral Associate Stony Brook University http://weecology.org/people/sarahsupp/Sarah_Supp/About_Me

I am a co-chair for the Gordon Research Seminar Unifying Ecology Across Scales in 2014: (http://tinyurl.com/UnifyingEcologyGRS) Ask me about it!

[dfalster]

Great work Ethan.

Alas I have same problem as Sarah: box with "retriever error" and gives me two options to click on: open console or terminate. . Am running 10.8.5.

Clicking on console yields these error messages:

3/01/14 9:53:19.625 AM retriever[7241]: retriever Error
3/01/14 9:53:22.206 AM com.apple.launchd.peruser.502[156]:
([0x0-0x406406].org.pythonmac.unspecified.retriever[7241]) Exited with
code: 255
3/01/14 9:53:38.385 AM retriever[7255]: retriever Error
3/01/14 9:53:41.875 AM com.apple.launchd.peruser.502[156]:
([0x0-0x408408].org.pythonmac.unspecified.retriever[7255]) Exited with
code: 255
3/01/14 9:54:22.070 AM retriever[7271]: retriever Error
3/01/14 9:54:40.519 AM com.apple.launchd.peruser.502[156]:
([0x0-0x409409].org.pythonmac.unspecified.retriever[7271]) Exited with
code: 255

[sarahsupp]

I should also have attached the error messages that the console gave me:

1/2/14 6:00:34.126 PM [0x0-0x100100].org.pythonmac.unspecified.retriever: ImportError: dlopen(/Users/sarah/Downloads/retriever.app/Contents/Resources/lib/python2.7/lib-dynload/wx/core.so, 2): Library not loaded: /System/Library/Frameworks/CoreGraphics.framework/Versions/A/CoreGraphics

1/2/14 6:00:34.126 PM [0x0-0x100100].org.pythonmac.unspecified.retriever: Referenced from: /Users/sarah/Downloads/retriever.app/Contents/Resources/lib/python2.7/lib-dynload/wx/core.so

1/2/14 6:00:34.126 PM [0x0-0x100100].org.pythonmac.unspecified.retriever: Reason: image not found

1/2/14 6:00:34.227 PM retriever: retriever Error

1/2/14 6:00:36.676 PM com.apple.launchd.peruser.501: ([0x0-0x100100].org.pythonmac.unspecified.retriever[2163]) Exited with code: 255

On Thu, Jan 2, 2014 at 5:57 PM, Daniel Falster notifications@github.comwrote:

Great work Ethan.

Alas I have same problem as Sarah: box with "retriever error" and gives me two options to click on: open console or terminate. . Am running 10.8.5.

Clicking on console yields these error messages:

3/01/14 9:53:19.625 AM retriever[7241]: retriever Error
3/01/14 9:53:22.206 AM com.apple.launchd.peruser.502[156]:
([0x0-0x406406].org.pythonmac.unspecified.retriever[7241]) Exited with
code: 255
3/01/14 9:53:38.385 AM retriever[7255]: retriever Error
3/01/14 9:53:41.875 AM com.apple.launchd.peruser.502[156]:
([0x0-0x408408].org.pythonmac.unspecified.retriever[7255]) Exited with
code: 255
3/01/14 9:54:22.070 AM retriever[7271]: retriever Error
3/01/14 9:54:40.519 AM com.apple.launchd.peruser.502[156]:
([0x0-0x409409].org.pythonmac.unspecified.retriever[7271]) Exited with
code: 255

On 3 January 2014 08:59, Sarah Supp notifications@github.com wrote:

The retriever file is not working for me. I am using Mac OSX 10.7.5. When I download it from the dropbox link, it seems to download and unzip automatically, but when I double click on the icon, a box pops up that says "retriever error" and gives me two options to click on: open console or terminate.

I'm not sure if I have a setting that is not allowing me to use the retriever app, or how to change it.

Sarah

On Thu, Jan 2, 2014 at 3:14 PM, Ethan White notifications@github.comwrote:

I'm in the process of trying to get the EcoData Retriever fully functional on OSX (since so many of the awesome ecological informaticsy people I know use Macs). As I've mentioned elsewhere it looks like building from source now works, at least when using homebrew ( http://ecodataretriever.org/getting_started.html).

What I'm working on now is getting the .app working so that you don't need to be comfortable in the shell (and have XCode installed) to use the Retriever. I have a version that is working on the machine that I built it on (our only Mac) and was wondering if some kind Mac folks like @karthik< https://github.com/karthik>, @sckott https://github.com/sckott, @emhart < https://github.com/emhart>,

@sarahsupp https://github.com/sarahsupp and @dfalster< https://github.com/dfalster>might have a few minutes to give it a trial run.

The file is here: https://www.dropbox.com/s/26b1pj91mqucc0l/retriever.zip

Basically I'm just looking for folks to unzip it, double click on it, and see if:

1. It opens at all.
2. You can install things successfully when setting the database management system to CSV and sqlite (these don't have any external dependencies).
3. If you have either MySQL or PostgreSQL installed if it works with them. (MySQL is a bit fragile at the moment. It is currently working for most datasets, but not all, so just try a few if you get errors).
4. Report back.

Thanks in advance. And, yes, I wrote this issue... on a Mac.

— Reply to this email directly or view it on GitHub< https://github.com/weecology/retriever/issues/141> .

Sarah R. Supp Postdoctoral Associate Stony Brook University http://weecology.org/people/sarahsupp/Sarah_Supp/About_Me

I am a co-chair for the Gordon Research Seminar Unifying Ecology Across Scales in 2014: (http://tinyurl.com/UnifyingEcologyGRS) Ask me about it!

— Reply to this email directly or view it on GitHub< https://github.com/weecology/retriever/issues/141#issuecomment-31488500> .

— Reply to this email directly or view it on GitHubhttps://github.com/weecology/retriever/issues/141#issuecomment-31492348 .

Sarah R. Supp Postdoctoral Associate Stony Brook University http://weecology.org/people/sarahsupp/Sarah_Supp/About_Me

I am a co-chair for the Gordon Research Seminar Unifying Ecology Across Scales in 2014: (http://tinyurl.com/UnifyingEcologyGRS) Ask me about it!

[emhart]

It might also be worth noting @ethanwhite that my default python installation is the latest version of Anaconda. Not sure if that matters or not. (python 2.7.6)

[karthik]

I have the same set up as ted.

[dfalster]

whereas I am using the python setup you (Ethan) previously suggested for getting retriever working ( https://github.com/weecology/retriever/wiki/Mac-OSX-Build), installed via homebrew /usr/local/bin/python

On 3 January 2014 10:15, Karthik Ram notifications@github.com wrote:

I have the same set up as ted.

— Reply to this email directly or view it on GitHubhttps://github.com/weecology/retriever/issues/141#issuecomment-31493483 .

[sckott]

I'm on -OSX 10.9.1 -Python 2.7.3, I think at /usr/bin/python

[ethanwhite]

@sarahsupp and @dfalster thanks for reporting the problems you're having. Can you go to the directory where the .app file is and run the following from the terminal:

retriever.app/Contents/MacOS/retriever

and then post the full output here (unless it's identical to the error you already posted).

Also can you run

retriever.app/Contents/MacOS/retriever --help

And see if you get the same error or something different. Thanks!

[sarahsupp]

When I run run the following from the terminal: retriever.app/Contents/MacOS/retriever

This is the message I get - first, a bunch of downloads that look successful (I kept them so you can see how far it went), then some python-related errors:

sarahsupp:Downloads sarah$ retriever.app/Contents/MacOS/retriever Downloading script: USDA_plants.script 0.0-USDA_plants.script 100.0-USDA_plants.script Downloading script: npn.py 0.0-npn.py 100.0-npn.py Downloading script: EA_woods2009.script 0.0-EA_woods2009.script 100.0-EA_woods2009.script Downloading script: EA_clark2006.py 0.0-EA_clark2006.py 100.0-EA_clark2006.py Downloading script: EA_avianbodysize2007.script 0.0-EA_avianbodysize2007.script 100.0-EA_avianbodysize2007.script Downloading script: EA_mcglinn2010.script 0.0-EA_mcglinn2010.script 100.0-EA_mcglinn2010.script Downloading script: EA_ernest2003.script 0.0-EA_ernest2003.script 100.0-EA_ernest2003.script Downloading script: gentry.py 0.0-gentry.py 81.4-gentry.py 100.0-gentry.py Downloading script: EA_capellini2010.script 0.0-EA_capellini2010.script 100.0-EA_capellini2010.script Downloading script: bbs50stop.py 0.0-bbs50stop.py 62.3-bbs50stop.py 100.0-bbs50stop.py Downloading script: EA_portal_mammals.script 0.0-EA_portal_mammals.script 100.0-EA_portal_mammals.script Downloading script: fia.py 0.0-fia.py 100.0-fia.py Downloading script: EA_thibault2011.script 0.0-EA_thibault2011.script 100.0-EA_thibault2011.script Downloading script: EA_mom2003.script 0.0-EA_mom2003.script 100.0-EA_mom2003.script Downloading script: EA_ramesh2010.script 0.0-EA_ramesh2010.script 100.0-EA_ramesh2010.script Downloading script: EA_pantheria.script 0.0-EA_pantheria.script 100.0-EA_pantheria.script Downloading script: EA_zachmann2010.script 0.0-EA_zachmann2010.script 100.0-EA_zachmann2010.script Downloading script: bbs.py 0.0-bbs.py 80.7-bbs.py 100.0-bbs.py Downloading script: EA_petraitis2008.py 0.0-EA_petraitis2008.py 100.0-EA_petraitis2008.py Downloading script: EA_barnes2008.script 0.0-EA_barnes2008.script 100.0-EA_barnes2008.script Downloading script: EA_adler2007.script 0.0-EA_adler2007.script 100.0-EA_adler2007.script Downloading script: EA_del_moral_2010.script 0.0-EA_del_moral_2010.script 100.0-EA_del_moral_2010.script Downloading script: fia.py 0.0-fia.py 100.0-fia.py Downloading script: bbs.py 0.0-bbs.py 80.7-bbs.py 100.0-bbs.py Traceback (most recent call last): File "/Users/sarah/Downloads/retriever.app/Contents/Resources/boot.py", line 336, in _run() File "/Users/sarah/Downloads/retriever.app/Contents/Resources/boot.py", line 331, in _run exec(compile(source, path, 'exec'), globals(), globals()) File "/Users/sarah/Downloads/retriever.app/Contents/Resources/main.py", line 127, in main() File "/Users/sarah/Downloads/retriever.app/Contents/Resources/main.py", line 34, in main from retriever.app.main import launch_app File "/Users/sarah/Downloads/retriever.app/Contents/Resources/lib/python2.7/retriever/app/main.py", line 10, in from retriever.app.app import App File "/Users/sarah/Downloads/retriever.app/Contents/Resources/lib/python2.7/retriever/app/app.py", line 3, in import wx File "wx/init.pyo", line 45, in File "wx/_core.pyo", line 4, in File "wx/core.pyo", line 14, in File "wx/core.pyo", line 10, in __load ImportError: dlopen(/Users/sarah/Downloads/retriever.app/Contents/Resources/lib/python2.7/lib-dynload/wx/core.so, 2): Library not loaded: /System/Library/Frameworks/CoreGraphics.framework/Versions/A/CoreGraphics Referenced from: /Users/sarah/Downloads/retriever.app/Contents/Resources/lib/python2.7/lib-dynload/wx/core.so Reason: image not found 2014-01-03 09:54:58.114 retriever[2700:707] retriever Error

When I run retriever.app/Contents/MacOS/retriever --help, I get a list of the arguments, so it seems to be working OK.

On Fri, Jan 3, 2014 at 9:41 AM, Ethan White notifications@github.com wrote:

@sarahsupp https://github.com/sarahsupp and @dfalsterhttps://github.com/dfalster thanks for reporting the problems you're having. Can you go to the directory where the .app file is and run the following from the terminal:

retriever.app/Contents/MacOS/retriever

and then post the full output here (unless it's identical to the error you already posted).

Also can you run

retriever.app/Contents/MacOS/retriever --help

And see if you get the same error or something different. Thanks!

— Reply to this email directly or view it on GitHubhttps://github.com/weecology/retriever/issues/141#issuecomment-31525926 .

Sarah R. Supp Postdoctoral Associate Stony Brook University http://weecology.org/people/sarahsupp/Sarah_Supp/About_Me

I am a co-chair for the Gordon Research Seminar Unifying Ecology Across Scales in 2014: (http://tinyurl.com/UnifyingEcologyGRS) Ask me about it!

On Fri, Jan 3, 2014 at 9:41 AM, Ethan White notifications@github.comwrote:

@sarahsupp https://github.com/sarahsupp and @dfalsterhttps://github.com/dfalsterthanks for reporting the problems you're having. Can you go to the directory where the .app file is and run the following from the terminal:

retriever.app/Contents/MacOS/retriever

and then post the full output here (unless it's identical to the error you already posted).

Also can you run

retriever.app/Contents/MacOS/retriever --help

And see if you get the same error or something different. Thanks!

— Reply to this email directly or view it on GitHubhttps://github.com/weecology/retriever/issues/141#issuecomment-31525926 .

Sarah R. Supp Postdoctoral Associate Stony Brook University http://weecology.org/people/sarahsupp/Sarah_Supp/About_Me

I am a co-chair for the Gordon Research Seminar Unifying Ecology Across Scales in 2014: (http://tinyurl.com/UnifyingEcologyGRS) Ask me about it!

[ethanwhite]

Thanks Sarah! What happens if you run this from the terminal:

ls /System/Library/Frameworks/CoreGraphics.framework/Versions/A/CoreGraphics

[sarahsupp]

sarahsupp:Downloads sarah$ ls /System/Library/Frameworks/CoreGraphics.framework/Versions/A/CoreGraphics

ls: /System/Library/Frameworks/CoreGraphics.framework/Versions/A/CoreGraphics: No such file or directory

On Fri, Jan 3, 2014 at 10:14 AM, Ethan White notifications@github.comwrote:

Thanks Sarah! What happens if you run this from the terminal:

ls /System/Library/Frameworks/CoreGraphics.framework/Versions/A/CoreGraphics

— Reply to this email directly or view it on GitHubhttps://github.com/weecology/retriever/issues/141#issuecomment-31528191 .

Sarah R. Supp Postdoctoral Associate Stony Brook University http://weecology.org/people/sarahsupp/Sarah_Supp/About_Me

I am a co-chair for the Gordon Research Seminar Unifying Ecology Across Scales in 2014: (http://tinyurl.com/UnifyingEcologyGRS) Ask me about it!

[ethanwhite]

Notes on investigation so far:

Sarah's issue (at least) is related to wx. For some reason _core_.so in the .app (and 3 additional files as well) is trying to load something in /System/ that doesn't exist on Sarah's machine. It looks like the location of this file changed from 10.7 to 10.8, but that doesn't explain why the .app is failing for Dan on 10.8 or why this sort of dependency is there in the first place.

[ethanwhite]

OK so looking at what's going on and what everyone's setups are it looks like this is probably an OS version difference, with everything working properly on the build OS, 10.9, but not on the prior operating systems. I'm looking into what I need to do to address this.

@dfalster, when you wake up and have a few minutes I'd still like to see the output from the commands I mentioned above so that I can make sure that this is really a single problem that I'm running down. Thanks!

[dfalster]

Here you go. I am now running this on my laptop as I am at home, but that has same setup (10.8.5) and behaves similarly.

~/Downloads » retriever.app/Contents/MacOS/retriever
Downloading script: USDA_plants.script
0.0-USDA_plants.script 100.0-USDA_plants.script Downloading script: npn.py
0.0-npn.py 100.0-npn.py Downloading script: EA_woods2009.script
0.0-EA_woods2009.script 100.0-EA_woods2009.script Downloading script:
EA_clark2006.py
0.0-EA_clark2006.py 100.0-EA_clark2006.py Downloading script:
EA_avianbodysize2007.script
0.0-EA_avianbodysize2007.script 100.0-EA_avianbodysize2007.script
Downloading script: EA_mcglinn2010.script
0.0-EA_mcglinn2010.script 100.0-EA_mcglinn2010.script Downloading script:
EA_ernest2003.script
0.0-EA_ernest2003.script 100.0-EA_ernest2003.script Downloading script:
gentry.py
0.0-gentry.py 81.4-gentry.py 100.0-gentry.py Downloading script:
EA_capellini2010.script
0.0-EA_capellini2010.script 100.0-EA_capellini2010.script Downloading
script: bbs50stop.py
0.0-bbs50stop.py 62.3-bbs50stop.py 100.0-bbs50stop.py Downloading script:
EA_portal_mammals.script
0.0-EA_portal_mammals.script 100.0-EA_portal_mammals.script Downloading
script: fia.py
0.0-fia.py 100.0-fia.py Downloading script: EA_thibault2011.script
0.0-EA_thibault2011.script 100.0-EA_thibault2011.script Downloading script:
EA_mom2003.script
0.0-EA_mom2003.script 100.0-EA_mom2003.script Downloading script:
EA_ramesh2010.script
0.0-EA_ramesh2010.script 100.0-EA_ramesh2010.script Downloading script:
EA_pantheria.script
0.0-EA_pantheria.script 100.0-EA_pantheria.script Downloading script:
EA_zachmann2010.script
0.0-EA_zachmann2010.script 100.0-EA_zachmann2010.script Downloading script:
bbs.py
0.0-bbs.py 80.7-bbs.py 100.0-bbs.py Downloading script: EA_petraitis2008.py
0.0-EA_petraitis2008.py 100.0-EA_petraitis2008.py Downloading script:
EA_barnes2008.script
0.0-EA_barnes2008.script 100.0-EA_barnes2008.script Downloading script:
EA_adler2007.script
0.0-EA_adler2007.script 100.0-EA_adler2007.script Downloading script:
EA_del_moral_2010.script
0.0-EA_del_moral_2010.script 100.0-EA_del_moral_2010.script Downloading
script: fia.py
0.0-fia.py 100.0-fia.py Downloading script: bbs.py
0.0-bbs.py 80.7-bbs.py 100.0-bbs.py Failed to load script: USDA_plants
(/Users/dfalster/.retriever/scripts/)
Traceback (most recent call last):
  File
"/Users/dfalster/Downloads/retriever.app/Contents/Resources/__boot__.py",
line 336, in <module>
    _run()
  File
"/Users/dfalster/Downloads/retriever.app/Contents/Resources/__boot__.py",
line 331, in _run
    exec(compile(source, path, 'exec'), globals(), globals())
  File
"/Users/dfalster/Downloads/retriever.app/Contents/Resources/__main__.py",
line 127, in <module>
    main()
  File
"/Users/dfalster/Downloads/retriever.app/Contents/Resources/__main__.py",
line 32, in main
    lists = get_lists()
  File
"/Users/dfalster/Downloads/retriever.app/Contents/Resources/lib/python2.7/retriever/lib/lists.py",
line 48, in get_lists
    SCRIPT_LIST = SCRIPT_LIST()
  File
"/Users/dfalster/Downloads/retriever.app/Contents/Resources/lib/python2.7/retriever/__init__.py",
line 82, in SCRIPT_LIST
    return [module.SCRIPT for module in MODULE_LIST(force_compile)]
  File
"/Users/dfalster/Downloads/retriever.app/Contents/Resources/lib/python2.7/retriever/__init__.py",
line 71, in MODULE_LIST
    new_module = imp.load_module(script_name, file, pathname, desc)
  File "/Users/dfalster/.retriever/scripts/USDA_plants.py", line 5
    SCRIPT = BasicTextTemplate(<p>xid="2270582442</p>",
                               ^
SyntaxError: invalid syntax
2014-01-04 06:43:00.656 retriever[19620:707] retriever Error

~/Downloads » retriever.app/Contents/MacOS/retriever --help
Failed to load script: USDA_plants (/Users/dfalster/.retriever/scripts/)
Traceback (most recent call last):
  File
"/Users/dfalster/Downloads/retriever.app/Contents/Resources/__boot__.py",
line 336, in <module>
    _run()
  File
"/Users/dfalster/Downloads/retriever.app/Contents/Resources/__boot__.py",
line 331, in _run
    exec(compile(source, path, 'exec'), globals(), globals())
  File
"/Users/dfalster/Downloads/retriever.app/Contents/Resources/__main__.py",
line 127, in <module>
    main()
  File
"/Users/dfalster/Downloads/retriever.app/Contents/Resources/__main__.py",
line 40, in main
    script_list = SCRIPT_LIST()
  File
"/Users/dfalster/Downloads/retriever.app/Contents/Resources/lib/python2.7/retriever/__init__.py",
line 82, in SCRIPT_LIST
    return [module.SCRIPT for module in MODULE_LIST(force_compile)]
  File
"/Users/dfalster/Downloads/retriever.app/Contents/Resources/lib/python2.7/retriever/__init__.py",
line 71, in MODULE_LIST
    new_module = imp.load_module(script_name, file, pathname, desc)
  File "/Users/dfalster/.retriever/scripts/USDA_plants.py", line 5
    SCRIPT = BasicTextTemplate(<p>xid="2270582442</p>",
                               ^
SyntaxError: invalid syntax
2014-01-04 06:43:39.558 retriever[19679:707] retriever Error

~/Downloads » ls
/System/Library/Frameworks/CoreGraphics.framework/Versions/A/CoreGraphics
/System/Library/Frameworks/CoreGraphics.framework/Versions/A/CoreGraphics

On 4 January 2014 04:54, Ethan White notifications@github.com wrote:

OK so looking at what's going on and what everyone's setups are it looks like this is probably an OS version difference, with everything working properly on the build OS, 10.9, but not on the prior operating systems. I'm looking into what I need to do to address this.

@dfalster https://github.com/dfalster, when you wake up and have a few minutes I'd still like to see the output from the commands I mentioned above so that I can make sure that this is really a single problem that I'm running down. Thanks!

— Reply to this email directly or view it on GitHubhttps://github.com/weecology/retriever/issues/141#issuecomment-31540167 .

[ethanwhite]

That's... unexpected. It looks like the script file isn't being parsed properly. Can you post the contents of /Users/dfalster/.retriever/scripts/USDA_plants.script and /Users/dfalster/.retriever/scripts/USDA_plants.py. They should both be short so code blocks are fine or feel free to just email them to me. Thanks.

[dfalster]

Hi Ethan,

Sure, here it is.

USDA_plants.script

<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
 "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html>
  <head>
    <title>503 backend read error</title>
  </head>
  <body>
    <h1>Error 503 backend read error</h1>
    <p>backend read error</p>
    <h3>Guru Meditation:</h3>
    <p>XID: 2270582442</p>
    <hr>
    <p>Varnish cache server</p>
  </body>
</html>

USDA_plant.py

#retriever
from retriever.lib.templates import BasicTextTemplate
from retriever.lib.models import Table, Cleanup, correct_invalid_value

SCRIPT = BasicTextTemplate(<p>xid="2270582442</p>",
                           tables={},
                           <h3>guru meditation="</h3>",
                           urls={},
                           "http="//
www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">")

On 4 January 2014 07:39, Ethan White notifications@github.com wrote:

That's... unexpected. It looks like the script file isn't being parsed properly. Can you post the contents of /Users/dfalster/.retriever/scripts/USDA_plants.script and /Users/dfalster/.retriever/scripts/USDA_plants.py. They should both be short so code blocks are fine or feel free to just email them to me. Thanks.

— Reply to this email directly or view it on GitHubhttps://github.com/weecology/retriever/issues/141#issuecomment-31551808 .

[dfalster]

Out of interest, I deleted the contents of .retriever/scripts and reran - same crash behaviour but with different error messages.

~/Downloads » retriever.app/Contents/MacOS/retriever
Downloading script: USDA_plants.script
0.0-USDA_plants.script 100.0-USDA_plants.script Downloading script: npn.py
0.0-npn.py 100.0-npn.py Downloading script: EA_woods2009.script
0.0-EA_woods2009.script 100.0-EA_woods2009.script Downloading script:
EA_clark2006.py
0.0-EA_clark2006.py 100.0-EA_clark2006.py Downloading script:
EA_avianbodysize2007.script
0.0-EA_avianbodysize2007.script 100.0-EA_avianbodysize2007.script
Downloading script: EA_mcglinn2010.script
0.0-EA_mcglinn2010.script 100.0-EA_mcglinn2010.script Downloading script:
EA_ernest2003.script
0.0-EA_ernest2003.script 100.0-EA_ernest2003.script Downloading script:
gentry.py
0.0-gentry.py 81.4-gentry.py 100.0-gentry.py Downloading script:
EA_capellini2010.script
0.0-EA_capellini2010.script 100.0-EA_capellini2010.script Downloading
script: bbs50stop.py
0.0-bbs50stop.py 62.3-bbs50stop.py 100.0-bbs50stop.py Downloading script:
EA_portal_mammals.script
0.0-EA_portal_mammals.script 100.0-EA_portal_mammals.script Downloading
script: fia.py
0.0-fia.py 100.0-fia.py Downloading script: EA_thibault2011.script
0.0-EA_thibault2011.script 100.0-EA_thibault2011.script Downloading script:
EA_mom2003.script
0.0-EA_mom2003.script 100.0-EA_mom2003.script Downloading script:
EA_ramesh2010.script
0.0-EA_ramesh2010.script 100.0-EA_ramesh2010.script Downloading script:
EA_pantheria.script
0.0-EA_pantheria.script 100.0-EA_pantheria.script Downloading script:
EA_zachmann2010.script
0.0-EA_zachmann2010.script 100.0-EA_zachmann2010.script Downloading script:
bbs.py
0.0-bbs.py 80.7-bbs.py 100.0-bbs.py Downloading script: EA_petraitis2008.py
0.0-EA_petraitis2008.py 100.0-EA_petraitis2008.py Downloading script:
EA_barnes2008.script
0.0-EA_barnes2008.script 100.0-EA_barnes2008.script Downloading script:
EA_adler2007.script
0.0-EA_adler2007.script 100.0-EA_adler2007.script Downloading script:
EA_del_moral_2010.script
0.0-EA_del_moral_2010.script 100.0-EA_del_moral_2010.script Downloading
script: fia.py
0.0-fia.py 100.0-fia.py Downloading script: bbs.py
0.0-bbs.py 80.7-bbs.py 100.0-bbs.py
Traceback (most recent call last):
  File
"/Users/dfalster/Downloads/retriever.app/Contents/Resources/__boot__.py",
line 336, in <module>
    _run()
  File
"/Users/dfalster/Downloads/retriever.app/Contents/Resources/__boot__.py",
line 331, in _run
    exec(compile(source, path, 'exec'), globals(), globals())
  File
"/Users/dfalster/Downloads/retriever.app/Contents/Resources/__main__.py",
line 127, in <module>
    main()
  File
"/Users/dfalster/Downloads/retriever.app/Contents/Resources/__main__.py",
line 34, in main
    from retriever.app.main import launch_app
  File
"/Users/dfalster/Downloads/retriever.app/Contents/Resources/lib/python2.7/retriever/app/main.py",
line 10, in <module>
    from retriever.app.app import App
  File
"/Users/dfalster/Downloads/retriever.app/Contents/Resources/lib/python2.7/retriever/app/app.py",
line 3, in <module>
    import wx
  File "wx/__init__.pyo", line 45, in <module>
  File "wx/_core.pyo", line 4, in <module>
  File "wx/_core_.pyo", line 14, in <module>
  File "wx/_core_.pyo", line 10, in __load
ImportError:
dlopen(/Users/dfalster/Downloads/retriever.app/Contents/Resources/lib/python2.7/lib-dynload/wx/_core_.so,
2): Symbol not found: ___sincos_stret
  Referenced from:
/Users/dfalster/Downloads/retriever.app/Contents/MacOS/../Frameworks/libwx_osx_cocoau-2.9.5.0.0.dylib
  Expected in: /usr/lib/libSystem.B.dylib
 in
/Users/dfalster/Downloads/retriever.app/Contents/MacOS/../Frameworks/libwx_osx_cocoau-2.9.5.0.0.dylib
2014-01-05 21:00:10.145 retriever[20263:707] retriever Error
~/Downloads »  retriever.app/Contents/MacOS/retriever --help
usage: __main__.py [-h] [-v] [-q]
                   {install,update,gui,new,ls,citation,help} ...

positional arguments:
  {install,update,gui,new,ls,citation,help}
                        sub-command help
    install             download and install dataset
    update              download updated versions of scripts
    gui                 launch retriever in graphical mode
    new                 create a new sample retriever script
    ls                  display a list all available dataset scripts
    citation            view citation
    help

optional arguments:
  -h, --help            show this help message and exit
  -v, --version         show program's version number and exit
  -q, --quiet           suppress command-line output
~/Downloads »

And the contents of the USDA files now looks more suitable

USDA_plants.script

name: USDA plants
description: Plant taxonomy system available on the USDA plants site.
shortname: PlantTaxonomy
tags: Taxon > Plants, Data Type > Taxonomy
url: http://plants.usda.gov

table: PlantTaxonomy,
http://plants.usda.gov/java/downloadData?fileName=plantlst.txt&static=true
*column: record_id, pk-auto
*column: symbol, char, 7
*column: synonym_symbol, char, 7
*column: scientific_name, char
*column: common_name, char, 50
*column: family, char, 30
*do_not_bulk_insert: True

USDA_plants.py

#retriever
from retriever.lib.templates import BasicTextTemplate
from retriever.lib.models import Table, Cleanup, correct_invalid_value

SCRIPT = BasicTextTemplate(tables={'PlantTaxonomy': Table('PlantTaxonomy',
do_not_bulk_insert=True,columns=[('record_id', ('pk-auto',)), ('symbol',
('char', '7')), ('synonym_symbol', ('char', '7')), ('scientific_name',
('char',)), ('common_name', ('char', '50')), ('family', ('char', '30'))])},
                           name="USDA plants",
                           tags=['Taxon > Plants', 'Data Type > Taxonomy'],
                           ref="http://plants.usda.gov",
                           urls={'PlantTaxonomy': '
http://plants.usda.gov/java/downloadData?fileName=plantlst.txt&static=true'
},
                           shortname="PlantTaxonomy",
                           description="Plant taxonomy system available on
the USDA plants site.")

Can check my desktop (imac) tomorrow if that helps?

On 5 January 2014 20:58, Daniel Falster daniel.falster@mq.edu.au wrote:

Hi Ethan,

Sure, here it is.

USDA_plants.script

<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
 "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html>
  <head>
    <title>503 backend read error</title>
  </head>
  <body>
    <h1>Error 503 backend read error</h1>
    <p>backend read error</p>
    <h3>Guru Meditation:</h3>
    <p>XID: 2270582442</p>
    <hr>
    <p>Varnish cache server</p>
  </body>
</html>

USDA_plant.py

#retriever
from retriever.lib.templates import BasicTextTemplate
from retriever.lib.models import Table, Cleanup, correct_invalid_value

SCRIPT = BasicTextTemplate(<p>xid="2270582442</p>",
                           tables={},
                           <h3>guru meditation="</h3>",
                           urls={},
                           "http="//
www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">")

On 4 January 2014 07:39, Ethan White notifications@github.com wrote:

That's... unexpected. It looks like the script file isn't being parsed properly. Can you post the contents of /Users/dfalster/.retriever/scripts/USDA_plants.script and /Users/dfalster/.retriever/scripts/USDA_plants.py. They should both be short so code blocks are fine or feel free to just email them to me. Thanks.

— Reply to this email directly or view it on GitHubhttps://github.com/weecology/retriever/issues/141#issuecomment-31551808 .

[ethanwhite]

Thanks @dfalster! So the initial error was just a failed script download. We should explicitly catch those kinds of errors and I've started an issue for it #146.

The other problem is the same as the one reported by @sarahsupp. The app is not currently finding core system files that are moving around from OS X version to OS X version. I'm looking for solutions that will address this problem, but without much luck so far. If we can't find them then we'll have to build the app on each version of OS X we want to support.

[ethanwhite]

OK, so it turns out that homebrew doesn't install Python and associated packages to work with the OS X SDK, which makes the resulting apps system specific. I have rebuilt our Mac Python development setup in a way that will hopefully (fingers crossed, knock on wood) solve this issue. Could I impose on @dfalster and @sarahsupp to give this version a try:

https://www.dropbox.com/s/a8plcz6z6ztmu5w/retriever2.zip

Thanks again for all your help.

[dfalster]

Works! Great work Ethan.

On 7 January 2014 08:27, Ethan White notifications@github.com wrote:

OK, so it turns out that homebrew doesn't install Python and associated packages to work with the OS X SDK, which makes the resulting apps system specific. I have rebuilt our Mac Python development setup in a way that will hopefully (fingers crossed, knock on wood) solve this issue. Could I impose on @dfalster https://github.com/dfalster and @sarahsupphttps://github.com/sarahsuppto give this version a try:

https://www.dropbox.com/s/a8plcz6z6ztmu5w/retriever2.zip

Thanks again for all your help.

— Reply to this email directly or view it on GitHubhttps://github.com/weecology/retriever/issues/141#issuecomment-31688853 .

[ethanwhite]

Awesome! Thanks for all the help Dan!

Now that we're working on something other than 10.9 could I get one more test from someone on 10.9 (pinging @emhart, @karthik, and @sckott). If it works there and it works for @sarahsupp then I think we can call this good.

[emhart]

The new one worked for me still.

On Mon, Jan 6, 2014 at 4:56 PM, Ethan White notifications@github.comwrote:

Awesome! Thanks for all the help Dan!

Now that we're working on something other than 10.9 could I get one more test from someone on 10.9 (pinging @emhart https://github.com/emhart, @karthik https://github.com/karthik, and @sckotthttps://github.com/sckott). If it works there and it works for @sarahsupphttps://github.com/sarahsuppthen I think we can call this good.

— Reply to this email directly or view it on GitHubhttps://github.com/weecology/retriever/issues/141#issuecomment-31700611 .

Edmund M. Hart, PhD Staff Scientist - Ecoinformatics National Ecological Observatory Network @distribecology http://emhart.github.com

[sarahsupp]

Awesome! It works great! :)

On Mon, Jan 6, 2014 at 4:27 PM, Ethan White notifications@github.comwrote:

OK, so it turns out that homebrew doesn't install Python and associated packages to work with the OS X SDK, which makes the resulting apps system specific. I have rebuilt our Mac Python development setup in a way that will hopefully (fingers crossed, knock on wood) solve this issue. Could I impose on @dfalster https://github.com/dfalster and @sarahsupphttps://github.com/sarahsuppto give this version a try:

https://www.dropbox.com/s/a8plcz6z6ztmu5w/retriever2.zip

Thanks again for all your help.

— Reply to this email directly or view it on GitHubhttps://github.com/weecology/retriever/issues/141#issuecomment-31688853 .

Sarah R. Supp Postdoctoral Associate Stony Brook University http://weecology.org/people/sarahsupp/Sarah_Supp/About_Me

I am a co-chair for the Gordon Research Seminar Unifying Ecology Across Scales in 2014: (http://tinyurl.com/UnifyingEcologyGRS) Ask me about it!

[ethanwhite]

Fantastic! Thanks so much to all of you for your help. I'm so excited to finally have the Retriever fully supported on the Mac.

Time to make a short list of last critical things to clean up and get a release out the door.

[bendmorris]

Great work Ethan, very exciting!