sync users fails with error

[murphysj]

WHO: Sylvia

Not sure what the issue is...

Checking user: Thomas Hamill url: http://cog-esgf.esrl.noaa.gov/user/byopenid/?openid=https://hydra.fsl.noaa.gov/esgf-idp/openid/tom.hamill User found.

Checking user: Shrinivas Moorthi url: http://cog-esgf.esrl.noaa.gov/user/byopenid/?openid=https://hydra.fsl.noaa.gov/esgf-idp/openid/smoorthi User found.

Checking user: Eun-Soon IM url: http://cog-esgf.esrl.noaa.gov/user/byopenid/?openid=https://hydra.fsl.noaa.gov/esgf-idp/openid/esim1559 User found.

Checking user: Sai Kumar Popuri url: http://cog-esgf.esrl.noaa.gov/user/byopenid/?openid=https://hydra.fsl.noaa.gov/esgf-idp/openid/saiku1 User found. Traceback (most recent call last): File "manage.py", line 12, in execute_from_command_line(sys.argv) File "/data/web/virtualenv/cog-cu/lib/python2.7/site-packages/Django-1.9.4-py2.7.egg/django/core/management/init.py", line 353, in execute_from_command_line utility.execute() File "/data/web/virtualenv/cog-cu/lib/python2.7/site-packages/Django-1.9.4-py2.7.egg/django/core/management/init.py", line 345, in execute self.fetch_command(subcommand).run_from_argv(self.argv) File "/data/web/virtualenv/cog-cu/lib/python2.7/site-packages/Django-1.9.4-py2.7.egg/django/core/management/base.py", line 348, in run_from_argv self.execute(_args, _cmd_options) File "/data/web/virtualenv/cog-cu/lib/python2.7/site-packages/Django-1.9.4-py2.7.egg/django/core/management/base.py", line 399, in execute output = self.handle(_args, _options) File "/data/web/projects/cog_cu/COG/cog/management/commands/sync_users.py", line 27, in handle print "\nChecking user: %s url: %s" % (user, userProfileUrl) UnicodeDecodeError: 'ascii' codec can't decode byte 0xe5 in position 16: ordinal not in range(128)

[murphysj]

The user the script checked right before, does not seem to have any funky characters. Would the script eventually bomb on pcmdi9 openIDs? There are still a few in there from power users.

[LucaCinquini]

HI Sylvia, I think it’s because that user has some non-ascii characters somewhere in his/her data. Could you check all of its fields, perhaps print them out ? thanks, L

On May 2, 2016, at 10:58 AM, Sylvia Murphy notifications@github.com<mailto:notifications@github.com> wrote:

The user the script checked right before, does not seem to have any funky characters. Would the script eventually bomb on pcmdi9 openIDs? There are still a few in there from power users.

— You are receiving this because you were assigned. Reply to this email directly or view it on GitHubhttps://github.com/EarthSystemCoG/COG/issues/1275#issuecomment-216292296

[murphysj]

I did. Nothing is showing up in the profile. Could it be someplace else in the DB?

On Mon, May 2, 2016 at 11:26 AM, Luca Cinquini notifications@github.com wrote:

HI Sylvia, I think it’s because that user has some non-ascii characters somewhere in his/her data. Could you check all of its fields, perhaps print them out ? thanks, L

On May 2, 2016, at 10:58 AM, Sylvia Murphy <notifications@github.com mailto:notifications@github.com> wrote:

The user the script checked right before, does not seem to have any funky characters. Would the script eventually bomb on pcmdi9 openIDs? There are still a few in there from power users.

— You are receiving this because you were assigned. Reply to this email directly or view it on GitHub< https://github.com/EarthSystemCoG/COG/issues/1275#issuecomment-216292296>

— You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub https://github.com/EarthSystemCoG/COG/issues/1275#issuecomment-216302225

---

Sylvia Murphy NESII/CIRES/NOAA Earth System Research Laboratory 325 Broadway, Boulder CO 80305 Time Zone: U.S. Mountain Web: http://www.esrl.noaa.gov/nesii/ Email: sylvia.murphy@noaa.gov Phone: 303-497-7753

[LucaCinquini]

Not sure, I’ll try to debug the problem. thanks, L

On May 2, 2016, at 11:27 AM, Sylvia Murphy notifications@github.com<mailto:notifications@github.com> wrote:

I did. Nothing is showing up in the profile. Could it be someplace else in the DB?

On Mon, May 2, 2016 at 11:26 AM, Luca Cinquini notifications@github.com<mailto:notifications@github.com> wrote:

HI Sylvia, I think it’s because that user has some non-ascii characters somewhere in his/her data. Could you check all of its fields, perhaps print them out ? thanks, L

On May 2, 2016, at 10:58 AM, Sylvia Murphy notifications@github.com<mailto:notifications@github.com mailto:notifications@github.com> wrote:

The user the script checked right before, does not seem to have any funky characters. Would the script eventually bomb on pcmdi9 openIDs? There are still a few in there from power users.

— You are receiving this because you were assigned. Reply to this email directly or view it on GitHub< https://github.com/EarthSystemCoG/COG/issues/1275#issuecomment-216292296>

— You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub https://github.com/EarthSystemCoG/COG/issues/1275#issuecomment-216302225

---

Sylvia Murphy NESII/CIRES/NOAA Earth System Research Laboratory 325 Broadway, Boulder CO 80305 Time Zone: U.S. Mountain Web: http://www.esrl.noaa.gov/nesii/ Email: sylvia.murphy@noaa.govmailto:sylvia.murphy@noaa.gov Phone: 303-497-7753

— You are receiving this because you were assigned. Reply to this email directly or view it on GitHubhttps://github.com/EarthSystemCoG/COG/issues/1275#issuecomment-216302454

[LucaCinquini]

Ok, it turns out the script was crashing on the user AFTER that one, specially the user with this openid:

http://esgf-data.dkrz.de/user/byopenid/?openid=https://esgf-data.dkrz.de/esgf-idp/openid/songjunbo

which probably has some bad characters in first or last name. I will modify the script to only print the openid, not the full name, which should avoid this problem.

thanks, L

P.S.: I am running the script at CU right now

On May 2, 2016, at 12:13 PM, Cinquini, Luca (398G) Luca.Cinquini@jpl.nasa.gov wrote:

Not sure, I’ll try to debug the problem. thanks, L

On May 2, 2016, at 11:27 AM, Sylvia Murphy notifications@github.com wrote:

I did. Nothing is showing up in the profile. Could it be someplace else in the DB?

On Mon, May 2, 2016 at 11:26 AM, Luca Cinquini notifications@github.com wrote:

HI Sylvia, I think it’s because that user has some non-ascii characters somewhere in his/her data. Could you check all of its fields, perhaps print them out ? thanks, L

On May 2, 2016, at 10:58 AM, Sylvia Murphy <notifications@github.com mailto:notifications@github.com> wrote:

The user the script checked right before, does not seem to have any funky characters. Would the script eventually bomb on pcmdi9 openIDs? There are still a few in there from power users.

— You are receiving this because you were assigned. Reply to this email directly or view it on GitHub< https://github.com/EarthSystemCoG/COG/issues/1275#issuecomment-216292296>

— You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub https://github.com/EarthSystemCoG/COG/issues/1275#issuecomment-216302225

---

Sylvia Murphy NESII/CIRES/NOAA Earth System Research Laboratory 325 Broadway, Boulder CO 80305 Time Zone: U.S. Mountain Web: http://www.esrl.noaa.gov/nesii/ Email: sylvia.murphy@noaa.gov Phone: 303-497-7753 — You are receiving this because you were assigned. Reply to this email directly or view it on GitHub

[murphysj]

I ran it at CU dev as well but that user does not exist in there. If your run worked we are good. closing.

[murphysj]

Luca, I just ran this script at CU as part of the monthly metrics. It is bombing again, so we must be getting back characters in openIDs. Please take a look.

Checking user: Kishore Dhavala url: http://pcmdi.llnl.gov/user/byopenid/?openid=https://pcmdi.llnl.gov/esgf-idp/openid/kishoredhavala@gmail.com User found.

Checking user: JungKab Wang url: http://cog-esgf.esrl.noaa.gov/user/byopenid/?openid=https://hydra.fsl.noaa.gov/esgf-idp/openid/wangjg User found.

Checking user: Juan Mauricio Bedoya-Soto url: http://pcmdi.llnl.gov/user/byopenid/?openid=https://pcmdi.llnl.gov/esgf-idp/openid/jmbedoya User found. Traceback (most recent call last): File "manage.py", line 12, in execute_from_command_line(sys.argv) File "/data/web/virtualenv/cog-cu/lib/python2.7/site-packages/Django-1.9.4-py2.7.egg/django/core/management/init.py", line 353, in execute_from_command_line utility.execute() File "/data/web/virtualenv/cog-cu/lib/python2.7/site-packages/Django-1.9.4-py2.7.egg/django/core/management/init.py", line 345, in execute self.fetch_command(subcommand).run_from_argv(self.argv) File "/data/web/virtualenv/cog-cu/lib/python2.7/site-packages/Django-1.9.4-py2.7.egg/django/core/management/base.py", line 348, in run_from_argv self.execute(_args, _cmd_options) File "/data/web/virtualenv/cog-cu/lib/python2.7/site-packages/Django-1.9.4-py2.7.egg/django/core/management/base.py", line 399, in execute output = self.handle(_args, _options) File "/data/web/projects/cog_cu/COG/cog/management/commands/sync_users.py", line 27, in handle print "\nChecking user: %s url: %s" % (user, userProfileUrl) UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 27: ordinal not in range(128)

[LucaCinquini]

I changed the script to skip users that give problems, for example:

Checking user at URL: http://pcmdi.llnl.gov/user/byopenid/?openid=https://pcmdi.llnl.gov/esgf-idp/openid/Abhi8013 Error checking URL: http://pcmdi.llnl.gov/user/byopenid/?openid=https://pcmdi.llnl.gov/esgf-idp/openid/Abhi8013 skipping... ('http protocol error', 0, 'got a bad status line', None)

Those users can be deleted manually, if necessary. Running the new script on CoG-CU right now.

[LucaCinquini]

Looks like the script completed correctly...

[murphysj]

I ran the entire script and it did not fail.