haghish / github

a module for building, searching, installing, managing, and mining Stata packages from GitHub
http://haghish.github.io/github/
100 stars 41 forks source link

HTTPS problem with Stata 15.1 (only problem with github search) #6

Closed jpazvd closed 5 years ago

jpazvd commented 5 years ago

Hello,

I am running the github ado in Stata 15.1. and I keep getting the follow error message when I use the search option

. github search azevedo the GitHub API is not responsive right now. Try again in 10 or 20 seconds. this can happen if you search GitHub very frequent... r; t=0.19 14:29:08

The options query, install and uninstall work fine in the same machine on the same Stata instalation.

The option search works fine in my personal desktop running Stata 14.

Do others have this problem? Is there a fix?

Best regards,

JP

haghish commented 5 years ago

before we proceed, what version of github are you using?

the error message is clear, your machine cannot access the GitHub API, this happens occasionally, or when you have searched the GitHub API many times within a short period and have exceeded your API usage limit. It will be fixed automatically, i.e. GitHub will allow you to contact the API again after a while. When you use a different machine, you will get a different IP so that is why you can get different results from identical programs when you reach the limit (actually you should wait much more than 20 seconds, I will correct the error message)

please reopen the issue if the error remains

jpazvd commented 5 years ago

Hello,

If I type :

which github

I do not get a version number.

However, on the help file it shows github version 1.5.1

I am afraid the problem might be of a different nature. I just ran the four commands manually (with set rmsg on):

github search azevedo github install haghish/markdoc github query haghish/markdoc github search azevedo

As you can see the log below, both github install and github query worked fine. The only problem was the github search (even after waiting 4 minutes minute execution).

============= Stata Log =====================

. github search azevedo the GitHub API is not responsive right now. Try again in 10 or 20 seconds. this can happen if you search GitHub very frequent... r; t=0.24 16:21:50

. github install haghish/markdoc checking markdoc consistency and verifying not already installed... installing into c:\ado\plus... installation complete.

Checking package dependencies installing markdoc package dependencies:

. github install haghish/weaver , version(3.4.1) checking weaver consistency and verifying not already installed... installing into c:\ado\plus... installation complete.

Checking package dependencies installing weaver package dependencies:

. github install haghish/statax , replace checking statax consistency and verifying not already installed... installing into c:\ado\plus... installation complete.

Checking package dependencies statax package has no dependency

. end of do-file

. github install haghish/md2smcl checking md2smcl consistency and verifying not already installed... installing into c:\ado\plus... installation complete.

Checking package dependencies md2smcl package has no dependency

. . end of do-file r; t=19.61 16:22:22

. github query haghish/markdoc


Version Release Date Install

4.2.0 2018-11-23 Install 4.1.0 2018-11-22 Install 4.0.5 2018-11-22 Install 4.0.3 2018-11-21 Install 4.0.2 2018-11-11 Install 4.0.1 2018-11-09 Install 4.0.0 2018-10-06 Install 3.9.7 2017-04-04 Install 3.9.6 2017-01-03 Install 3.9.5 2016-12-14 Install 3.9.4 2016-12-05 Install 3.9.3 2016-12-01 Install 3.9.2 2016-11-30 Install 3.9.1 2016-11-28 Install 3.9.0 2016-11-24 Install 3.8.9 2016-11-22 Install 3.8.8 2016-11-16 Install 3.8.7 2016-11-10 Install 3.8.6 2016-11-10 Install 3.8.5 2016-10-16 Install 3.8.4 2016-10-13 Install 3.8.3 2016-10-03 Install 3.8.2 2016-10-01 Install 3.8.1 2016-09-29 Install 3.8.0 2016-09-24 Install 3.7.9 2016-09-20 Install 3.7.8 2016-09-19 Install 3.7.7 2016-09-18 Install 3.7.6 2016-09-13 Install 3.7.5 2016-09-08 Install

r; t=1.04 16:22:31

. github search azevedo the GitHub API is not responsive right now. Try again in 10 or 20 seconds. this can happen if you search GitHub very frequent... r; t=0.19 16:22:35

. github search azevedo the GitHub API is not responsive right now. Try again in 10 or 20 seconds. this can happen if you search GitHub very frequent... r; t=0.24 16:24:18 .

In Stata 14, I have github 1.3.3 and the github search works fine.

I have tried to got back to github 1.3.3 but I am not sure which syntax to use. I have tried both below with no success.

github haghish/github, version("1.3.3")

github install github, version("1.3.3")

Any help will be much appreciated.

Best regards,

JP

From: E. F. Haghish notifications@github.com Sent: Saturday, December 8, 2018 2:56 PM To: haghish/github github@noreply.github.com Cc: Joao Pedro Wagner De Azevedo jazevedo@worldbank.org; Author author@noreply.github.com Subject: Re: [haghish/github] HTTPS problem with Stata 15.1 (only problem with github search) (#6)

[External]

before we proceed, what version of github are you using?

the error message is clear, your machine cannot access the GitHub API, this happens occasionally, or when you have searched the GitHub API many times within a short period and have exceeded your API usage limit. It will be fixed automatically, i.e. GitHub will allow you to contact the API again after a while. When you use a different machine, you will get a different IP so that is why you can get different results from identical programs when you reach the limit (actually you should wait much more than 20 seconds, I will correct the error message)

please reopen the issue if the error remains

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/haghish/github/issues/6#issuecomment-445485473, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AdqfKWFLy1Zzxe5x6SSeLAh4ewULfLQ3ks5u3BlFgaJpZM4ZJyGC.

jpazvd commented 5 years ago

If I type : which github

I do not get a version number.

However, on the help file it shows github version 1.5.1

I am afraid the problem might be of a different nature. I just ran the four commands manually (with set rmsg on):

github search azevedo github install haghish/markdoc github query haghish/markdoc github search azevedo

As you can see the log below, both github install and github query worked fine. The only problem was the github search (even after waiting 4 minutes minute execution).

============= Stata Log =====================

. github search azevedo the GitHub API is not responsive right now. Try again in 10 or 20 seconds. this can happen if you search GitHub very frequent... r; t=0.24 16:21:50

. github install haghish/markdoc checking markdoc consistency and verifying not already installed... installing into c:\ado\plus... installation complete.

Checking package dependencies installing markdoc package dependencies:

. github install haghish/weaver , version(3.4.1) checking weaver consistency and verifying not already installed... installing into c:\ado\plus... installation complete.

Checking package dependencies installing weaver package dependencies:

. github install haghish/statax , replace checking statax consistency and verifying not already installed... installing into c:\ado\plus... installation complete.

Checking package dependencies statax package has no dependency

. end of do-file

. github install haghish/md2smcl checking md2smcl consistency and verifying not already installed... installing into c:\ado\plus... installation complete.

Checking package dependencies md2smcl package has no dependency

. . end of do-file r; t=19.61 16:22:22

. github query haghish/markdoc


Version Release Date Install

4.2.0 2018-11-23 Install 4.1.0 2018-11-22 Install 4.0.5 2018-11-22 Install 4.0.3 2018-11-21 Install 4.0.2 2018-11-11 Install 4.0.1 2018-11-09 Install 4.0.0 2018-10-06 Install 3.9.7 2017-04-04 Install 3.9.6 2017-01-03 Install 3.9.5 2016-12-14 Install 3.9.4 2016-12-05 Install 3.9.3 2016-12-01 Install 3.9.2 2016-11-30 Install 3.9.1 2016-11-28 Install 3.9.0 2016-11-24 Install 3.8.9 2016-11-22 Install 3.8.8 2016-11-16 Install 3.8.7 2016-11-10 Install 3.8.6 2016-11-10 Install 3.8.5 2016-10-16 Install 3.8.4 2016-10-13 Install 3.8.3 2016-10-03 Install 3.8.2 2016-10-01 Install 3.8.1 2016-09-29 Install 3.8.0 2016-09-24 Install 3.7.9 2016-09-20 Install 3.7.8 2016-09-19 Install 3.7.7 2016-09-18 Install 3.7.6 2016-09-13 Install 3.7.5 2016-09-08 Install

r; t=1.04 16:22:31

. github search azevedo the GitHub API is not responsive right now. Try again in 10 or 20 seconds. this can happen if you search GitHub very frequent... r; t=0.19 16:22:35

. github search azevedo the GitHub API is not responsive right now. Try again in 10 or 20 seconds. this can happen if you search GitHub very frequent... r; t=0.24 16:24:18 .

In Stata 14, I have github 1.3.3 and the github search works fine.

I have tried to got back to github 1.3.3 but I am not sure which syntax to use. I have tried both below with no success.

github haghish/github, version("1.3.3")

github install github, version("1.3.3")

Any help will be much appreciated.

Best regards,

JP

haghish commented 5 years ago

ok so:

  1. only the search subcommand contacts the search API. this is a whole different procedure for installing, uninstalling, query, etc.

  2. You can surely install an older version of github, which is why we support GitHub ;) the syntax is the same as of any package. First make sure you get the right version you wish to install and then type github install username/repository, version(dada) and it should be fine

  3. the problem still is that GitHub website has locked your computer. Which means, if you use a different version of Stata on the same machine you should still get nothing because your IP is blocked.

  4. If your IP address is blocked by GitHub temporarily, there are ways you can check that. I don't quite remember the command line code that you could run to see whether your access is denied or not but you will find these links helpful:

haghish commented 5 years ago

You can try updating your github on the other machine with Stata 14 and there should be no problem. Also, on my machines I do not get any problem doing your search...

jpazvd commented 5 years ago

Thanks. I will try tomorrow. I have been trying for the last days, and it has always failed. Even behind a VPN.... It is quite strange.

ok so:

  1. only the search subcommand contacts the search API. this is a whole different procedure for installing, uninstalling, query, etc.
  2. You can surely install an older version of github, which is why we support GitHub ;) the syntax is the same as of any package. First make sure you get the right version you wish to install and then type github install username/repository, version(dada) and it should be fine
  3. the problem still is that GitHub website has locked your computer. Which means, if you use a different version of Stata on the same machine you should still get nothing because your IP is blocked.
  4. If your IP address is blocked by GitHub temporarily, there are ways you can check that. I don't quite remember the command line code that you could run to see whether your access is denied or not but you will find these links helpful:

You can try updating your github on the other machine with Stata 14 and there should be no problem. Also, on my machines I do not get any problem doing your search...

jpazvd commented 5 years ago

Thanks. I have updated github to version 1.5.5 on my desktop running Stata 14. And github search is working fine.

You can try updating your github on the other machine with Stata 14 and there should be no problem. Also, on my machines I do not get any problem doing your search...

haghish commented 5 years ago

Ok, so there is something with the other machine, I hope it'd be fixed by tomorrow. Otherwise, I look forward to the bug reports. thank you for your effort and bug report, btw

jpazvd commented 5 years ago

Thanks for your help. I will keep you posted. Best, JP

From: E. F. Haghish notifications@github.com Sent: Saturday, December 8, 2018 5:19 PM To: haghish/github github@noreply.github.com Cc: Joao Pedro Wagner De Azevedo jazevedo@worldbank.org; Author author@noreply.github.com Subject: Re: [haghish/github] HTTPS problem with Stata 15.1 (only problem with github search) (#6)

[External]

Ok, so there is something with the other machine, I hope it'd be fixed by tomorrow. Otherwise, I look forward to the bug reports. thank you for your effort and bug report, btw

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/haghish/github/issues/6#issuecomment-445494149, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AdqfKeHl3U6yjj9glTLcioWN8Tz7x3u4ks5u3DrlgaJpZM4ZJyGC.

jpazvd commented 5 years ago

Hello, I have tried again. And it is still not working. below follows the full log file. (set trace on) Best, JP

=================================================

. github search azevedo ------------------------------------------------------------------------------ begin github ---

.

haghish commented 5 years ago

It is difficult to help you without having access to your computer. I can only make suggestions to help you. There could be a possibility that there would be a bug from:

Please report information about your machine and I will think of a helpful procedure.


The problem is that this json file cannot be copied by your Stata. From this point, we know that the error is not caused by github package. But it'd be interesting to see why you cannot copy this.

https://api.github.com/search/repositories?q=azevedo+language:stata+in:name,description&sort=stars&order=desc&per_page=50 

you may try copying this file from your Stata and see if you can copy it to a file and see if the error can be reproduced.

jpazvd commented 5 years ago

I fully understand. I have a Stata ado called wbopendata which also brings data from an API. In order to help users debug I have written a simple checklist to make sure if all the API calls are working, properly. I wonder is something of this nature could be adapted to GitHub

if ("`l2'" == "") {
    noi di ""
    noi di as err "{p 4 4 2} Sorry... No data was downloaded for " as result "`queryspec2'. {p_end}"
    noi di ""
    noi dis as text `"{p 4 4 2} (1) Please check your internet connection by {browse "http://data.worldbank.org/" :clicking here}, if does not work please check with your internet provider or IT support, otherwise... {p_end}"'
    noi dis as text `"{p 4 4 2} (2) Please check your access to the World Bank API by {browse "http://api.worldbank.org/indicator" :clicking here}, if does not work please check with your firewall settings or internet provider or IT support, otherwise...  {p_end}"'
    noi dis as text `"{p 4 4 2} (3) Please check the availability of your indicator or topic by {browse "`queryspec'" :clicking here}. If the paramater value is not valid...  {p_end}"'
    noi dis as text `"{p 4 4 2} (4) Please check the list of available indicator(s) or topic(s) in the help {help wbopendata} or by visiting the {browse "http://data.worldbank.org/querybuilder" :API query builder}, if all the above seems fine...  {p_end}"'
    noi dis as text `"{p 4 4 2} (5) Please consider adjusting your Stata timeout parameters. For more details see {help netio}. {p_end}
    noi dis as text `"{p 4 4 2} (6) Please send us an email to report this error by {browse "mailto:data@worldbank.org, ?subject= wbopendata query error at `c(current_date)' `c(current_time)': `queryspec' "  :clicking here} or writing to:  {p_end}"'
    noi dis as result "{p 12 4 2} email: " as input "data@worldbank.org  {p_end}"
    noi dis as result "{p 12 4 2} subject: " as input `"wbopendata query error at `c(current_date)' `c(current_time)': `queryspec'  {p_end}"'
    noi di ""
    noi di ""
    break
    exit 20
}

Best regards, JP

From: E. F. Haghish notifications@github.com Sent: Sunday, December 9, 2018 3:12 PM To: haghish/github github@noreply.github.com Cc: Joao Pedro Wagner De Azevedo jazevedo@worldbank.org; Author author@noreply.github.com Subject: Re: [haghish/github] HTTPS problem with Stata 15.1 (only problem with github search) (#6)

[External]

It is difficult to help you without having access to your computer. I can only make suggestions to help you. There could be a possibility that there would be a bug from:

Please report information about your machine and I will think of a helpful procedure.


The problem is that this json file cannot be copied by your Stata. From this point, we know that the error is not caused by github package. But it'd be interesting to see why you cannot copy this.

https://api.github.com/search/repositories?q=azevedo+language:stata+in:name,description&sort=stars&order=desc&per_page=50

you may try copying this file from your Stata and see if you can copy it to a file and see if the error can be reproduced.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/haghish/github/issues/6#issuecomment-445567446, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AdqfKcBohBvJ-2ipC_hBdu89GS0_j9Q5ks5u3W6FgaJpZM4ZJyGC.

jpazvd commented 5 years ago

I found the cause and solution… It is stata checksum command…

I have to make sure that checksum is off

set checksum off, perm

you should be able to replicate my problem in your machine

* Problem *** set checksum on github search azevedo, in(all) language(Stata) all

* Solution *** set checksum off github search azevedo, in(all) language(Stata) all

I still have not figure out why some installations of Stata the checksum is turned on, while in others it is turned off, what is worse is that the option permanently does not seem to work….

What is more worrisome is why APIs do not work with a checksum option (which in principle is useful to preserve the integrity of the transferred data)

Best regards,

JP

From: E. F. Haghish notifications@github.com Sent: Sunday, December 9, 2018 3:12 PM To: haghish/github github@noreply.github.com Cc: Joao Pedro Wagner De Azevedo jazevedo@worldbank.org; Author author@noreply.github.com Subject: Re: [haghish/github] HTTPS problem with Stata 15.1 (only problem with github search) (#6)

[External]

It is difficult to help you without having access to your computer. I can only make suggestions to help you. There could be a possibility that there would be a bug from:

Please report information about your machine and I will think of a helpful procedure.


The problem is that this json file cannot be copied by your Stata. From this point, we know that the error is not caused by github package. But it'd be interesting to see why you cannot copy this.

https://api.github.com/search/repositories?q=azevedo+language:stata+in:name,description&sort=stars&order=desc&per_page=50

you may try copying this file from your Stata and see if you can copy it to a file and see if the error can be reproduced.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/haghish/github/issues/6#issuecomment-445567446, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AdqfKcBohBvJ-2ipC_hBdu89GS0_j9Q5ks5u3W6FgaJpZM4ZJyGC.

haghish commented 5 years ago

That would have taken ages to figure out. Thank you for debugging this problem. And it's weird why this error occurs. I haven't used checksum before...