Justintime50 / github-archive

A powerful tool to concurrently clone, pull, or fork user and org repos and gists to create a GitHub archive.
MIT License
186 stars 48 forks source link

token not working with users, is this by design? #52

Closed tg12 closed 1 year ago

tg12 commented 1 year ago

Using the Token would only allow you to download your own repos, Is it possible to combine the users and a token to download other users repos? Am I misinterpreting the documentation here?

I get the error

github.GithubException.RateLimitExceededException: 403 {"message": "API rate limit exceeded for xxxxxxxx. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.)", "documentation_url": "https://docs.github.com/rest/overview/resources-in-the-rest-api#rate-limiting"}

Justintime50 commented 1 year ago

The token should allow to clone/pull anyone's public repos if used correctly. Can you provide the command you ran so I can see your usage? (Please don't include your token)

tg12 commented 1 year ago

Yeah sure thing, So for example my user is obviously tg12, I want to download all the BBC repos as pure example. I get that there's a rate limit so fair enough, I provide a token and then this happens....

github-archive --users bbc --clone --https --token 8435y83453485034580345u345(fake)

I added https as well it happens with SSH.

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.9/bin/github-archive", line 8, in <module>
    sys.exit(main())
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/github_archive/cli.py", line 191, in main
    GithubArchiveCli().run()
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/github_archive/cli.py", line 187, in run
    github_archive.run()
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/github_archive/archive.py", line 125, in run
    self.users.remove(self.authenticated_username)
ValueError: list.remove(x): x not in list
Justintime50 commented 1 year ago

Ah yes, do not include the https flag when using a token. The https flag is meant to bypass the need of a token but you will get rate limited (per GitHub since it's not authenticated); however, using the token param should work. Also, I assume when you say BBC you mean this? https://github.com/bbc If so, this is an organization and so you should use the --orgs flag instead of --users.

Let me know if that works. If it does, I'll add a check in the tool to make sure users don't pass path the https and token flags together to warn about this in the future.

tg12 commented 1 year ago

Yeah, Awesome! Thanks so can confirm if you use https and token all it does it clone your own personal repos and then any user that you pass it gets this error ...

`# Cloning missing personal repos...

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.9/bin/github-archive", line 8, in <module>
    sys.exit(main())
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/github_archive/cli.py", line 191, in main
    GithubArchiveCli().run()
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/github_archive/cli.py", line 187, in run
    github_archive.run()
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/github_archive/archive.py", line 125, in run
    self.users.remove(self.authenticated_username)
ValueError: list.remove(x): x not in list
# GitHub Archive started...

`

Justintime50 commented 1 year ago

Oh 😂 That is almost certainly a bug. I've patched this and will release this in a new version now.

tg12 commented 1 year ago

ha! Phew I thought I was misreading the documentation. It's working as expected with token and without https flag and changed to org for me.

tg12 commented 1 year ago

Thanks for your swift help and response! I'll close this.

8465231 commented 1 year ago

Was the patch for this ever released? I am still getting the same error. It works on my own repos but no one elses.

Justintime50 commented 1 year ago

Was the patch for this ever released? I am still getting the same error. It works on my own repos but no one elses.

What version of the tool are you using? What command are you running? Please do not include your token

8465231 commented 1 year ago

What version of the tool are you using? What command are you running? Please do not include your token

I do not see a way to see the version? It was just installed today with pip3 though.

I was testing with this command

github-archive -t token -p -u 0x0ece -v

It gives this error

Pulling changes to personal repos...

Traceback (most recent call last): File "/usr/local/bin/github-archive", line 8, in sys.exit(main()) ^^^^^^ File "/usr/local/lib/python3.11/site-packages/github_archive/cli.py", line 191, in main GithubArchiveCli().run() File "/usr/local/lib/python3.11/site-packages/github_archive/cli.py", line 187, in run github_archive.run() File "/usr/local/lib/python3.11/site-packages/github_archive/archive.py", line 126, in run self.users.remove(self.authenticated_username) ValueError: list.remove(x): x not in list

8465231 commented 1 year ago

opps, used the wrong user at first, updated the command with the correct user, got too many windows open lol.

Same issue though.

Justintime50 commented 1 year ago

So it's working as expected?

8465231 commented 1 year ago

So it's working as expected?

No, still same issue sadly, I just grabbed an org for the test at first instead of a user. I corrected it and then edited the above post.

Still refuses to download any user repos and crashes with the error above.

Justintime50 commented 1 year ago

Will continue this convo on #55