JeremiahPetersen / RepoToText

Turn an entire GitHub Repo into a single organized .txt file to use with LLM's (GPT-4, Claude Opus, Gemini, etc)
MIT License
451 stars 57 forks source link

"Submit" Button Does Nothing #3

Open TheTallestHobbit opened 6 months ago

TheTallestHobbit commented 6 months ago

I got everything running via Docker, and I'm on the localhost:3000 web app. I Entered my repo URL in the form of https://github.com/GitHubUser/GitHubRepo and left the documentation URL blank. I have "All Files" selected. I click the "Submit" button and nothing happens. I don't see any debug output in the "docker compose up" window - not sure if I'm supposed to. I don't see any log files in my local copy of the repo. Nothing happens at all when I click "Submit"

JeremiahPetersen commented 6 months ago

@TheTallestHobbit thank you for bringing this up. I am going through the repo again today and will see if i can figure out what is happening here. Will report back after some testing. is there anything that comes up in terminal when you click submit, or no action at all? Also are you on mac/pc/linux ?

JeremiahPetersen commented 6 months ago

@TheTallestHobbit i just updated the repo to remove .dockerfile in the yml. pull the latest change and try it out again. let me know if this fixes the issue for you. if not, I will look into this further. -- i just went through cloning the repo to another computer and ran RepoToText successfully on itself. (https://github.com/JeremiahPetersen/RepoToText) it did take a little bit to populate after clicking submit (8 seconds ish). if the repo is large, it may take longer than you expect. should not take longer than 30 seconds though. you can check in the terminal and see

backend-1 | XXX.XX.X.X - - [07/Mar/2024 00:34:06] "OPTIONS /scrape HTTP/1.1" 200 - backend-1 | XXX.XX.X.X - - [07/Mar/2024 00:34:18] "POST /scrape HTTP/1.1" 200 -

you then should see it populate in the text area and also in the "Data" folder in the repo.

shaayaansayed commented 5 months ago

UPDATE: I was using the wrong URL. You want to use the HTTPS URL under the green "Code" button on the repo's page. I also needed to strip the ".git" extension.

The original URL was: https://github.com/smart-on-fhir/client-py.git which threw an error. But https://github.com/smart-on-fhir/client-py worked.


I'm getting the following error. Note, I'm on mac.

Looks like its failing at:

repo = github_instance.get_repo(self.repo_name)

I'm using the https link in the "Enter Github repo URL" entry, specifically: https://github.com/smart-on-fhir/client-py/tree/master which points to here.

backend-1   | 192.168.65.1 - - [13/Mar/2024 23:57:09] "OPTIONS /scrape HTTP/1.1" 200 -
backend-1   | [2024-03-13 23:57:09,972] ERROR in app: Exception on /scrape [POST]
backend-1   | Traceback (most recent call last):
backend-1   |   File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1463, in wsgi_app
backend-1   |     response = self.full_dispatch_request()
backend-1   |   File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 872, in full_dispatch_request
backend-1   |     rv = self.handle_user_exception(e)
backend-1   |   File "/usr/local/lib/python3.8/site-packages/flask_cors/extension.py", line 176, in wrapped_function
backend-1   |     return cors_after_request(app.make_response(f(*args, **kwargs)))
backend-1   |   File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 870, in full_dispatch_request
backend-1   |     rv = self.dispatch_request()
backend-1   |   File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 855, in dispatch_request
backend-1   |     return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)  # type: ignore[no-any-return]
backend-1   |   File "RepoToText.py", line 133, in scrape
backend-1   |     filename = scraper.run()
backend-1   |   File "RepoToText.py", line 107, in run
backend-1   |     files_data = self.fetch_all_files()
backend-1   |   File "/usr/local/lib/python3.8/site-packages/decorator.py", line 232, in fun
backend-1   |     return caller(func, *(extras + args), **kw)
backend-1   |   File "/usr/local/lib/python3.8/site-packages/retry/api.py", line 73, in retry_decorator
backend-1   |     return __retry_internal(partial(f, *args, **kwargs), exceptions, tries, delay, max_delay, backoff, jitter,
backend-1   |   File "/usr/local/lib/python3.8/site-packages/retry/api.py", line 33, in __retry_internal
backend-1   |     return f()
backend-1   |   File "RepoToText.py", line 65, in fetch_all_files
backend-1   |     repo = github_instance.get_repo(self.repo_name)
backend-1   |   File "/usr/local/lib/python3.8/site-packages/github/MainClass.py", line 427, in get_repo
backend-1   |     headers, data = self.__requester.requestJsonAndCheck("GET", url)
backend-1   |   File "/usr/local/lib/python3.8/site-packages/github/Requester.py", line 537, in requestJsonAndCheck
backend-1   |     return self.__check(*self.requestJson(verb, url, parameters, headers, input, self.__customConnection(url)))
backend-1   |   File "/usr/local/lib/python3.8/site-packages/github/Requester.py", line 598, in __check
backend-1   |     raise self.createException(status, responseHeaders, data)
backend-1   | github.GithubException.UnknownObjectException: 404 {"message": "Not Found", "documentation_url": "https://docs.github.com/rest"}
backend-1   | 192.168.65.1 - - [13/Mar/2024 23:57:09] "POST /scrape HTTP/1.1" 500 -

EDIT: I tried running it with the https "git" link as well, but got the error below. This time I used: https://github.com/smart-on-fhir/client-py.git

backend-1   | Traceback (most recent call last):
backend-1   |   File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1463, in wsgi_app
backend-1   |     response = self.full_dispatch_request()
backend-1   |   File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 872, in full_dispatch_request
backend-1   |     rv = self.handle_user_exception(e)
backend-1   |   File "/usr/local/lib/python3.8/site-packages/flask_cors/extension.py", line 176, in wrapped_function
backend-1   |     return cors_after_request(app.make_response(f(*args, **kwargs)))
backend-1   |   File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 870, in full_dispatch_request
backend-1   |     rv = self.dispatch_request()
backend-1   |   File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 855, in dispatch_request
backend-1   |     return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)  # type: ignore[no-any-return]
backend-1   |   File "RepoToText.py", line 133, in scrape
backend-1   |     filename = scraper.run()
backend-1   |   File "RepoToText.py", line 107, in run
backend-1   |     files_data = self.fetch_all_files()
backend-1   |   File "/usr/local/lib/python3.8/site-packages/decorator.py", line 232, in fun
backend-1   |     return caller(func, *(extras + args), **kw)
backend-1   |   File "/usr/local/lib/python3.8/site-packages/retry/api.py", line 73, in retry_decorator
backend-1   |     return __retry_internal(partial(f, *args, **kwargs), exceptions, tries, delay, max_delay, backoff, jitter,
backend-1   |   File "/usr/local/lib/python3.8/site-packages/retry/api.py", line 33, in __retry_internal
backend-1   |     return f()
backend-1   |   File "RepoToText.py", line 65, in fetch_all_files
backend-1   |     repo = github_instance.get_repo(self.repo_name)
backend-1   |   File "/usr/local/lib/python3.8/site-packages/github/MainClass.py", line 427, in get_repo
backend-1   |     headers, data = self.__requester.requestJsonAndCheck("GET", url)
backend-1   |   File "/usr/local/lib/python3.8/site-packages/github/Requester.py", line 537, in requestJsonAndCheck
backend-1   |     return self.__check(*self.requestJson(verb, url, parameters, headers, input, self.__customConnection(url)))
backend-1   |   File "/usr/local/lib/python3.8/site-packages/github/Requester.py", line 598, in __check
backend-1   |     raise self.createException(status, responseHeaders, data)
backend-1   | github.GithubException.UnknownObjectException: 404 {"message": "Not Found", "documentation_url": "https://docs.github.com/rest/repos/repos#get-a-repository"}
spookyuser commented 5 months ago

One possible reason might be the port 5000 weirdness on macos e.g https://github.com/dotnet/aspnetcore/issues/48932

But yeah also didn't work for me on mac, i was using codespaces though soo maybe that was why

JeremiahPetersen commented 5 months ago

thank you for sharing data on this. it seems it may be a mac specific issue. i will leave this issue raised while i continue researching a solution.