Extravi / araa-search

A privacy-respecting, ad-free, self-hosted Google metasearch engine with strong security that offers full API support and utilizes Qwant for images, and DuckDuckGo for auto-complete.
https://araa.extravi.dev
GNU Affero General Public License v3.0
257 stars 23 forks source link

Add view source button to torrents #121

Closed amogusussy closed 9 months ago

amogusussy commented 10 months ago

This PR adds a link under each torrent that links to the original post for the torrent. This is useful for if you're downloading software and want to check if the person who made the torrent is well known and trusted.

A lot of the PR is also just changing a few things so that they conform with the PEP 8 style guide, so that my editor stops giving me tons of errors.

amogusussy commented 10 months ago

I've also just made some changes to add multiprocessing, which makes the torrent requests go a lot faster (~4 times faster). I'll make a separate PR for that once this has been merged.

Extravi commented 10 months ago

I've also just made some changes to add multiprocessing, which makes the torrent requests go a lot faster (~4 times faster). I'll make a separate PR for that once this has been merged.

you can add it into this pr if you want ill wait

amogusussy commented 10 months ago

Ok. Ill do it when i get home later.

Extravi commented 10 months ago

let me know once this pr is ready thanks

amogusussy commented 10 months ago

I think it should be ready now. Can you just check if it's working for torrentgalaxy? I get a ConnectionTimeoutError for torrentgalaxy (it's blocked). I've rewritten torrentgalaxy's search function because it was too complicated to add the view source link to it in the way that it was (this also makes adding things in the future easier), so there's probably gonna be some issues with that, because I've no way of checking if it works or not.

Extravi commented 10 months ago
Process Process-12:
Traceback (most recent call last):
  File "/usr/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap
    self.run()
  File "/usr/lib/python3.10/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/mnt/c/Users/hecker/Desktop/araa-search-torrent_api_values/src/torrent_sites/torrentgalaxy.py", line 44, in search
    list_of_anchors = results.find_all("a")
AttributeError: 'list' object has no attribute 'find_all'
amogusussy commented 10 months ago

Its probably because i said results instead of result.

amogusussy commented 10 months ago

Can you test it now?

amogusussy commented 10 months ago

Can it be merged yet?

Extravi commented 10 months ago

oh I forgot about this I'll test later

Extravi commented 9 months ago
Process Process-7:
Traceback (most recent call last):
  File "/usr/lib/python3.11/multiprocessing/process.py", line 314, in _bootstrap
    self.run()
  File "/usr/lib/python3.11/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/hecker/Downloads/araa-search-torrent_api_values/src/torrent_sites/torrentgalaxy.py", line 52, in search
    "magnet": helpers.apply_trackers(list_of_anchors[4]),
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/hecker/Downloads/araa-search-torrent_api_values/src/helpers.py", line 232, in apply_trackers
    name = get_magnet_name(hash)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/hecker/Downloads/araa-search-torrent_api_values/src/helpers.py", line 227, in get_magnet_name
    return magnet.split("&dn=")[1].split("&tr")[0]
           ^^^^^^^^^^^^^^^^^^^^
TypeError: 'NoneType' object is not callable
amogusussy commented 9 months ago

Could you show the output when you print the list_of_anchors variable?

amogusussy commented 9 months ago

I think the commit I just did should fix this issue.

Extravi commented 9 months ago

That does not return any errors, but it also won't return any results from torrentgalaxy. I tested it both on my ISP's network and a VPN.

Extravi commented 9 months ago

You could try this for testing and download the config file from settings on their website; they do have free servers.

https://protonvpn.com/support/wireguard-manual-linux/

amogusussy commented 9 months ago

Can you check if it's even returning anything in the python request? Try printing the soup variable after it's declared. It might be because the request isn't finishing in the 8 second time period set for the timeout.

amogusussy commented 9 months ago

I've just installed the VPN, and tested it. It ends up getting rate limited though. Even if I use the normal araa.extravi.dev instance, there's no torrentgalaxy results. So it's probably been rate limited too. I think we should probably just remove TG now. Maybe replace it with something else, like YTS.

Extravi commented 9 months ago

I've just installed the VPN, and tested it. It ends up getting rate limited though. Even if I use the normal araa.extravi.dev instance, there's no torrentgalaxy results. So it's probably been rate limited too. I think we should probably just remove TG now. Maybe replace it with something else, like YTS.

yeah i just checked my instance is getting rate limited but i still think we should keep torrentgalaxy because the results are very good

Extravi commented 9 months ago

thank you ill pull this now

Extravi commented 9 months ago

https://araa.extravi.dev/search?q=test&t=torrent

does not work on my server for some reason ill check Screenshot_20240217_001946

Extravi commented 9 months ago

it randomly started working so i cant even test and find out why

Extravi commented 9 months ago

it works only when i already have an ssh session but when i close the ssh session it breaks

Extravi commented 9 months ago

so i need to keep an open ssh session for it to work idk why

Extravi commented 9 months ago

also why was urlparse removed?

amogusussy commented 9 months ago

You could try setting debug=True in app.run in the init.py file. Then see what errors show up. Did you remove torrentgalaxy? It's probably because of some issue with that. I'll make a PR to remove torrentgalaxy in a few minutes.

urlparse was removed because it wasn't used anywhere in the file, so importing it had no purpose.

amogusussy commented 9 months ago

You could try setting debug=True in app.run in the init.py file. Then see what errors show up. Did you remove torrentgalaxy? It's probably because of some issue with that. I'll make a PR to remove torrentgalaxy in a few minutes.

urlparse was removed because it wasn't used anywhere in the file, so importing it had no purpose.

Extravi commented 9 months ago

https://github.com/Extravi/araa-search/commit/fecc69065f9be3ca27893f942b948df9b5007d07

i removed this pr this because it does not fix the issue on my server https://github.com/Extravi/araa-search/pull/124

the issue is not torrentgalaxy its something in this pr because it only works when i have an active ssh session to the server

Extravi commented 9 months ago

Screenshot_20240218_182902

Extravi commented 9 months ago

and torrentgalaxy works fine on my instance with an older version of the code

Extravi commented 9 months ago

Screenshot_20240218_183014 also speed is not much of an issue with the older version so maybe we should just keep using it

Extravi commented 9 months ago

we can however add back the view source button as that works without issue