arvidn / libtorrent

an efficient feature complete C++ bittorrent implementation
http://libtorrent.org
Other
5.27k stars 997 forks source link

Seed LAN only without tracker #7777

Open MrNonoss opened 4 weeks ago

MrNonoss commented 4 weeks ago

libtorrent version (or branch): 2.0.9.0 platform/architecture: MacOS intel compiler and compiler version: Python3.10

I often need to share private large files across several computers in a Local Area Network, sometimes without internet access. For a fast and efficient file transfer allowing data integrity verification, torrent protocol seems the best option to me

I try to write a python script to (amongst other things):

I am not a very knowledgeable torrent user (nor a very good python guy). From what I understand, to share in my LAN only, without the need to setup trackers, and the hassle to manually add peers in each clients, LSD should be used.

The script seems to be working as:

However, no downloads occurs. I tried with a really tiny file.

Server is MacOS (firewall disabled), client is qBittorrent on Windows (firewall disabled) and BiglyBit on mobile.

I tried:

Not sure how to further troubleshoot the issue. Any help, would be really welcomed ^^

Here are the functions:

# Create Torrent
def create_torrent(file_path, share_dir):
    fs = lt.file_storage()
    lt.add_files(fs, file_path)
    if fs.num_files() == 0:
        print(f"XXXXXXXXXX Error: No files added from {file_path}.")
        return

    t = lt.create_torrent(fs)

    # Manually set piece hashes with a progress callback
    def progress(p):
        print(f"#---> Hashing progress: {p * 100:.2f}%")
    lt.set_piece_hashes(t, os.path.dirname(file_path), progress)

    torrent_file = t.generate()

    # Save torrent file
    torrent_name = os.path.basename(file_path) + ".torrent"
    torrent_path = os.path.join(share_dir, torrent_name)

    with open(torrent_path, "wb") as f:
        f.write(lt.bencode(torrent_file))

    print(f"#---> Torrent Created: {torrent_path}")
    return torrent_path

# Seed Torrent 
def seed_torrent(torrent_file, save_path, local_ip=get_local_ip()):
    # Create a libtorrent session
    session = lt.session()

    # Configure session settings to bind to the local interface
    settings = {
        'enable_dht': False,
        'enable_lsd': True,
        'listen_interfaces': f'{local_ip}:6881,{local_ip}:6891',
        'alert_mask': lt.alert.category_t.all_categories  # Enable all alerts for debugging
    }

    # Apply settings
    session.apply_settings(settings)

    # Load .torrent and seed
    info = lt.torrent_info(torrent_file)
    h = session.add_torrent({
        'ti': info,
        'save_path': save_path,
        'flags': lt.torrent_flags.seed_mode
    })

    print(f'#---> Seeding: {info.name()}')

    # Loop to continue seeding
    try:
        while True:
            s = h.status()
            print(f'Peers: {s.num_peers}, Upload Rate: {s.upload_rate / 1000:.2f} kB/s', end='\r')
            time.sleep(5)  # Adjust the sleep time as needed
    except KeyboardInterrupt:
        print("\nXXXXXXXXXX Seeding stopped manually.")

I noticed that even though I disabled DHT and did not add any trackers, I see some random external IP addresses popping up in the peers. But if I add the flag t.set_priv(True), I need to manually add the IP+port of the server in each clients (no downloads anyway).

PS: the function get_local_ip() is working and returns the actual local IP

# Getting local IP address
def get_local_ip():
    try:
        s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
        s.connect(("10.255.255.255", 1))  # Non routable IP
        local_ip = s.getsockname()[0]  # Extract only the IP address
        s.close()
        return local_ip
    except Exception as e:
        print(f"XXXXXXXXXX Error Impossible to get the local IP address {e}")
        sys.exit(1)  # Exit with an error code
Full script ```python import libtorrent as lt import time import argparse import os import sys import socket import subprocess ############################# # Validate requirements ############################# # Validate "share" directory does exist def ensure_share_directory_exists(): script_dir = os.path.dirname(os.path.abspath(__file__)) # Get the absolute path of the script share_dir = os.path.join(script_dir, 'share') # Path to the 'share' directory if not os.path.exists(share_dir): print(f"XXXXXXXXXX Error: 'share' directory does not exist in: {share_dir}") sys.exit(1) # Exit with an error code print(f"#---> '{share_dir}' directory does exist") return share_dir # Check if file path exists def check_file_path(file_path): if os.path.exists(file_path): print(f"#---> '{file_path}' does exist") else: print(f"XXXXXXXXXX Error: {file_path} is invalid.") sys.exit(1) # Exit with an error code # Getting local IP address def get_local_ip(): try: s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) s.connect(("10.255.255.255", 1)) # Non routable IP local_ip = s.getsockname()[0] # Extract only the IP address s.close() return local_ip except Exception as e: print(f"XXXXXXXXXX Error Impossible to get the local IP address {e}") sys.exit(1) # Exit with an error code # Spacers def show_spacers(text): print(f'\033[1;31m-----------------------------------\033[0m') print(f'\033[1;31m------ {text} ------\033[0m') print(f'\033[1;31m-----------------------------------\033[0m') ############################# # Start HTTP Server subprocess ############################# # Handle HTTP server def run_http_server_in_new_terminal(port, share): command = f"python3 -m http.server {port} --directory '{share}'" if os.name == 'nt': # Windows subprocess.Popen(['start', 'cmd', '/k', command], shell=True, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL) elif os.name == 'posix': # macOS and Linux if sys.platform == 'darwin': # macOS apple_script_command = f'tell application "Terminal" to do script "{command}"' subprocess.Popen(['osascript', '-e', apple_script_command], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL) else: # Linux subprocess.Popen(['gnome-terminal', '--', 'bash', '-c', command], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL) ############################# # Handling the torrent server ############################# # Create Torrent def create_torrent(file_path, share_dir): fs = lt.file_storage() lt.add_files(fs, file_path) if fs.num_files() == 0: print(f"XXXXXXXXXX Error: No files added from {file_path}.") return t = lt.create_torrent(fs) # Manually set piece hashes with a progress callback def progress(p): print(f"#---> Hashing progress: {p * 100:.2f}%") lt.set_piece_hashes(t, os.path.dirname(file_path), progress) torrent_file = t.generate() # Save torrent file torrent_name = os.path.basename(file_path) + ".torrent" torrent_path = os.path.join(share_dir, torrent_name) with open(torrent_path, "wb") as f: f.write(lt.bencode(torrent_file)) print(f"#---> Torrent Created: {torrent_path}") return torrent_path # Generate Magnet Link in a magn.html file within the "share" folder def generate_magnet_link(torrent_file, share_dir): info = lt.torrent_info(torrent_file) magnet_uri = lt.make_magnet_uri(info) magnet_file_path = os.path.join(share_dir, "magn.html") with open(magnet_file_path, "w") as f: f.write(magnet_uri) print(f"#---> Magnet link generated and saved to: {magnet_file_path}") # Seed Torrent def seed_torrent(torrent_file, save_path, local_ip=get_local_ip()): # Create a libtorrent session session = lt.session() # Configure session settings to bind to the local interface settings = { 'enable_dht': False, 'enable_lsd': True, 'listen_interfaces': f'{local_ip}:6881,{local_ip}:6891', 'alert_mask': lt.alert.category_t.all_categories # Enable all alerts for debugging } # Apply settings session.apply_settings(settings) # Load .torrent and seed info = lt.torrent_info(torrent_file) h = session.add_torrent({ 'ti': info, 'save_path': save_path, 'flags': lt.torrent_flags.seed_mode }) print(f'#---> Seeding: {info.name()}') # Loop to continue seeding try: while True: s = h.status() print(f'Peers: {s.num_peers}, Upload Rate: {s.upload_rate / 1000:.2f} kB/s', end='\r') time.sleep(5) # Adjust the sleep time as needed except KeyboardInterrupt: print("\nXXXXXXXXXX Seeding stopped manually.") # Activation if __name__ == "__main__": # Argument and Helper Handling parser = argparse.ArgumentParser(description="Share student folder across the LAN") parser.add_argument('file_path', help='Path of the file to share') args = parser.parse_args() file_path = args.file_path # Run Functions os.system('cls' if os.name == 'nt' else 'clear') # Clear terminal # show_spacers("Server Information") ip = get_local_ip() print(f'#---> HTTP Server available on IP {ip}') # Print IP print(f'#---> Torrent Seed available on port 6881 and 6891') # Print IP # show_spacers("Checking Requirements") share_dir = ensure_share_directory_exists() # Check share directory check_file_path(file_path) # Check shared data run_http_server_in_new_terminal(80, share_dir) # Run HTTP server # show_spacers("Running Torrent") torrent_path = create_torrent(file_path, share_dir) # Generate torrent file generate_magnet_link(torrent_path, share_dir) # show_spacers("Seeding Torrent") seed_torrent(torrent_path, share_dir) # Seed torrent ```

The full script is also handling an HTTP server for an easy way to share the torrent file or magnet link to the clients

arvidn commented 2 weeks ago

by "server" I imagine you mean a bittorrent client seeding the file, right? local service discovery uses SSDP (simple service discovery protocol), which uses IP multicast. Some networks disable multicast, which could be an explanation.

One next step you could take is to capture SSDP traffic in wireshark to see if the packets ever make it to the other end.

The messages are broadcast regularly, based on the setting in settings_pack::local_service_announce_interval. This defaults to 5 minutes. However, since both the seeder and download are expected to broadcast, they should find each other quickly. However, if only one party is broadcasting, it may take up to 5 minutes to discover the other peer.

I just realized you say: "Server seed and see the client as a peer, client also see the server as peer."

So, the peers find each other, it's just that the client isn't downloading from the seed. There's a trouble-shooting guide here:

https://libtorrent.org/troubleshooting.html

MrNonoss commented 2 weeks ago

Many thanks for your answer arvin, I troubleshoot furthermore and adding the alerts in the script showed me info-hash issues leading to understand I had something wrong in the seeding function. Basically I was providing a wrong path instead of the file to seed.

Now it seems better, however, the client (for now qBittorrent) gives a flag "K" saying it is not interested - Local. I suspect it is because we are in the same LAN. My goal is to also create a "simple" client with libtorrent and the same session parameters, so that I hope will bypass this issue.

arvidn commented 2 weeks ago

make sure the client that's supposed to be seeding actually is seeding. i.e. has the whole file. If the save path is wrong, it will try to download instead.

Even if the path is right, it might need to check the piece hashes of the file to ensure it's correct (unless you provide resume data). That may take some time if the file is large.

MrNonoss commented 1 week ago

Based on the status ({s.state}), it is showing "seeding".

I also implemented a verification for the hash with :def progress(p)

But I need to troubleshoot further. Now, the client doesn't see the server anymore and vice versa. Unfortunately this week, I don't have an open network to check if it is a network issue or code issue.

I'll be sure to post the answer when found ^^