omkarcloud / botasaurus

The All in One Framework to build Awesome Scrapers.
https://www.omkar.cloud/botasaurus/
MIT License
1.35k stars 123 forks source link

Botasaurus Backend Service and Browser #131

Closed CrazedCoderNate closed 3 months ago

CrazedCoderNate commented 3 months ago
def run():
    if "--help" in sys.argv:
            show_help()
    else:
        if not Server.get_scrapers_names():
            raise RuntimeError("No scrapers found. Please add a scraper using Server.add_scraper.")

        if len(sys.argv) == 1:
            print_frontend_run_message()
            killfrontendandbackendports()
            # No arguments provided, run both backend and frontend
            run_backend_in_thread()
            open_browser_in_thread()
            run_frontend(False)
        elif sys.argv[1] == "install":
            install()
        elif sys.argv[1] == "backend":
            # Argument "backend" provided, run only backend
            killbackendport()
            run_backend()
        elif sys.argv[1] == "dev":
            print_frontend_run_message()
            killfrontendandbackendports()
            # No arguments provided, run both backend and frontend
            run_backend_in_thread()
            open_browser_in_thread()
            run_frontend(True)
        else:
            print(f"Error: No such command: {sys.argv[1]}")
            print("Try '--help' for help.")
            sys.exit(1) 

Found within botasaurus_server/run.py

It would be very nice to have something similar to the no argument where we don't run the frontend at all. My use case uses browser and backend only. Something like:

elif sys.argv[1] == "backend_browser":
            killbackendports()
            run_backend_in_thread()
            open_browser_in_thread()

The front-end server is quite heavy and it would be nice to be able to spin up lightweight pods with just the back-end service of drivers. I will do this manually by editing the source, but an out of the box solution may be useful for future users.