Integrating with flask and handling routes

[gbopola]

Hi, I want to build a full stack application scraping yellow pages where the user can enter the yellow pages url that they want to scrape and get the scraped data. I'm having issues integrating flask and the api routes in the application using requests module. There is a conflict because the @request decorator is the same name as the requests module in Flask. Do you guys have any examples of how this can be done? Thanks in advance.

from botasaurus import *
from flask import Flask, jsonify, request
from scraper.yp_usa_scraper import *
from flask_cors import CORS

app = Flask(__name__)
CORS(app)

@app.route('/scrape/yp-usa', methods=["POST"])
@request(use_stealth=True)
def scrape_heading_task(request: AntiDetectRequests, data):
    data = request.get_json()
    response = request.get('https://www.yell.com/ucs/UcsSearchAction.do?scrambleSeed=1475848896&keywords=hairdressers&location=hatfield%2C+hertfordshire')
    return response.text

if __name__ == "__main__":
    # Run the Flask development server
    app.run(debug=True)

AttributeError: 'AntiDetectRequests' object has no attribute 'get_json'

[Chetan11-dev]

I recommend separating it in 2 functions as follows:

from botasaurus import *
from flask import Flask, jsonify, request
from scraper.yp_usa_scraper import *
from flask_cors import CORS

app = Flask(__name__)
CORS(app)

@request(use_stealth=True)
def scrape_heading_task(request: AntiDetectRequests, data):
    data = request.get_json()
    response = request.get('https://www.yell.com/ucs/UcsSearchAction.do?scrambleSeed=1475848896&keywords=hairdressers&location=hatfield%2C+hertfordshire')
    return response.text

@app.route('/scrape/yp-usa', methods=["POST"])
def scrape_yp(request):
    data = request.get_json()
    return scrape_heading_task(data )

if __name__ == "__main__":
    # Run the Flask development server
    app.run(debug=True)

[gbopola]

thank you :)