I need to crawl the e-commerce product page of the product name, pictures and videos,but you can't get accurate video and image urls
Code down
def create_speech_graph(html):
speech_graph = SmartScraperGraph(
prompt="List me all the images with their visual description",
source= html,
config=graph_config,
)
return speech_graph
speech_graph = create_speech_graph(html)
if speech_graph:
result = speech_graph.run()
output = json.dumps(result, indent=2)
line_list = output.split("\n") # Sort of line replacing "\n" with a new line
for line in line_list:
print(line)
VinciGit00
commented
1 month ago
I need to crawl the e-commerce product page of the product name, pictures and videos,but you can't get accurate video and image urls
Code down def create_speech_graph(html): speech_graph = SmartScraperGraph( prompt="List me all the images with their visual description", source= html, config=graph_config, ) return speech_graph
给定一个url下载html内容
def download_html(url): import requests response = requests.get(url) return response.text
执行download_html方法，在执行create_speech_graph方法
html = download_html("https://www.amazon.com/Misxi-Waterproof-Anti-Scratch-Protector-Transparent/dp/B0C9PSGY6Q?th=1")
如果speech_graph不为空，则运行speech
speech_graph = create_speech_graph(html) if speech_graph: result = speech_graph.run() output = json.dumps(result, indent=2) line_list = output.split("\n") # Sort of line replacing "\n" with a new line for line in line_list: print(line)