philippta / flyscrape

Flyscrape is a command-line web scraping tool designed for those without advanced programming skills.
https://flyscrape.com
Mozilla Public License 2.0
1.02k stars 29 forks source link

Question #19

Closed BluesYoung-web closed 9 months ago

BluesYoung-web commented 9 months ago

Is it only support ssr pages?

leopku commented 9 months ago

I got codes below worked with json endpoint. Maybe it can work with rss endpoint by little changing.

import http from "flyscrape/http";

export default function ({ url }) {
  const resp = http.get(url);
  const result = resp.body;
  console.log(result);
  return { result };
}
flyscrape run httpbin.js --url 'http://httpbin.org/anything'
BluesYoung-web commented 9 months ago

I got codes below worked with json endpoint. Maybe it can work with rss endpoint by little changing.我得到了下面与json端点一起使用的代码。也许它可以通过一点改变与rss端点一起工作。

import http from "flyscrape/http";

export default function ({ url }) {
  const resp = http.get(url);
  const result = resp.body;
  console.log(result);
  return { result };
}
flyscrape run httpbin.js --url 'http://httpbin.org/anything'

I think you maybe misunderstanding. What i really want to ask is: 'If i request a page that rendered in browser instead of rendered in server, does it still work?'

philippta commented 9 months ago

Flyscrape currently does not support browser rendering.

As a possible workaround you could try and integrate a paid services like ScrapingBee, that offer browser rendering by setting them as a proxy (Scraping Bee Proxy Mode).

Disclaimer: I am not affiliated with them or have tried this out.

philippta commented 8 months ago

I am currently working on a hosted proxy service that can render websites in Chrome and I am looking for some testers.

Let me know if you are interested and I can share the details.