search

webrecorder / archiveweb.page

A High-Fidelity Web Archiving Extension for Chrome and Chromium based browsers!
https://chrome.google.com/webstore/detail/webrecorder/fpeoodllldobpkbkabpblcfaogecpndd
GNU Affero General Public License v3.0
794 stars 57 forks source link

Without installing Chrome extension #27

Open johnss opened 3 years ago

johnss commented 3 years ago

I mostly using mobile, and nowadays more traffic come from mobile than desktop, as you already know Chrome android cannot install extension, I hope in the future creating WARC without installing anything will be possible.

Currently using Conifer but it record my username and collection name in the WARC metadata can you make anonymous archiving like in the old wbrc.io or webrecorder.io available again?

ikreymer commented 3 years ago

The browser doesn't allow archiving without either an extension, or some sort of proxy in the middle. The Conifer approach, and the other tool you mentioned, uses such a proxy. There are tradeoffs to each approach. Logging in to other sites will also have to go through a proxy, and they may reject logins form unknown IPs. Without an extension, all the data that'd be written to WARC is what the proxy sees, not what your device would see necessarily, which may be a big difference.

Probably what you want is an ArchiveWeb.page mobile app to do this, I think in theory it's doable, but don't really have resources to work on that right now unfortunately.

Can you say more about your use case? Do you want the data stored on your mobile phone?

johnss commented 3 years ago

How about localhost proxy

  1. Install NodeJS using Termux
  2. Install JS based proxy using npm
  3. Start a proxy server
  4. Tell user to use that proxy in the proxy setting
  5. Proxy will log all request and response then write WARC in the background
johnss commented 3 years ago

Can you say more about your use case?

I used it primarily to archive YouTube video, download the video and reupload to cloud storage, but now i regret it because i only archive the video file and the video title, now i want to archive the entire page because i want to archive the video description, url, comments etc.

Do you want the data stored on your mobile phone?

I will reupload to cloud storage

ghost commented 3 years ago

I used it primarily to archive YouTube video, download the video and reupload to cloud storage, but now i regret it because i only archive the video file and the video title, now i want to archive the entire page because i want to archive the video description, url, comments etc.

@johnss Check out youtube-dl; it can do all of that and more.

johnss commented 3 years ago

How archiving web pages possible will youtube-dl, youtube-dl do not have such functionality, maybe you mean saving video metadata in JSON file right? What i want is view video in browser exactly the same as it was before it was deleted

ghost commented 3 years ago

How archiving web pages possible will youtube-dl, youtube-dl do not have such functionality, maybe you mean saving video metadata in JSON file right? What i want is view video in browser exactly the same as it was before it was deleted

@johnss Yeah, I was talking about saving all of the metadata with youtube-dl. There are two browser on Android that can install Chrome extensions btw (Yandex Browser and Kiwi Browser); it might be worth trying it with them.

tsemachh commented 1 year ago

BTW: In android you can use KIWI Browser which is based on chrome , it allows to install chrome extension

johnss commented 11 months ago

This is electron app, I think convert to an Android app should be easy, if you want to archive using Android browser you can use https://express.archiveweb.page/