-
I needed a small warc file for testing, so I took a regular wget download and picked a few files that interconnected and used warcit to create the warc file. When I looked at it in Replayweb.page ther…
-
I mostly using mobile, and nowadays more traffic come from mobile than desktop, as you already know Chrome android cannot install extension, I hope in the future creating WARC without installing anyth…
-
Just to formalise a comment from #887,
fmt/1840 is emerging as a leading format for web archiving. Structurally it is a zip file containing a JSON manifest file and other structural elements along …
-
ipwb indexer currently supports WARC files as input. [HTTP Archive (HAR)](https://dvcs.w3.org/hg/webperf/raw-file/tip/specs/HAR/Overview.html) files may also serve as a trace of HTTP communication, re…
-
This is a meta-issue tracking related work and discussions (moved from https://github.com/ipfs-shipyard/ipfs-companion/issues/96).
## Feasible
- [x] Image Rehosting via HTTP API ([ipfs-compani…
lidel updated
3 months ago
-
When choosing a browser profile during crawl template creation, only profiles where the origin matching seeds should be shown.
_Originally posted by @ikreymer in https://github.com/webrecorder/brow…
-
While [ReSpec](https://github.com/w3c/respec) allows Markdown to be embedded in an HTML file there doesn't appear to be an automated way to turn a standalone Markdown file into a ReSpec HTML file. It …
-
I wanted to offer some thoughts on the /webdata endpoint in general and some possible areas of improvement for supporting other services, such as Webrecorder.
One issue that I see is the time for h…
-
Authsigner on k8 is in a crash loop:
```
root@org2:~# kubectl logs auth-signer-0
2023-04-18 13:15:41,132: INFO - Started server process [1]
2023-04-18 13:15:41,230: INFO - Waiting for applicatio…
-
We have three things which can stop the crawler in the middle of a run:
- `--sizeLimit`: the maximum warc size
- `--timeLimit`: the maximum duration of the crawl
- `--diskUtilization`: the maximum …