icy / google-group-crawler

[Deprecated] Get (almost) original messages from google group archives. Your data is yours.
215 stars 38 forks source link

A 1-year bug and fix! #3

Closed icy closed 9 years ago

icy commented 9 years ago

When I was in Paris, Daniel sent an email to ask for help. Unfortunately, I couldn't figure out the cause after quick search; and even worse, I almost forgot his email.

The problem comes from a lynx configuration. For some reason, it's UTF-8 by default. And after some upgrades, it switched to an ISO-xxx charset and I haven't awared of that change on my laptop. Tonight I tried with Daniel's archive and could reproduce the problem. And voilà, I could get it fixed!

I put this issue on public domain. Please accept my apologies, Daniel.

From: Daniel Lind______
To: kyanh@theslinux.org
Subject: craw.sh
Date: Wed, 12 Mar 2014 16:54:26 -0700

Greetings,

Your craw.sh script looks incredibly useful (I also enjoy craw's sense of
humour).

Forgive the ignorant question, but I'm not much of a .bash programmer.

When ever I try to use it (using, for example apml-public as the target
group) I get something like:

mkdir: created directory `./apml-public'
:: Creating './apml-public//threads.0' with 'forum/apml-public'

Looking up groups.google.com
Making HTTPS connection to groups.google.com

lynx: Can't access startfile
https://groups.google.com/forum/?_escaped_fragment_=forum/apml-public
cat: ./apml-public//msgs.*: No such file or directory

Am I doing something boneheaded, or have changes in google's system
rendered craw temporarily out of comission.

Best regards, and thank you for sharing your code (and your wit),
Daniel
icy commented 9 years ago

Fixed in https://github.com/icy/google-group-crawler/commit/f061f38810fdc4f8bf68669705654eb0ef71bfad . See also https://github.com/icy/google-group-crawler/commit/75b726b183843690e8f6a7fe0fcc81a8030b4789

berashith commented 9 years ago

No apologies required! Totally awesome that you would take the time to fix this so long afterwards. Beer shall wend its way towards you!