Open GoogleCodeExporter opened 9 years ago
It would make sense, though I don't know how to implement it or when I will get
around to it (especially since I'm unfamiliar with testing proxies), so if
someone
wants to give it a go, go!
Original comment by nekoh...@gmail.com
on 7 Apr 2008 at 1:16
As an example, look at the patch that added Proxy support to Conseil :
http://conseil.fritalk.com/attachment/ticket/68/protocol_launchpadweb.py.diff
Original comment by zepl...@gmail.com
on 7 Apr 2008 at 1:25
Nekohayo, I've tried and it's very simple to add proxy support for a given
watch.
Example : in watch_web_static.py add :
import gconf
and, just before the urllib2 request, line 107, add :
client = gconf.client_get_default()
http_proxy = "http://%s:%s" % (client.get_string("/system/http_proxy/host"),
client.get_int("/system/http_proxy/port"))
https_proxy = "https://%s:%s" % (client.get_string("/system/proxy/secure_host"),
client.get_int("/system/proxy/secure_port"))
proxies = {"http": http_proxy, "https": https_proxy}
urllib2 automatically use the proxies variable. I've tested it and it works for
me
and it should still work for people without proxies.
I'm not sure about the best way to handle this in Specto (should it be global
or per
watch ?) so I let you make a correct patch. You can send me a patch against the
0.2.2
sources if you want me to test it.
Original comment by zepl...@gmail.com
on 7 Apr 2008 at 1:55
hmm, any idea if proxies would need to work with
mail/whatever_accesses_the_Internet
watches too?
Original comment by nekoh...@gmail.com
on 7 Apr 2008 at 2:11
As long as it uses http/https/ftp, yes.
For whatever reason, I've never seen any IMAP/POP3 proxies. Strange, I never
realized.
A simple implementation like this would be a first step and enough to be
released.
Anyway, it should be noted that a real proxy support should also take care of
proxy's
exception (intranet website,etc). I'm not sure that it's automatically
supported by
the code pasted above but it should not be that hard.
Original comment by zepl...@gmail.com
on 7 Apr 2008 at 2:17
Original comment by woutclym...@gmail.com
on 25 Apr 2008 at 8:08
I added this in watch.py but i am not sure if it will work for other watches
than the
web watch, because the call from urllib2 is not done in the watch class but in
another class....so should i make it a global variable so it will be available
in all
classes?
Original comment by woutclym...@gmail.com
on 13 May 2008 at 6:46
Fixed in rev 103.
Please test!
Original comment by woutclym...@gmail.com
on 14 Sep 2008 at 3:32
Please note that this is only in Wout's branch (not yet in main):
https://code.launchpad.net/~woutc/specto/specto-woutc
Original comment by nekoh...@gmail.com
on 14 Sep 2008 at 8:15
I think that this issue should not be marked as "fixed". Unfortunatly, I'm
unable to
test wout's branch for a very simple reason : bzr does not have good proxy
support so
I cannot bzr get it !
Original comment by zepl...@gmail.com
on 15 Oct 2008 at 3:19
Attached you will find my current specto development snapshot :)
unpack it and run ./launch.sh to start it
enjoy
Original comment by woutclym...@gmail.com
on 15 Oct 2008 at 5:53
Attachments:
Thanks for the tarball :-)
Testing it. So far, it seems that the proxy is working well (which is a really
good
news :-) ) with one exception : it doesn't handle exceptions as configured in
Gnome.
Meaning that it will not work with intranet website.
Original comment by zepl...@gmail.com
on 16 Oct 2008 at 8:00
I m happy that it is almost working for you :)
Indeed exceptions are not handled, but i can have a look at it... although the
current implementation is too dynamic to handle it.
Original comment by woutclym...@gmail.com
on 16 Oct 2008 at 8:10
I said "it was working" because it was checking the website with no error. But
now,
it appears that it didn't detect any change. As a test, I'm checking the
slashdot
page since this morning : specto doesn't see any change (which is obviously not
true)...
Sad :-(
Original comment by zepl...@gmail.com
on 16 Oct 2008 at 9:11
ok, this requires some debugging:
could you put this in spectlib/plugins/watch_web_static.py between line nr 122
and
123:
print self.page_source
this prints the source from the webpage to the terminal.
Copy it and send it to me (my username + "@gmail.com")
Sorry but i don't use proxies so i could not test this decently.
Original comment by woutclym...@gmail.com
on 16 Oct 2008 at 9:17
printing the content print the source of the page so all seems to be fine.
I attach the result because there's nothing confidential (only 3 websites :
taskcoach, slashdot, linuxfr.org/journal )
Original comment by zepl...@gmail.com
on 16 Oct 2008 at 9:40
Attachments:
very strange, indeed the correct website is fetched (so proxy support works)
now add after line nr 123
print self.diff
this will show the difference between the cached source and the new source, so
you
can see if something actually changed or not.
It is just an idea, but maybe the page is fetched from the proxies cache? so no
changes are visible?
Original comment by woutclym...@gmail.com
on 16 Oct 2008 at 9:49
It seems that the diff is correct. It output exactly the change between the
current
page and the page like it was last time I launched specto.
Is your tarball working for you for those website ?
Original comment by zepl...@gmail.com
on 16 Oct 2008 at 9:57
I can not test it right now, but this is how it checks if the website was
updated:
f self.cached and self.diff and (self.filesize_difference >=
float(self.error_margin)*100) and (self.filesize_difference != 0.0)
self.cached = not first check
self.diff = is filled (as you could see when you printed it)
self.filesize_difference = difference in size between previous and new (correct
i
guess)
so maybe your error margin is too high?
or try to change on line 171 this: and self.diff
to: and self.diff <> ""
Hope this helps and thank you for testing!
Original comment by woutclym...@gmail.com
on 16 Oct 2008 at 10:04
After some debugging, it appears that the error is in the *100 factor. My
treshold
was at 0.9% which seems to be quite low. But after looking at the number, it
really
looks like this "100" factor is not needed at all.
Putting this factor at "1" seems to have a perfect behaviour.
Original comment by zepl...@gmail.com
on 16 Oct 2008 at 12:31
hmm strange :)
But thank you for finding the bug!
I will also test it when I'm home, but it is possible that the percentage is
already
converted when I rewrote specto.
Original comment by woutclym...@gmail.com
on 16 Oct 2008 at 12:34
mmm, no there's definitly something else because the rss watch suffer of the
same
problem and that, when changing the "100" factor, the popup display a change of
121%.
Original comment by zepl...@gmail.com
on 16 Oct 2008 at 12:47
The rss watch uses the same logic, so if one is broken, also is the other...
Maybe you should look at this line:
self.filesize_difference = (fabs(int(self.new_filesize) -
int(self.old_filesize)) /
int(self.old_filesize))*100
it should give the difference in filesize as a percentage and is also used in
the
balloon.
so print it and see what happens :)
Original comment by woutclym...@gmail.com
on 16 Oct 2008 at 12:54
OK, this feature doesn't work, and if I'm not mistaken on this, then it means
that
it's a major security hole.
Two test cases to prove that proxies don't seem to be taken into account:
1- go in the gnome proxy settings and point to some fake host (in my
screenshot, I
point to "whatever" as a socks proxy). In such a case, if specto really takes
proxies
into account, it should fail with errors like "can't resolve the host" and such
(epiphany does that). In my case, I saw all my mail and website watches refresh
perfectly, thus bypassing my SSH tunnel proxy. This is bad.
2- if one makes an SSH tunnel as a proxy using the following command: "ssh -v
-N -D
8000 the-remote-host", then whenever an application uses this proxy, we should
see
network channels opening in the terminal... In Specto's case, this doesn't
happen (it
does in Pidgin/Epiphany's case).
Am I missing something fundamental here? This basically looks to me like Specto
ignoring my gnome proxy settings, and thus bypassing any security I may have
set up
over untrusted network connections.
Original comment by nekoh...@gmail.com
on 6 Mar 2009 at 8:08
Attachments:
Took a look at the code, it seems we only have http and https proxy support, no
ftp
and Socks proxy support. I don't know if urllib2.ProxyHandler would handle that.
Maybe we could look into python-libproxy too.
But there's still the problem that if I put a fake host as an http or https
proxy,
specto still refreshes as if nothing bad was going on (it /should/ be failing,
afaik). Pretty suspicious. Further investigation welcome.
Original comment by nekoh...@gmail.com
on 7 Mar 2009 at 1:26
Too complex according to Wout, so the full proxy support will have to wait.
Original comment by nekoh...@gmail.com
on 3 Apr 2009 at 4:16
Original comment by nekoh...@gmail.com
on 3 Oct 2009 at 3:02
http://socksipy.sourceforge.net/ is what we are looking for. We only need to
use that
module in Specto's web_proxy.py.
Original comment by nekoh...@gmail.com
on 7 Oct 2009 at 2:29
http://en.opensuse.org/GNOME/Proxy_configuration sheds some light on the mess
that are
gconf keys for proxy support in GNOME.
Original comment by nekoh...@gmail.com
on 7 Oct 2009 at 5:58
I forgot there was the start of an implementation here:
https://code.launchpad.net/~kiddo/specto/socks-proxy
But, for some reason, it breaks (watch plugins don't load).
Original comment by nekoh...@gmail.com
on 13 Nov 2010 at 3:46
Original issue reported on code.google.com by
zepl...@gmail.com
on 7 Apr 2008 at 10:34