Closed cosmicbrucee closed 12 months ago
The example you are using is not working in IDL too. With
a=webget('http://www.mpia.de/index.html')
I received the same output in IDL and GDL (current git)
second example in file webget.pro is not working in IDL or GDL too
just to mention : webget.pro is based on the use of SOCKET. On my laptop, the IDL examples on SOCKET are not working. Alternatively, you can use :
QueryDSS, '00.00+30.00', image, header, survey = '2b'
in the AstroLib.
Salut Alain!
QueryDSS will not help me - The point is that I want to use gdl for HTTP communications - in fact, there are lots of devices that communicate via an HTTP server.
I suppose I could just use a system escape, i.e. spawn, to the system’s HTTPget command, but I dont’ know how to do that, I’ll have to learn….
I will also post this to the IDL group.
Thanks,
-Bruce
On Nov 9, 2018, at 2:53 AM, Alain notifications@github.com wrote:
just to mention : webget.pro is based on the use of SOCKET. On my laptop, the IDL examples on SOCKET are not working. Alternatively, you can use :
QueryDSS, '00.00+30.00', image, header, survey = '2b' in the AstroLib.
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/gnudatalanguage/gdl/issues/513#issuecomment-437324151, or mute the thread https://github.com/notifications/unsubscribe-auth/Aqz1292eIpWMogHjsYcGGlGj3_JItwTTks5utV6MgaJpZM4YWCtS.
GDL> spawn, 'curl http://www.mpia.de/index.html', a
?
Yes, @slayoo but the devil in details : I did not succeed to call the example (provided by WEBGET())
wget 'archive.eso.org/dss/dss/image?ra=0.00000&dec=30.0000&x=10&y=10&Sky-Survey=DSS1&mime-type=download-fits'
from SPAWN in IDL or GDL, but this is working perfectly at (direct bash) command line level
@cosmicbrucee could you provide a template of what you would like to do ?
Dear Sylwester:
I have no problem using curl through spawn in gdl on a linux system:
GDL> spawn,'curl -L www.yahoo.com',result % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 8 100 8 0 0 41 0 --:--:-- --:--:-- --:--:-- 51 100 485k 0 485k 0 0 319k 0 --:--:-- 0:00:01 --:--:-- 406k GDL> help,result RESULT STRING = Array[3897] GDL>
curl prints some stuff to the screen, the “%Total… etc.” which you see above
PRO HINT: yahoo.com http://yahoo.com/ or many other popular sites have re-directs, which will not work with curl, unless you add the -L option, as I did above.
My main question / point here is that when you escape to the system, instead of using an IDL function, you lose portability of code, a big advantage of idl/gdl. For example, how do you do a curl command on a windows machine? I have no idea about using curl on windows machines, and have none to test here now. Unfortunately, various instruments may require me to do so.
-Bruce
On Nov 9, 2018, at 2:33 PM, Sylwester Arabas notifications@github.com wrote:
GDL> spawn, 'curl http://www.mpia.de/index.html', a ?
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/gnudatalanguage/gdl/issues/513#issuecomment-437515970, or mute the thread https://github.com/notifications/unsubscribe-auth/Aqz12-DcYb2QNZSdJg9a3GfPzJHrydsnks5utgKygaJpZM4YWCtS.
Re: your question of: “what I would like to do”: I need to both do a standard “get” of information as well as posting some arguments to the HTTP request.
In my last post to this group, I showed that you could use spawn with linux curl utility to do an http get: IDL>spawn,’curl -L www.yahoo.com http://www.yahoo.com/',result
I also noted that not going with an IDL/gdl command you loose machine independence. Specifically, I have no idea if curl works on a windows machine.
Continuing with the linux machine, however, I am pretty sure I have a solution when you have to do a post operation, i.e. when you have to send arguments over http: I haven’t tested this yet, but I believe this will work fine:
spawn,’curl -d “device=mount&cmd=move&ra2000=11 22 33&dec2000=88 54 32” http://127.0.0.1 http://127.0.0.1:8080/', result
where “device=mount&cmd=move&ra2000=11 22 33&dec2000=88 54 32” is the argument passed in the http request. This should be equivalent to typing into a browser : http://127.0.0.1 http://127.0.0.1:8080//?device=mount&cmd=move&ra2000=11 22 33&dec2000=88 54 32 … i.e. a command of the format: [URL]?[arguments].
The only question for a linux system then, I think, is the quotation marks - I may need to escape the inner double quotes or something.
Again, no idea if this will work in Windows, or if I could install curl on a windows machine.
-B
On Nov 10, 2018, at 1:44 AM, Alain notifications@github.com wrote:
Yes, @slayoo https://github.com/slayoo but the devil in details : I did not succeed to call the example (provided by WEBGET())
wget 'archive.eso.org/dss/dss/image?ra=0.00000&dec=30.0000&x=10&y=10&Sky-Survey=DSS1&mime-type=download-fits' from SPAWN in IDL or GDL, but this is working perfectly at (direct bash) command line level
@cosmicbrucee https://github.com/cosmicbrucee could you provide a template of what you would like to do ?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/gnudatalanguage/gdl/issues/513#issuecomment-437571715, or mute the thread https://github.com/notifications/unsubscribe-auth/Aqz123n4gqO2ytkKpabILRLZAQH3Uasrks5utqAEgaJpZM4YWCtS.
@alaingdl this seems to work (and also perhaps solve the escape/argument passing issue for @cosmicbrucee ):
GDL> spawn, /noshell, ['curl', 'http://archive.eso.org/dss/dss/image?ra=0.00000&dec=30.0000&x=10&y=10&Sky-Survey=DSS1&mime-type=download-fits'], a
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 264k 100 264k 0 0 303k 0 --:--:-- --:--:-- --:--:-- 303k
GDL> help, a
A STRING = Array[748]
This is getting very positive: On the IDL google group
https://groups.google.com/forum/#!topic/idl-pvwave/AsL0qbCZ_Z8
people answered that webget works fine after version 8. Indeed, my machine with version 8 works fine, and has a different version of webget. My IDL machine v7.1.1. does NOT work.
So, what can we do to get this working in gdl?
Let me reproduce my reply here:
Thank you very much to all who answered. Indeed, something was fixed somewhere between 7.1.1 and 8.5.1, where it works fine.
So, what would it take to get it working on my older versions of IDL and um, I hope I can mention this here, gdl. Can I just get newer version of the code?
It would be really helpful to those of us with limited licenses and students and etc. if we could help get this running on gdl, so I would much appreciate anyone with suggestions after looking at the following:
my IDL 8.5.1 machine version says "V5.4 (uses SOCKET)... then the last comment is "Barth, Nov. 2006"
My IDL 7.1.1 machine versions says "V5.4 (uses Socket).... but then the last comment is "Landsman April 2003"
So, I replaced and re-complied the code, and got something that looks like SOCKET is trying to look at only local sockets, or maybe it's port is blocked:
IDL> .run ~/idlpro/gsfc-astro/pro/webget.pro
% Compiled module: MIMETYPE.
% Compiled module: WEBGET.
IDL> aa=webget('yahoo.com')
% SOCKET: Unable to connect to host. Unit: 100
File: localhost.80
Connection refused
% Execution halted at: WEBGET 181
/home/bruce/idlpro/gsfc-astro/pro/webget.pro
% $MAIN$
IDL>
So, maybe SOCKET is the culprit? Maybe if we can fix that, and use the v8+ version of webget, we could get this working.
Any ideas on the problem with SOCKET?
Thanks,
-B
On Nov 11, 2018, at 11:37 PM, Sylwester Arabas notifications@github.com wrote:
@alaingdl this seems to work (and also perhaps sove the escape/argument passing issue for @cosmicbrucee ):
GDL> spawn, /noshell, ['curl', 'http://archive.eso.org/dss/dss/image?ra=0.00000&dec=30.0000&x=10&y=10&Sky-Survey=DSS1&mime-type=download-fits' ], a % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload
Total Spent Left Speed100 264k 100 264k 0 0 303k 0 --:--:-- --:--:-- --:--:-- 303k GDL
help , a A
STRING = Array[748] — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.
UC Berkeley Space Sciences Laboratory Bruce_Grossan@lbl.gov voice- 510.735.8806 fax - please send scans by email
There are very interesting developments on webget on the IDL message board, so I'd like to share there here:
Originally we found that webget() will not get a URL unless you are in IDL v8+
Next, "Lajos Foldy" noted that the code requires a trailing slash in the URL; it turns out that helps a lot....
(Note that redirects can be handled by IDL>spawn,'curl -L [url]',result ) "
...
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
The requested URL /~bruce/ was not found on this server.
works for me. Try a=webget('http://www.google.com/')
not sure this works?
This does not work for me either. At times I have had limited success adding a trailing slash. At this point, I use spawn,'curl -L [url]' for the best results.
B
GDL> result=webget('http://google.com')
% SOCKET: WEBGET: Unable to lookup host. Unit: 103, File:
% Execution halted at: WEBGET 154 /Users/bruce/idlpro/gsfc-astro/pro/webget.pro
% WEBGET 154 /Users/bruce/idlpro/gsfc-astro/pro/webget.pro
% WEBGET 154 /Users/bruce/idlpro/gsfc-astro/pro/webget.pro
% WEBGET 154 /Users/bruce/idlpro/gsfc-astro/pro/webget.pro
% $MAIN$
GDL>
On Mar 19, 2021, at 12:51 PM, Giloo @.***> wrote:
not sure this works?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.
UC Berkeley Space Sciences Laboratory @.*** voice- 510.735.8806 fax - please send scans by email
in fact, works at times.
GDL> a=webget('http://www.google.com/')
% Compiled module: WEBGET.
GDL> a
{ HTTP/1.1 200 OK Date: Sun, 21 Mar 2021 18:37:34 GMT Expires: -1 Cache-Control: private, max-age=0 Content-Type: text/html; charset=ISO-8859-1 (...)
closed, webget works, it's just the other end that does not work, as "http" is now refused in favor or 'https".
Dear gdl users/developers: wget is an astro library function to request an http get (or post).
I cannot get it to work, nor get an informative error message: GDL> a=webget('http://www.yahoo.com') % Compiled module: WEBGET. % SOCKET: WEBGET: Unable to lookup host. Unit: 100, File: % Execution halted at: WEBGET 154 /Users/Bruce/idlpro/gsfc-astro/pro/webget.pro % $MAIN$
GDL>
This has the flavor to me of some port needing to be open, and having nothing to do with gdl itself.
Any help in getting this working would be much appreciated.
Thanks,
-Bruce