geotoad fails to load Premium caches

[GoogleCodeExporter]

geotoad 3.14.3 under an older Ubuntu release (9.04)

geotoad fails to see Premium caches because (apparently) you can't pass a 
Waypoint ID to /seek/cdpf.aspx anymore -- it needs the GUID. And that doesn't 
seem to be available when it constructs the URL for /seek/cdpf.aspx when 
querying a Premium cache (though it is for a non-Premium one.)

Passing the waypoint ID results in a "Cache is Unpublished" page.

Alas, I don't speak Ruby, so I don't have a suggested fix...

Original issue reported on code.google.com by pdhomew...@gmail.com on 25 Feb 2011 at 12:32

[GoogleCodeExporter]

3.14.3 fails to work at all in my case. 
Using command line -u -p -q location -y distance -x gpx -o out.gpx results in 
NO CACHES FOUND. Using the TUI the window closes w/o any chance to get a 
logfile.

Tried it under XP

Original comment by GC.Womba...@gmail.com on 25 Feb 2011 at 5:30

[GoogleCodeExporter]

@ thread starter:
Both helixblue and myself don't have premium-member status, therefore it is 
hard for us to test code related to PM-only caches.
Your observation is correct, cdpf.aspx at least for PM-only, doesn't allow 
wp=GC... - perhaps that was too easy as a workaround.
What works (and is done by GeoToad) is to determine the GUID of the cache, and 
use that instead. Unfortunately, us mere mortals only see that we don't see any 
details.

If you are PM, you might help us testing - if you're using the source 
distribution (not the Windows executable) and know how to use "patch".

I have collected some thoughts over the past months, and might come up with 
some experimental patches. If you're willing to test, you're welcome!

@ Wombat:
I cannot see how your problem is related to this issue. "The window closes" 
doesn't read like anything PM-only related. Did you start a cmd window, run 
geotoad.exe in it, and the window is closed? Unfortunately I don't have XP (or 
any other MS Windows) to reproduce. I can only suggest to temporarily rename 
your GeoToad directory (probably below "Documents and Settings"? - the one that 
contains the cached pages as well), and re-run.
May I also ask you to check the "ReportingBugs" wiki page? Thanks.

Original comment by Steve8x8 on 26 Feb 2011 at 10:45

[GoogleCodeExporter]

@Steve: I'm a premium member, I was using the Ubuntu package but I'm happy to 
use the source distro to assist in debugging, and I'm plenty comfortable with 
"patch" (just not with Ruby). Send me a diff and I'll be glad to play.

(Incidentally, the Ubuntu package failed to find its libraries for me. I added 
a one-line hack to make it work, but I probably should file a separate bug on 
that) :-) 

Original comment by pdhomew...@gmail.com on 26 Feb 2011 at 11:50

[GoogleCodeExporter]

@PD:
May I ask you to find a random PM-only cache in your vicinity, set the search 
center and shrink the search radius to select only this one, and run "very 
verbose" from the command line? I'd like to compare your output with mine, to 
come up with some patches.
(Don't forget to remove your login data.)
Thanks.
(the one-liner has been fixed, see issue 199)

Original comment by Steve8x8 on 28 Feb 2011 at 9:57

[GoogleCodeExporter]

@Steve: Searching like that works just fine. It appears to only break when you 
search by waypoint ID (which was the only thing I'd tried.)

Still want a trace, or does that simplify the problem sufficiently?

Original comment by pdhomew...@gmail.com on 28 Feb 2011 at 11:15

[GoogleCodeExporter]

As there's obviously something I don't quite understand in GeoToad :D a 
debugging log would be very welcome.
I'd expect GeoToad to run a cache_details.aspx query (without prior login - 
this looks wrong but isn't!), pick up the GUID from there, and access the 
cdpf.aspx?guid=... page after logging in (of course) - just the same as it 
would do with a search. If it doesn't, something is seriously wrong.
I almost never use WID queries... so I never came across any issues related.

Feel free to send the log directly to my gmail account. Please check for the 
two files in your cache directory as well (I suppose they only contain your GC 
name, not the password)

Original comment by Steve8x8 on 1 Mar 2011 at 8:41

• Changed state: InfoNeeded

[GoogleCodeExporter]

Issue 131 has been merged into this issue.

Original comment by Steve8x8 on 8 Mar 2011 at 9:53

[GoogleCodeExporter]

The original poster has explained (in Private Mail) what's the point of -q wid 
queries is, even for Premium Members: to automatically run some script at home, 
preparing the details of a recently published cache, to send to a mobile device.

Makes a lot of sense to me. (Prio goes to Medium.)

@helixblue: Is it feasible to make a (single!) exception to the rule, and fetch 
the cache_details page _with_ valid login credentials (or at least, repeat the 
fetch should the first attempt fail to return coordinates and/or GUID)?

For now, I'm stepping back from ownership: I don't see a simple solution - and 
I'm currently too busy to think about it :(

Original comment by Steve8x8 on 8 Mar 2011 at 10:06

• Added labels: Priority-Medium
• Removed labels: Priority-Low

[GoogleCodeExporter]

Any news about this problem?

Original comment by law.sky...@gmail.com on 10 Jun 2011 at 8:52

[GoogleCodeExporter]

Still thinking, but getting closer. 
To fix this, a major rewrite of several components has to be done.
Since issue 205 can be solved too by early logging into gc.com, please also 
look there.
Please stay tuned for a few more weeks (rough estimate).

Original comment by Steve8x8 on 11 Jun 2011 at 8:03

• Added labels: Milestone-Release3.15

[GoogleCodeExporter]

FYI, I decided the small task I wanted geotoad to do would be fairly simple to 
code in a language I do understand, so I went and did it. :-) (If I spoke Ruby, 
I'd probably have hacked geotoad and sent you the patches, but, well, I don't, 
so I didn't.)

So don't go spending too much time on it on my behalf -- my problem is solved 
-- but there does apparently seem to be reason to fix this (previous comment as 
well as that other issue...)

Original comment by pdhomew...@gmail.com on 11 Jun 2011 at 11:58

[GoogleCodeExporter]

Started working on issues 198 and 205. Status adjusted, but no target date yet.

Original comment by Steve8x8 on 14 Jun 2011 at 7:36

• Changed state: Started

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

P.D., can you by any means get the latest branch revision, or the deb package 
from my homepage, and check whether this resolves the issue?

Original comment by Steve8x8 on 29 Jun 2011 at 12:30

[GoogleCodeExporter]

Will be in 3.16.0

Original comment by Steve8x8 on 2 May 2012 at 1:20

• Changed state: FixAvail

[GoogleCodeExporter]

This issue is believed to be fixed in 3.15, and in 3.16.0 released recently.

Original comment by Steve8x8 on 25 May 2012 at 12:21

• Changed state: Fixed

[GoogleCodeExporter]

Hello, I tried several times the myfindgpx with the latest dev 17.11 version.

I found two things out:
1. if you are asking for myfinds as a different user as the one you want to 
have the finds there is a missstake in the logdate. In the gpx file is a area 
for the log and the date is put to 1980 something.
if you are asking for your finds as the same user it is ok

2. PM onlycaches are still not taken into account even if you are a PM

Original comment by HeinzBro...@gmail.com on 14 Sep 2013 at 3:43

[GoogleCodeExporter]

Hm, as I'm no PM I cannot reproduce your report, I'm afraid. But I have a few 
comments:

1. If you're asking for the found list of someone different you must use the 
"yourfind*" templates. (Yes, there's a difference between "myfind*" and 
"yourfind*". The log dates you got probably were the last found dates?)
"area for the log" - can you show a snippet?

2. Um. Shouldn't happen - how does the search page look like in this case? Can 
you provide a trimmed down screenshot, taken while logged in as PM, with a PMO 
cache? Also, an excerpt from debugging output may help. (You may use the "-L" 
option to limit the number of 20-entry pages processed, if that helps.) I guess 
I have to relax the now rather strict exclusion of PMO caches... should be 
possible, indeed. (Do you use the code from SVN, or the tar.gz version? I might 
want to ask you to test some patches.)

Issue reopened.

Original comment by Steve8x8 on 16 Sep 2013 at 7:11

• Changed state: InfoNeeded

[GoogleCodeExporter]

Heinz, I can offer a patch that should take care of item 2. See the attached 
file.
For me (non-PM!) it seems to work, just returns no coordinates (as there are 
none *visible*). For PMs, the cdpf should look like every other one in this 
respect.
Can you confirm it resolves the issue?

Q: Do cache_details pages (a.k.a. /geocache/GC...) contain a single hint that a 
cache is PMO? (I'd like to see the corresponding HTML...)

Original comment by Steve8x8 on 16 Sep 2013 at 9:41

[GoogleCodeExporter]

... and here comes the patch ...

Original comment by Steve8x8 on 16 Sep 2013 at 9:42

Attachments:

• 198b-membersonly.diff

[GoogleCodeExporter]

And another, somewhat enhanced, version of the patch (which now adds some flags 
to output files for PMO?)
Feedback is appreciated...

Original comment by Steve8x8 on 16 Sep 2013 at 10:32

• Removed labels: Milestone-Release3.15

Attachments:

• 198b-membersonly.diff

[GoogleCodeExporter]

There is a side effect to the change: Non-PMs may be notified about the mere 
existence of a PMO cache, and see D and T values, but _no coordinates_.
Is this a desired behaviour? I guess yes - as GC search pages would also list 
them.

Original comment by Steve8x8 on 16 Sep 2013 at 10:42

[GoogleCodeExporter]

Thinking a bit longer about ex-/inclusion of PMO caches, I now think that 
having PMO caches in GeoToad's output is a good idea for almost everyone in 
almost every situation:

- The majority of GC users seem to be PMs. (I'm not sure about PM percentages 
in the GeoToad user base, but making them angry is not a good idea in the long 
run.)
- Even Basic Members can, will, and do log PMOs. It's possible. (URL 
/seek/log.aspx?id=... with the numerical equivalent of the GC code.) They want 
to have all found caches listed.
- If there should be a request to _ex_clude PMOs from search results, and 
therefore GeoToad output, it should be done using another option --- I'd 
suggest "-O", see the attached patch (to be applied over the previous one).

Comments?

Original comment by Steve8x8 on 16 Sep 2013 at 11:54

Attachments:

• 198c-exclude_pmo.diff

[GoogleCodeExporter]

Heinz, can you confirm that SVN branch resolves the issue? I'd like to merge 
the patch into trunk for the upcoming release...

I'm currently thinking about complementing the "-O" option (to _ex_clude PMO 
caches) with another one to select _only_ PMO caches - we're running out of 
letters, what about "-Q" (-o is already taken, and it's not a good idea to 
change its meaning, but "Q" is close enough to "O" in appearance)? Does it make 
sense at all? (I think so, if only someone PM wants to "give" PMO details to a 
BM friend...)

Original comment by Steve8x8 on 19 Sep 2013 at 6:59

[GoogleCodeExporter]

Hello Steve,
sorry for my late reply but I was on a business trip. I can take care of it 
next week.
Thanks for your help!

Original comment by HeinzBro...@gmail.com on 19 Sep 2013 at 9:02

[GoogleCodeExporter]

What do I have to do with the patch file? 

Original comment by HeinzBro...@gmail.com on 22 Sep 2013 at 7:50

[GoogleCodeExporter]

There are several options:
a. use "patch -p1" in the source tree (if you know what "patch" is) - you'd 
miss some other changes
b. check out the latest SVN trunk which has it merged in (if you know what 
"svn" is) - preferred
c. wait for 3.18.0 - and get all in one package
Obviously "c" doesn't help getting things right before the release (which is 
due tomorrow, btw).

Original comment by Steve8x8 on 23 Sep 2013 at 7:15

• Changed state: FixAvail

[GoogleCodeExporter]

Waiting for feedback from 3.18.0 users (both BM and PM) before closing...

Original comment by Steve8x8 on 24 Sep 2013 at 10:11

[GoogleCodeExporter]

First trial to fetch the Myfind as PM seems to work good ... (still loading)

Original comment by HeinzBro...@gmail.com on 24 Sep 2013 at 7:20

[GoogleCodeExporter]

second trial: request the finds form a different user as BM: same number of 
finds ...

looks great
I will report the result

Original comment by HeinzBro...@gmail.com on 24 Sep 2013 at 7:22

[GoogleCodeExporter]

as BM some failure messages

Original comment by HeinzBro...@gmail.com on 24 Sep 2013 at 7:28

Attachments:

• ScreenShot001.jpg

[GoogleCodeExporter]

This is normal behaviour as BMs have no legal access to coordinate information.
I have already explained this in the forum: 
https://groups.google.com/forum/#!topic/geotoad/tKLiBwfO7Yg - perhaps the 
warning message needs some fine-tuning?
With PMO cache handling rather new in GeoToad, I might have to think about 
another criterion to detect "invalid" cache pages? Not sure whether there's 
anything more suitable.
OTOH, there's a flag already available that tells me about membership status - 
should GeoToad just "ignore" (or silence)  those warnings if BM has been 
detected? For now, just take it as a subtle hint to consider a PMship ;)

That the WID doesn't show up in the warning message - that's a bug, indeed.

BTW, 2500+ caches in one go, that's quite a lot. But the guy from Utah, 
querying all caches of the state (about 12k), still seems to be alive... 

Original comment by Steve8x8 on 25 Sep 2013 at 7:20

• Changed state: Started

[GoogleCodeExporter]

I tried it again as PM and received again a misstake? Is it the same reason as 
you mentioned?

BTW this is the list of myfinds  ... :)

Original comment by HeinzBro...@gmail.com on 25 Sep 2013 at 4:47

Attachments:

• ScreenShot001.jpg

[GoogleCodeExporter]

I suppose you use the same file cache tree for both accounts?
In that case, the cdpf.aspx file hasn't expired yet (and still contains the 
almost empty one delivered to the BM), and won't before 6 days are over...

Never mix BM and PM results.
(BTW, never mix results obtained with different date representations. Slightly 
different story.)
Setting the GEO_DIR environment variable can resolve this issue...

Original comment by Steve8x8 on 25 Sep 2013 at 5:00

[GoogleCodeExporter]

one more thing
I did as already written a myfinds as PM

but I have issues with importing the file into my statistic app.

I found out, that the archived caches do not get the right logging date:
    <groundspeak:date>1980-01-01T08:00:00Z</groundspeak:date>

Original comment by HeinzBro...@gmail.com on 25 Sep 2013 at 5:21

[GoogleCodeExporter]

Yes I did it with the same settings (BM and PM) but it was only for the test.
Normally I only use the BM ...

Original comment by HeinzBro...@gmail.com on 25 Sep 2013 at 5:24

[GoogleCodeExporter]

Re #33: Thinking a bit longer about it, "invalidating" will probably reload the 
cdpf file with correct settings (PM replacing BM), can't test... So no harm 
done (for the next 6 days).
For my own found list, I tend to use the "-Z" option (never ever overwrite 
existing cdpf files). This saves me from D/T changes and archiving side-effects 
- and never let me notice the bug below

Re #35: I can confirm this behaviour, but this doesn't belong here. Please open 
a new issue, this one is too crowded yet. Thanks.

Original comment by Steve8x8 on 26 Sep 2013 at 7:17

[GoogleCodeExporter]

Hello Steve, Hello Heinz,
i run in same problem. After run a query with a BM User the program fail to 
load PM caches with username and PW of a PM user ofcourse. So as you mentioned 
it's problem of the cache directory. 
First i crated another user on my OSX system. Then the query with the PM member 
works fine. But i wanted to do the query for my home area in one system so i 
think it's a solution to separate the cache directory for the geocache users. 
So i did some tests and for me with some modifications it's working fine.

I can get all my caches with two runs of the programm:
1. with BM member and a position and distance
2. with PM member and a file of PM only GC-Codes

Here are the changes:
Tested only with commandline and OSX for other systems like windows it's not 
implemented i think. for that someone has to look to function "findCacheDir".

May this is also an solution that works for all?

Best regards 
Michael

Change this source files:
common.rb: 

…
  def findCacheDir(geouser)
    # find out where we want our cache #############################
    cacheDir=selectDirectory([ENV['GEO_DIR'], "#{ENV['HOME']}/Library/Caches", "#{ENV['USERPROFILE']}/Documents and Settings", ENV['HOME'], ENV['TEMP'],
        "C:/temp/", "C:/windows/temp", "C:/tmp/", "/var/tmp"])

    # probably what we fallback to in most UNIX's.
    if cacheDir == ENV['HOME']
      cacheDir=cacheDir + '/.geotoad/cache'
    elsif cacheDir == "#{ENV['USERPROFILE']}/Documents and Settings"
      cacheDir=cacheDir + "/GeoToad/Cache"
    else
  if geouser != ''
        cacheDir=cacheDir + "/GeoToad/" + geouser
      else
    cacheDir=cacheDir + "/GeoToad/"
  end
      nodebug "#{cacheDir} is being used for cache"
    end

    FileUtils::mkdir_p(cacheDir, :mode => 0700)
    return cacheDir
  end

geotoad.rb
...
  def initialize
    $debugMode    = 0
    output        = Output.new
    $validFormats = output.formatList.sort
    @uin          = Input.new
    #$CACHE_DIR    = findCacheDir('') !!!Deleted!!!
    @configDir    = findConfigDir
    $mapping      = loadMapping()
  end

# This is new and called in the main loop:
  def setCacheDir(geouser)
    $CACHE_DIR    = findCacheDir(geouser)
  end
…

and later in the main loop we determine the cache dir after we know the user:

…
###### MAIN ACTIVITY 
###############################################################
# have some output before initializing the GeoToad, Output, Template classes
include Messages
displayTitleMessage "GeoToad #{$VERSION} (#{RUBY_PLATFORM}-#{RUBY_VERSION})"
displayInfo "Report bugs or suggestions at 
http://code.google.com/p/geotoad/issues/"
displayInfo "Please include verbose output (-v) without passwords in the bug 
report."
cli = GeoToad.new
cli.versionCheck
puts

while true
  options = cli.getoptions
# The actual geo user determins the cachedirectory:  
  if options['user']!=''
    cli.setCacheDir(options['user'])
  else
    cli.setCacheDir('')
  end

…

Original comment by kipp.mic...@googlemail.com on 26 Sep 2013 at 5:37

[GoogleCodeExporter]

Why do you try to work around an already existing solution (set GEO_DIR, done)?
Please try your modifications (diffs are btw preferred over long verbatim 
snippets) with "user" set to the owner of GC183RZ: J!/\/\/\/\`/ and report back.

Original comment by Steve8x8 on 27 Sep 2013 at 7:10

[GoogleCodeExporter]

Hello,
so didn't saw the set GEO_DIR option. 
But my solution also work in dialog mode. (For OSx, linux systems)
And you are right. With the sick name from the other side of the world my 
solution had a problem. 
I crated an own user M!/\/\/\/\`/  PW 'test123456' for testing.
With an small enhancment it is working: 

def setCacheDir(geouser)
    # replace all non word caracters with '_'
    geouser_dir = geouser.gsub(/[\x00\/\\:\*\?\"<>\|`]/, '_')
    $CACHE_DIR    = findCacheDir(geouser_dir)
end

....
( - ) Your cache directory is /Users/mkp/Library/Caches/GeoToad/M!__________
( o ) Logging in as M!/\/\/\/\`/
( o ) Login successful
( o ) Querying user preferences
....

Original comment by kipp.mic...@googlemail.com on 29 Sep 2013 at 9:36

[GoogleCodeExporter]

Hm, don't like :P

It should be simple to set GEO_DIR before invoking the TUI. (For any unixoid OS 
it's "GEO_DIR=/where/ever/you/want/to/have/it geotoad". Windows way is left as 
an exercise to the reader.)

Last time I checked, both Bill Gates and Steve Jobs already had invented 
multi-user capable operating systems, and the easiest way not to get into each 
other's way would be a "Switch user". If you're keen to put the cart before the 
horse, nobody stops you to maintain your own patch set.

Original comment by Steve8x8 on 30 Sep 2013 at 11:33

[GoogleCodeExporter]

Can we keep the "identity change" discussion off this issue?

3.18.1, released recently, includes some changes which are supposed to fix the 
original issue, but there may be some more overlooked side-effects of the 
August 2013 changes.
Status set to "InfoNeeded".

Original comment by Steve8x8 on 21 Oct 2013 at 10:48

• Changed state: InfoNeeded

[GoogleCodeExporter]

A recent commit fixed the (minor) issue that PMO caches would have been dropped 
from WID lists.
With the only exception that PMO caches don't get coordinates assigned (*) 
everything should be OK now (**). Right?

(*) In contrast to the (readily available, and publicly visible) cache name, 
owner, D, T, and size, the coordinates are *invisible* (but present).
(**) If a cache gets PMO status some time after creation, subsequent accesses 
will overwrite the information previously available to basic members. One might 
avoid that - but would the owner like it?

Original comment by Steve8x8 on 5 Dec 2013 at 8:57

• Changed state: FixAvail

[GoogleCodeExporter]

Closing... (3.19.1 release should fix all open issues related)

Original comment by Steve8x8 on 22 Dec 2013 at 12:59

• Changed state: Fixed