Survey scans were re-named which then broke Tunnel - now fixed?

[BeckaLawson]

Various scans from 2018 do not appear to have made it into the surveyscans folder or have been renamed for whatever reason thus breaking the paths that Tunnel uses to find them.

This is really irritating :-)

For example

Missing "surveyscans/2018/2018#36/Scanned Document-3.jpg" "surveyscans/2018/2018#45/scoopyloopyplan.jpg

Renamed "surveyscans/2018/2018#23/CatLitterPlan.jpg" was changed to "surveyscans/2018/2018#23/plan-catlitter.jpg" "surveyscans/2018/2018#46/ntdwyt_plan.jpg" was changed to "surveyscans/2018/2018#46/plan-ntdwyt.jpg"

[PhilipSargent]

Yes, this was me. It was incorrectly named when it was created. But changing the name "to the correct name" breaks things so don't do it!

Handbook: "Name the scanned pages "notes1.jpg, notes2.jpg" etc. This is important as a script detects whether these files exist and if you name them something else it will hassle you unnecessarily." http://expo.survex.com/handbook/survey/newcave.html#onlinew

It is clearly better that people get hassled unnecessarily than that the survey data chain gets broken.

The script wallets.py (originally by Martin Green in 2017) needs to be able to detect whether any "plan" and/or any "elev" scans have been created. So however you label the scanned files, you need to get that word into the filename. If you put that word ("plan" or "elev") at the beginning of the filename the existing script will work. (I am working on bespoke fixes for historical data.). In the very long term all this could become part of troggle itself but we have no manpower to do this.

[PhilipSargent]

Alternatively, a nerd could create a zero-size file "plan-exists-under-other-name.jpg" in any folder where a misnamed scan has been put, e.g. 2018#23/CatLitterPlan.jpg I will do this. It will stop the error messages produced by wallets.py and won't cause any other hassle?

[BeckaLawson]

Add this from 2018 Wookey / Becka email of to do s

On 2018-04-01 07:55 +0000, Lawson, Rebecca wrote:

Hi Wookey,

I've tried to work at my stuff including trips I was on but not doing notes.

Here I've attached the 6 drawings I managed to do (as I can't easily upload them, sorry)s. The file names have the folder numbers they are associated with (after 2017). Let me know if they aren't clear.

OK. Good work! I've uploaded those and updated the status. Well done for putting the references in the filenames so I could work out where to put them :-)

I'll print them out and put them in the folder too.

I then struggled with the remaining drawings mainly as the scanned notes in the folder were either incomplete (probably the person scanning forgot to do the back side of the notes) or were for completely different trips. Specifically:

28 The notes there are mostly for something else and the correct

notes are missing except for notes3 on the left side

Hmm. well those pics are the notes in the physical wallet. So the numbers in the (large-format) notes match: loser/caves/2017_cucc_23/keinwassermelone.svx and the index says #28 is 'Kein Wassermelon', Luke, Becka, Rob. So that all seems to fit. (Plan only in drawings). What makes you think it's wrong?

The small sheet seems to be from 2 days earlier (29th) visit to the same place (has same GPS co-ordinates)

Large notes says '31st' and so does wallet. But file says 30th, and small notes say 29th, and 'we returned +becka next day', which also implies 30th? Do you know which is right? Logbook writeup suggests 30th is right?

29 As for #28, the correct notes are missing and the ones there are

the wrong ones

Those notes are for 2017_cucc_24/gsoh4.svx (and also 2017_cucc_28/blower.svx : just one leg from tag to GPS, and GPS fix)

GSOH and GSH seem to be the same cave. GSH seems to me to be right (Glücklich Schmetterlingehöhle) and GSOH seems to be confusion? gsoh is not used at all in the logbook or index or on the notes. So I've changed that back to GSH now.

So again, this seems to be the same day and cave as the index says it should be. I guess you are looking for the survey on the same day as #28? (logbook writeup http://expo.survex.com/years/2017/logbook.html#t2017-07-30a says GSH and KWM were both surveyed). Right?

So it seems to me that what is in the wallet is right, but there is another GSH survey from the 30th? But I could easily be confused... Why do you think #29 is 'wrong'?

30 Dreadful notes by Luke - no elevation and terrible plan. I didn't

think I could even attempt drawing these up

I agree those are pretty bad. I will be giving some stern advice at the expo training about survey standards. Some of the 2017 stuff simply isn't up to an acceptable standard.

38 Only 2 of the 4 pages scanned - if the original notes can be found

I think they just need the back side of the notes scanning then I can do this

Agreed, backsides missing. I'll sort that tomorrow.

46 I'm not sure this data has been tied in to the main survex file

and these may be the wrong notes

I agree that it doesn't seem to be connected. It does seem to be the notes I expect to be in that wallet.

The Fishface survey is messy. Duplicate legs, passages a hopless mess, at least one disconnected survey. Crazy numbering: B6, 1, B5, X, 2, 3, 4 (ha, that was you :-). Surveying wrapping on themselves (resurveys? errors?)

Could do with a good tidy by someone who knows what's what. I've just added a couple of duplicate flags for now.

[PhilipSargent]

Bother, bother, bother. Updating and reconciling copies of expofiles (and problems with rsync not working with ssh keys on Windows) means that those fixes were overwritten and I will need to re-do those filename fixes. Alternatively, if someone could tell me where tunnel caches those filenames, I could fix them int he tunnel files instead which would be cleaner overall.

Anyone?

[BeckaLawson]

Hi Phil, sorry for the delay, did you get the changes made or are you still waiting for info about the Tunnel cache? If the latter can you nudge Julian who should be able to help

[PhilipSargent]

What changes? I'm not doing any changes.. Just fixing typos in old logbooks and hoping to have a look at the 2019 survey scans folders sometime.. Sorry.

On Sun, Feb 23, 2020, 13:57 Becka notifications@github.com wrote:

Hi Phil, sorry for the delay, did you get the changes made or are you still waiting for info about the Tunnel cache? If the latter can you nudge Julian who should be able to help

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/CaveSurveying/CUCCexposurveyissues/issues/46?email_source=notifications&email_token=ABK5ATPUXCZIHH4VMHWHPODREJ6EZA5CNFSM4ICYLOV2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMV4NDQ#issuecomment-590071438, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABK5ATOZZBSCE6IDGNMOKCLREJ6EZANCNFSM4ICYLOVQ .

[BeckaLawson]

Hi Phil, if you click on the link below it takes you to where you asked for help on github and my email was in reply to that question, cheers Becka

On Sun, 23 Feb 2020, 14:21 Philip Sargent, notifications@github.com wrote:

What changes? I'm not doing any changes.. Just fixing typos in old logbooks and hoping to have a look at the 2019 survey scans folders sometime.. Sorry.

On Sun, Feb 23, 2020, 13:57 Becka notifications@github.com wrote:

Hi Phil, sorry for the delay, did you get the changes made or are you still waiting for info about the Tunnel cache? If the latter can you nudge Julian who should be able to help

— You are receiving this because you commented. Reply to this email directly, view it on GitHub < https://github.com/CaveSurveying/CUCCexposurveyissues/issues/46?email_source=notifications&email_token=ABK5ATPUXCZIHH4VMHWHPODREJ6EZA5CNFSM4ICYLOV2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMV4NDQ#issuecomment-590071438 , or unsubscribe < https://github.com/notifications/unsubscribe-auth/ABK5ATOZZBSCE6IDGNMOKCLREJ6EZANCNFSM4ICYLOVQ

.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/CaveSurveying/CUCCexposurveyissues/issues/46?email_source=notifications&email_token=ACJRALW5BI2NKB2ZC2IMIKDREKA4ZA5CNFSM4ICYLOV2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMV465Y#issuecomment-590073719, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACJRALRIXRTJ4A7CT2OJXITREKA4ZANCNFSM4ICYLOVQ .

[PhilipSargent]

Aha. I had forgotten.

I need a bit more context for these things as there’s a lot on the go at once for me to remember.

I will try to fix it “properly” by editing the tunnel files. But I still haven’t got my head around that part of the system yet.

Troggle isn’t importing all the survey scan stuff for me, and it is not telling me where it is failing..

From: Becka [mailto:notifications@github.com] Sent: 23 February 2020 21:23 To: CaveSurveying/CUCCexposurveyissues Cc: Philip Sargent; Comment Subject: Re: [CaveSurveying/CUCCexposurveyissues] Survey scans were re-named which then broke Tunnel - now fixed? (#46)

Hi Phil, if you click on the link below it takes you to where you asked for help on github and my email was in reply to that question, cheers Becka

On Sun, 23 Feb 2020, 14:21 Philip Sargent, notifications@github.com wrote:

What changes? I'm not doing any changes.. Just fixing typos in old logbooks and hoping to have a look at the 2019 survey scans folders sometime.. Sorry.

On Sun, Feb 23, 2020, 13:57 Becka notifications@github.com wrote:

Hi Phil, sorry for the delay, did you get the changes made or are you still waiting for info about the Tunnel cache? If the latter can you nudge Julian who should be able to help

— You are receiving this because you commented. Reply to this email directly, view it on GitHub < https://github.com/CaveSurveying/CUCCexposurveyissues/issues/46?email_source=notifications&email_token=ABK5ATPUXCZIHH4VMHWHPODREJ6EZA5CNFSM4ICYLOV2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMV4NDQ#issuecomment-590071438 , or unsubscribe < https://github.com/notifications/unsubscribe-auth/ABK5ATOZZBSCE6IDGNMOKCLREJ6EZANCNFSM4ICYLOVQ

.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/CaveSurveying/CUCCexposurveyissues/issues/46?email_source=notifications&email_token=ACJRALW5BI2NKB2ZC2IMIKDREKA4ZA5CNFSM4ICYLOV2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMV465Y#issuecomment-590073719, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACJRALRIXRTJ4A7CT2OJXITREKA4ZANCNFSM4ICYLOVQ .

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/CaveSurveying/CUCCexposurveyissues/issues/46?email_source=notifications&email_token=ABK5ATKYHSATUHJ2SQ64P63RELSKDA5CNFSM4ICYLOV2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMWHXNI#issuecomment-590117813 , or unsubscribe https://github.com/notifications/unsubscribe-auth/ABK5ATNAYCURTT32UBPXMDTRELSKDANCNFSM4ICYLOVQ . https://github.com/notifications/beacon/ABK5ATKUOVASDPZTG5E5ZI3RELSKDA5CNFSM4ICYLOV2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMWHXNI.gif

[BeckaLawson]

I suggest then you forget about this one! The other stuff you have been doing seems important and renaming those files shouldn't be an issue now I know what the cause was ... just please don't rename survey scan files in the future please! Cheers Becka

On Sun, 23 Feb 2020, 22:00 Philip Sargent, notifications@github.com wrote:

Aha. I had forgotten.

I need a bit more context for these things as there’s a lot on the go at once for me to remember.

I will try to fix it “properly” by editing the tunnel files. But I still haven’t got my head around that part of the system yet.

Troggle isn’t importing all the survey scan stuff for me, and it is not telling me where it is failing..

From: Becka [mailto:notifications@github.com] Sent: 23 February 2020 21:23 To: CaveSurveying/CUCCexposurveyissues Cc: Philip Sargent; Comment Subject: Re: [CaveSurveying/CUCCexposurveyissues] Survey scans were re-named which then broke Tunnel - now fixed? (#46)

Hi Phil, if you click on the link below it takes you to where you asked for help on github and my email was in reply to that question, cheers Becka

On Sun, 23 Feb 2020, 14:21 Philip Sargent, notifications@github.com wrote:

What changes? I'm not doing any changes.. Just fixing typos in old logbooks and hoping to have a look at the 2019 survey scans folders sometime.. Sorry.

On Sun, Feb 23, 2020, 13:57 Becka notifications@github.com wrote:

Hi Phil, sorry for the delay, did you get the changes made or are you still waiting for info about the Tunnel cache? If the latter can you nudge Julian who should be able to help

— You are receiving this because you commented. Reply to this email directly, view it on GitHub <

https://github.com/CaveSurveying/CUCCexposurveyissues/issues/46?email_source=notifications&email_token=ABK5ATPUXCZIHH4VMHWHPODREJ6EZA5CNFSM4ICYLOV2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMV4NDQ#issuecomment-590071438

, or unsubscribe <

https://github.com/notifications/unsubscribe-auth/ABK5ATOZZBSCE6IDGNMOKCLREJ6EZANCNFSM4ICYLOVQ

.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub < https://github.com/CaveSurveying/CUCCexposurveyissues/issues/46?email_source=notifications&email_token=ACJRALW5BI2NKB2ZC2IMIKDREKA4ZA5CNFSM4ICYLOV2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMV465Y#issuecomment-590073719 , or unsubscribe < https://github.com/notifications/unsubscribe-auth/ACJRALRIXRTJ4A7CT2OJXITREKA4ZANCNFSM4ICYLOVQ

.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub < https://github.com/CaveSurveying/CUCCexposurveyissues/issues/46?email_source=notifications&email_token=ABK5ATKYHSATUHJ2SQ64P63RELSKDA5CNFSM4ICYLOV2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMWHXNI#issuecomment-590117813> , or unsubscribe < https://github.com/notifications/unsubscribe-auth/ABK5ATNAYCURTT32UBPXMDTRELSKDANCNFSM4ICYLOVQ> . < https://github.com/notifications/beacon/ABK5ATKUOVASDPZTG5E5ZI3RELSKDA5CNFSM4ICYLOV2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMWHXNI.gif>

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/CaveSurveying/CUCCexposurveyissues/issues/46?email_source=notifications&email_token=ACJRALQVIE34CBIZRGVFS43RELWZDA5CNFSM4ICYLOV2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMWITGA#issuecomment-590121368, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACJRALQ6D2U3ZJBYN5LYW6TRELWZDANCNFSM4ICYLOVQ .

[PhilipSargent]

Re: Renamed and uploaded to expo.survex.com

"surveyscans/2018/2018#23/CatLitterPlan.jpg" was changed to "surveyscans/2018/2018#23/plan-catlitter.jpg" "surveyscans/2018/2018#46/ntdwyt_plan.jpg" was changed to "surveyscans/2018/2018#46/plan-ntdwyt.jpg"

I see that someone (maybe me) had already renamed it back to CatLitterPlan.jpg I have inserted a dummy JPG file that wallets.py will recognise instead: plan-catlitter-dummy-file.jpg

I have just this second renamed /plan-ntdwyt.jpg back to /ntdwyt-plan.jpg too.

A change to the wallets.py code means that plans are identified if they either start or end with the word "plan" in the name now, instead of just at the beginning. ditto for "elev"

[PhilipSargent]

Re: Renamed and uploaded to expo.survex.com Becka wrote (on GitHub): Missing "surveyscans/2018/2018#36/Scanned Document-3.jpg" "surveyscans/2018/2018#45/scoopyloopyplan.jpg The "surveyscans/2018/2018#36/Scanned Document-3.jpg" is actually there, just renamed as plan1.jpg I have named it back again and inserted a dummy file to keep wallets.py happy. Similarly "surveyscans/2018/2018#45/scoopyloopyplan.jpg is actually there, but renamed as plan-snoopyloopy1.jpg I have named it back again and inserted a dummy file to keep wallets.py happy. I tried to find where the tunnel files are that record the paths to these files so that I could correct them in the tunnel files, but I failed. I can’t find the relevant tunnel files only entries in the troggle database itself. The plans are scanned black and white printouts. I have resampled them to take up less filespace without affecting the visual quality. I have not touched the original survey notes scans.

Philip

[PhilipSargent]

Tunnel files on the server in ::drawings:: repo now edited to reflect correct paths to scanned surveys on the server. in expofiles/surveyscans/...

This issue can be closed.