Define a directory/file structure on the expo server

goatchurchprime commented 5 years ago

Top level directory on the expo server at /home/expo

These could be reorganized to make accessibility from QGIS easier and clean out distractions no one needs.

Notable directories:

loser -- all survex svx data in a mercurial repository (Not Public). can be accessed by: hg clone ssh://expo@expo.survex.com/loser

tunneldata -- all TunnelX xml files in a mercurial repository. (Not automatically updated, so the repo can be ahead of the files you see here)

expofiles_offline -- some 1m resolution DEM data from Loser area (Not Public)

expoweb -- the static cucc-austria expo.survex.com website

expofiles -- a directory with many different types of data. These directories can be rsynced to your computer, or previewed through a web browser at this page here: http://expo.survex.com/expofiles/

Subdirectories of expofiles/

(These are given in approximate order of workflow.)

surveyscans -- scans of paper drawings of surveys from cave and drawn up, filed by year and wallet

tunneldata -- symlinked to ../tunneldata so it is accessible from the browser. (Not automatically updated, see above for details.)

geotiffsurveys -- rendered cave surveys converted to geotiff format with coordinate frame, which should be directly generated from tunnelx (or other cave drawing programs). These are all bitmaps for now.

surveys -- rendered cave surveys and mini surveys from tunnel (this file will be deprecated once all finished rendering is done using qgis)

qgis_resources -- everything needed to make great presentations, including shape files of survey data, shape files of austria, low-res DEMs, clipped, hill-shaded and contoured lidar sections.

qgis_projects -- place to store any QGIS project files that are created for making integrated GIS and cave maps and surveys. Central point from which all links to other files eg terrain files and geotiffs of surveys are relatively established.

writeups -- logbooks and reports filed by year

presentations -- published articles, reports and powerpoint presentations

photos -- member's underground and overground photos per year, per person

gpslogs -- a few gpx files and waypoints, filed by year

aerial_photos -- 8 photos from 1985

goatchurchprime commented 5 years ago

In particular, establish how we are doing the qgis projects so their links connect to other resources

goatchurchprime commented 5 years ago

rsync cheats sheet

rsync -nav expo@expo.survex.com:expofiles/ /home/expo/expofiles/

rsync -nav expo@expo.survex.com:expofiles/surveyscans/2017/ /home/julian/data/expodata/expofiles/surveyscans/2017/

rsync -nav expo@expo.survex.com:expofiles/geotiffsurveys/ /home/julian/data/expodata/expofiles/geotiffsurveys/

To send back:

rsync -nav /home/julian/data/expodata/expofiles/geotiffsurveys/ expo@expo.survex.com:expofiles/geotiffsurveys/

To send up an indivdual file up, do:

scp filename.tiff expo@expo.survex.com:expofiles/geotiffsurveys

wobrotson commented 5 years ago

rsync cheatsheet: Windows version

first, you will need to install Cygwin. Download the setup.exe file here:

https://www.cygwin.com/

Run the setup .exe file, and when on the install window select 'install from internet'. Leave the installation directory as default. After selecting a site to install from (top one is fine) then you will reach the 'select packages' window. Here, search for rsync, OpenSSH, OpenSSL and Dos2Unix individually and mark every package associated with them for install. It's enough to install the binaries (Bin? ), no need for the source files (Src? ). Proceed to run the installation. If everything went well, you will have packages marked for install in the Debug, Devel, Libs, Net and Utils categories.

Pulling from expofiles Open the terminal shell and navigate to the appropriate directory. Hint: to navigate to a directory in a different drive, use 'cd C://directory/directory' for example.

Then use the command structure as for linux above:

rsync -nav expo@expo.survex.com:expofiles/ /home/expo/expofiles/

rsync -nav expo@expo.survex.com:expofiles/surveyscans/2017/ /home/julian/data/expodata/expofiles/surveyscans/2017/

rsync -nav expo@expo.survex.com:expofiles/geotiffsurveys/ /home/julian/data/expodata/expofiles/geotiffsurveys/

To send back:

rsync -nav /home/julian/data/expodata/expofiles/geotiffsurveys/ expo@expo.survex.com:expofiles/geotiffsurveys/

{ Always a good idea to put in the -n flag whenever publishing rsync command examples - then numpties which copy things without understanding them don't destroy stuff. The -n flag is the "dry run" option. }

PhilipSargent commented 5 years ago

But while I have used Cygwin for probably more than 15 years, I (and I think most people doing this sort of thing) will be using the Linux Subsystem on Windows 10 in future.

Though if you are only going to be using rsync, then Cygwin would of course be much simpler.

See https://docs.microsoft.com/en-us/windows/wsl/about

https://www.linux.com/blog/learn/2018/2/how-use-wsl-linux-pro

Now documented in some detail in the expo handbook: http://expo.survex.com/handbook/computing/winlaptop.html

PhilipSargent commented 5 years ago

the /tunneldata/ folder which was under version control with mercurial has been deleted. All the survey data in it has been copied across to /drawings/ which is now under git.

Various symlinks need to be set up so that other things don't break?

PhilipSargent commented 4 years ago

Filesystem directories now documented in expo handbook https://expo.survex.com/handbook/troggle/serverconfig.html

CaveSurveying / CUCCexposurveyissues