.3d files;.3d file (*);() with the appropriate `csand*cs outcommands in the.svx`
source files (see below).
The current version (v1.3.1) should be available through the QGIS Python Plugins
Repository: launch QGIS3,
go to 'Plugins → Manage and Install Plugins...', then (under the
'All' tab) enter 'survex' in the search filter to find the 'Import .3d
file' plugin, select it and click 'Install Plugin'.
When installed, a menu item 'Import .3d file' should appear on the 'Vector' drop-down menu in the main QGIS3 window, and (if enabled) a .3d icon in a toolbar.
Manual installation is possible if the QGIS Python Plugins Repository route is not available:
survex_import to python/plugins/ in the current active
profile, the location of which can be found from within QGIS3 by
going to 'Settings → User Profiles → Open Active Profile
Folder' (*);(*) Alternatively if you have pb_tool you can run pb_tool deploy from within the survex_import directory.
Debian users may be able to install from a packaged version:
https://packages.debian.org/sid/qgis3-survex-import
Selecting 'Import .3d file' (or clicking on the .3d icon) brings up a
window for the user to select a .3d file with a number of options:
.3d file or inherit from the QGIS3 project;(*) In rare cases a station may be flagged both surface and underground, in which case it is imported even if the 'surface' option is left unchecked.
On clicking OK, vector layers are created to contain the imported features as desired. Legs, walls, cross sections, and traverses are imported as line strings in separate vector layers for convenience. All created layers are saved to the GeoPackage file if requested (any existing content is overwritten).
A CRS selector dialog box will appear, if neither of the CRS selector options
are checked, or if CRS from .3d file is selected but there is no CRS in the .3d
file.
If 'keep features' is selected, then previously imported features are
not discarded, and the newly-created layers will contain both the
previously imported features plus any new features imported from the
designated .3d file. This choice allows processed survey data sets to
be combined from multiple sources. Note that cumulative imports do
not result in features being overwritten, even if they happen to share
the same name, since all features are assigned a unique ID.
An earlier version of the plugin introduced a mismatch between the CRS
and a proj4 string in the .3d file. This bug is fixed since v1.2
and import of .3d files generated by Therion should now work.
All layers are created with an ELEVATION attribute, for convenience. For stations this is the just the z dimension. For all other features it is the mean elevation.
For station and leg layers, the following additional attribute fields that are created:
stations: NAME, and flags SURFACE, UNDERGROUND, ENTRANCE, EXPORTED, FIXED, ANON
legs: NAME, STYLE, DATE1, DATE2, NLEGS (*), LENGTH (*), ERROR (*), ERROR_HORIZ (*), ERROR_VERT (*), and flags SURFACE, DUPLICATE, SPLAY
(*) These fields correspond to the error data reported in the .3d
file, which is only generated (by survex) if loop closures are present.
The flags are integer fields set to 0 or 1.
The STYLE field for legs is one of NORMAL, DIVING, CARTESIAN, CYLPOLAR, or NOSURVEY.
The DATE fields are either the same, or represent a date range, in the standard QGIS3 format YYYY-MM-DD.
If up / down data for passage polygons is requested, then the polygons have MEAN_UP and MEAN_DOWN attributes in addition to ELEVATION. These are computed from the LRUD data for the two stations at either end of the leg. They can be used in 3d work (see end).
Passage walls (as line strings), polygons, and cross sections (as
lines) are computed from the left and right measurements in the LRUD
data in the same way that the aven viewer in survex displays passage
'tubes' (well, near enough...). The direction of travel (bearing) is
worked out, and used to compute the positions of points on the left
and right hand passage walls. These wall points are then assembled
into the desired features (walls, polygons, cross sections).
The direction of travel is inferred from the directions of the two legs on either side of the given station (with special treatment for stations at the start and end of a traverse). In averaging these, either the legs can be weighted equally (except true plumbs which break the sequence), or the option is given to weight legs by the cosine of the inclination (computed from the processed data, not the actual clino reading). The former is the default, and the latter corresponds to checking the 'use clino weights' box in the import dialog. This alternative option downplays the significance of the occasional steeply inclined leg in an otherwise horizontal passage.
One might want to do this for the following reason. In the 'good old days' steeply inclined legs were usually avoided as they are difficult to sight a compass along, and instead good practice was to keep legs mostly horizontal and add in the occasional plumbed leg when dealing with inclined passages. Also pitches were nearly always plumbed. This meant that inferring passage direction as a simple average, ignoring plumbed legs, was most likely correct. For modern surveying with digital instruments, this is no longer the case: there is no loss of accuracy for steeply inclined legs, and shining a laser down a pitch at an off-vertical angle is no problem. Therefore, the 'use clino weights' option has been invented to give such steeply included legs less weight when inferring the passage direction. Note that in a steeply inclined passage, all legs are likely roughly equally inclined, and therefore roughly equally weighted, so using clino weights shouldn't affect the inferred direction of travel in that situation.
TL;DR: if in doubt try first with the 'use clino weights' option selected.
Note that passage wall data is inferred and any resemblance to reality may be pure coincidence: if in doubt, use splays!
To be able to set the CRS from the .3d file on import, it's
necessary to add the appropriate *cs and *cs out commands to the
survex file. If you can, specify the output CRS using *cs out in
the top level .svx file with an
EPSG
number. Some recommended options here are:
*cs out EPSG:7405 =
OSGB36 (British
National Grid);*cs out EPSG:27700 = basically the same, OSGB36 (see below);*cs out EPSG:25830 = ETRS89 UTM zone
30N
(Europe, −6°W to 0°W, and ETRS89 by country);*cs out EPSG:32630 = WGS84 UTM zone
30N
(World, N hemisphere, −6°W to 0°W, by country);*cs out UTM30N = exactly the same WGS84 UTM zone
30N,
but as a convenient mnemonic.All these can of course equally well be used as *cs commands for
specifying fixed points such as entrances.
For the first, EPSG:7405 officially includes ODN (Ordnance Datum Newlyn) heights but in practice it doesn't make any difference to the output that survex produces. The OS online maps though use EPSG:27700.
The second and third options are UTM co-ordinate systems that both cover most of western Europe: EPSG:25830 uses the ETRS89 datum which is what is officially recommended, even by the UK government, and EPSG:32630 uses the WGS84 datum which is what corresponds to GPS co-ordinates in UTM format. The difference between WGS84 and ETRS89 though is minimal: the current (2023) datum shift is less than 1m (≈ 2.5 cm per year since 1989) so for almost all practical purposes where entrance locations are limited by the accuracy of hand-held GPS devices, there is not really a distinction between WGS84 UTM co-ordinates and ETRS89 UTM co-ordinates. But if you wanted to be finicky and use GPS entrance co-ordinates but have the output in a more respectful European-centric CRS, you might have something like
*cs out EPSG:25830 ; when in Europe, output in ETRS89 UTM zone 30N
*cs EPSG:32630 ; or *cs UTM30N ; GPS entrance co-ordinates are WGS84 UTM zone 30N,UTM stands for Universal Transverse Mercator by the way, and is a way of systematically metricising latitude and longitude for a given geodetic datum.
For the time being AVOID:
*cs out EPSG:3042 (ostensibly also ETRS89 UTM zone 30N)(and other CRS in the same sequence) since this messes up the grid convergence (see below).
More details on the *cs and *cs out commands can be found in the
survex manual.
In-depth explanations of co-ordinate reference systems in general can be found in the Ordnance Survey booklet entitled A Guide to Coordinate Systems in Great Britain which can be found on the Ordnance Survey website.
For the UK, survex provides a convenient shorthand for Ordnance
Survey (OS) grid
letter
system. For example DowProv.svx contains the following:
*cs out EPSG:7405 ; output is 12-figure OSGB36 (+ ODN height)
*cs OSGB:SD ; input co-ordinates are 10-figure grid refs for the SD squareThis specifies that the fixed points (e.g. entrances) and the location for the automatically calculated magnetic declination (see below) are 10-figure grid references in the OS 100km x 100km SD grid square, and that the output should be delivered using the all-numeric 12-figure British National Grid.
Cave entrance locations in the French (IGN) system can be a bit mysterious. Mainland France and Corsica is covered by Lambert zone III which is EPSG:27563, however the way co-ordinates are commonly written in this CRS requires some deciphering.
For example the Lambert zone III co-ordinates for the Grotte
Roche in the Bourne Gorge
(Vercors) are given as X: 848,86 Y: 3312,72 Z: 740. These are in km,
with a comma as the decimal point. To convert these to something
useful first change them to metres and then drop the initial '3'
from the Y co-ordinate (which presumably indicates III). This gets
848860, 312720, 740. These are now properly
EPSG:27563
co-ordinates which can readily be converted to a GPS location, and so
on. The whole thing can be done by 'abusing' survex's conversion
capabilities with a three-line .svx file
*cs out UTM31N ; for GPS here we want WGS84 UTM zone 31N
*cs EPSG:27563 ; the French Lambert zone III CRS
*fix entrance 848860 312720 740 ; from the given X: 848,86 Y: 3312,72 Z: 740Running this through survex and examining the .3d file with
dump3d, one finds the line
NODE 696589.86 4993921.60 740.00 [entrance] FIXEDThese are indeed the co-ordinates in WGS84 UTM zone 31N, or close enough: Speleo Vercors gives GPS: 31N 696593E 4993925N, but note that the Lambert III co-ordinates are only given to an implied accuracy of 10m (two decimal places in km)
In some cases it may be helpful to create beforehand a user-defined
CRS to select in the import dialog. For example, if the *cs
commands are omitted from DowProv.svx, the resulting .3d file lacks
CS metadata and all co-ordinates are relative to the OS SD grid
square. This .3d file can nevertheless still be imported into QGIS3
by first creating a custom CRS (in QGIS) for the SD grid square, then specifying
this custom CRS in the import dialog (or inheriting from the project
CRS if that is set appropriately).
For the OS SD grid square, the requisite custom CRS can be created from the following (long!) proj.4 string
+proj=tmerc +lat_0=49 +lon_0=-2 +k=0.9996012717 +x_0=100000 +y_0=-100000 +ellps=airy +datum=OSGB36 +units=m +no_defsThis is identical to the proj.4 string for EPSG:27700
(British National Grid) except that the +x_0 and +y_0 entries have
been shifted to create a new false origin for the SD grid square.
Another example is the Austrian Loser plateau data that accompanies the
survex distribution as sample data. Many of the cave entrances are
recorded using a truncated form of the MGI / Gauss-Krüger (GK) Central Austria
CRS (the non-truncated form is
EPSG:31255).
This truncated CRS corresponds to another (equally long!) proj.4 string
(see also the entry under *cs in the survex data file
documentation)
+proj=tmerc +lat_0=0 +lon_0=13d20 +k=1 +x_0=0 +y_0=-5200000 +ellps=bessel +towgs84=577.326,90.129,463.919,5.137,1.474,5.297,2.4232 +units=m +no_defsThis is derived from the proj.4 string for
EPSG:31255
by changing the +y_0 entry. The 13d20 in here means 13°20' (see
documentation),
and you will sometimes see this written as a decimal 13.3333333333333,
for instance in the proj.4 specification for
EPSG:31255
The CRS defined by *cs out gets written as metadata into the .3d
file. For example by using dump3d to inspect DowProv.3d one finds
the line
CS EPSG:7405The QGIS plugin uses this metadata in the .3d file to identify the
CRS:
If the .3d file does not contain CS metadata then the input filter
fall backs onto a CRS selector dialog.
Note that currently some options permissible by survex, such as specifying the CRS by an ESRI number, are not handled here. For the time being, the workaround is to identify what co-ordinate system these are in QGIS language, then use the CRS selector dialog on loading to set the layer(s) CRS appropriately, if necessary creating a custom CRS beforehand, as described next.
This is potentially a large subject, and one can again refer to the
survex documentation for more details. The modern approach, which
fits with the recommended use pattern for the .3d file importer, is
to use a *declination auto command with a position representative of
the cave survey data, and appropriate *date commands to reflect the
dates at which the compass bearings were taken. There are a few
'gotchas' though:
*declination auto should come after the *cs and *cs out
commands in order that survex knows which input CRS is being used to
define the location used for the magnetic declination calculation, and
which output CRS should be used to calculate the grid convergence
(difference between grid north and true north): if *declination auto
comes before *cs out, survex doesn't currently complain but it
sets the grid convergence to zero;
likewise avoid having a *declination <specified> in the top level
as this will have the side effect of setting the grid convergence for
the whole survey to zero: if you do need to mix *declination auto
and *declination <specified> commands, limit the scope of the latter
by bracketing them in *begin and *end statements;
for certain *cs out choices the 'interchange data' order is the
'wrong way around', i.e. (northing, easting) rather than the more
usual (easting, northing). An example is
EPSG:3042
(see above), which is ostensibly the same as
EPSG:25830 and indeed
you cannot tell the difference either from the proj.4 string or the
so-called 'Well Known
Text':
somehow one has to
know
these things, which seem even to be implementation
dependent!
Currently if you use *cs out with one of these co-ordinate systems
in combination with *declination auto, it has the unfortunate effect
of throwing the grid convergence calculation out by 90°, thus
completely screwing up the calculations (however, it is pretty obvious
that something extremely weird has happened). Such CRS choices with
the interchange data order the 'wrong way around' should therefore be
avoided but to my knowledge there is always an equivalent CRS with the
interchange data order the right way around which can be used instead.
For instance, for ETRS89 UTM 30N do NOT use *cs out EPSG:3042,
rather use *cs out EPSG:25830 which is the exact same co-ordinate
system but with the interchange data order the right way around.
Putting this all together, you need something like the following:
*cs out <output CRS>
*cs <declination auto CRS>
*declination auto <somewhere handy> ; specify the location to use for this
...
*date <date> ; declination at this date is automatically applied
...
*date <different date> ; declination automatically applied with different date
...
*begin <subsurvey>
*declination <specified> ; if you need to explicitly specify such...
*end <subsurvey>Usually, this would be spread across separate *include files, and
indeed one should normally split out each individual survey trip into
its own .svx file, with its own *begin and *end. Further *cs
commands can be sprinkled through these files if there is a mixture of
input co-ordinate systems. As is pointed out in the survex
documentation, it is the first *cs out that sets the output CRS,
all subsequent ones are silently ignored. This means it is relatively
easy to change the output CRS by 'wrapping' the top level survex file as
*cs out <new output CRS>
*include <original top level .svx file>These considerations summarise some recent exchanges on the survex mailing list. Further discussion of 'best practice' for organising cave survey data is probably best left to another place though!
Once the data is in QGIS3 one can do various things with it.
For example, features (stations, legs, polygons) can be colored
by elevation to mimic the behaviour of the aven viewer in survex
(hat tip Julian Todd for figuring some of this out). The easiest way
to do this is to use the .qml style files provided in this
repository. For example to color legs by depth, open the properties
dialog and under the 'Style' tab, at the bottom select 'Style →
Load Style', then choose one of the color_lines_by_elevation*.qml
style files. This will apply a color scheme to the ELEVATION field
data with an inverted spectral color ramp. Use lines for legs,
walls, cross sections and traverses; points for stations; and
polygons for polygons.
Two versions of these style files are provided.
The first version uses a graduated, inverted spectral color ramp to color ranges of ELEVATION. A small limitation is that these ranges are not automatically updated to match the vertical range of the current data set, but these can be refreshed by clicking on 'Classify' (then 'Apply' to see the changes).
The second version uses a simple marker (line, or fill) with the color set by an expression that maps the ELEVATION to a spectral color ramp. There are no ranges here, but rather these styles rely on zmin and zmax variables being set (see 'Variables' tab under layer → Properties). By matching zmin and zmax between layers with these styles, one can be assured that a common coloring scheme is being applied. A handy way to choose values for zmin and zmax is to open the statistics panel (View → Panels → Statistics Panel) to check out the min and max values in the ELEVATION field.
Color legs by date is possible using an expression like
day(age("DATE1",'1970-01-01')) (which gives the number of days
between the recorded DATE1 and the given date). Color legs by error
is also possible.
Another thing one can do is enable 'map tips', for example to use the NAME field. Then, hovering the mouse near a station (or leg) will show the name as a pop-up label. For this to work:
With a digital elevation model (DEM raster layer) even more interesting things can be done. For example one can use raster interpolation to find the surface elevation at all the imported stations and save for example to a SURFACE_ELEV field. Then, one can use the field calculator to make a DEPTH field containing the depth below surface, as SURFACE_ELEV minus ELEVATION. Stations can be colored by this, or the information can be added to the 'map tip', etc.
Three dimensional views can be made directly in QGIS3 with 3D Map View though more conveniently with the Qgis2threejs plugin, usually in combination with a DEM. To render features in 3d use the z co-ordinate for points and lines. Passage 'tubes' like those in aven can be approximately rendered using LRUD polygons, with the base set to floor level and the extruded height set to roof level. To do this import the MEAN_UP and MEAN_DOWN fields mentioned above and use the field calculator to make two new floating point (double) fields: FLOOR equal to ELEVATION minus MEAN_DOWN, and HEIGHT equal to MEAN_DOWN plus MEAN_UP. Then render the polygons with the z co-ordinate as the absolute FLOOR, and extruded height as HEIGHT.
Note that there has been a bug in the Qgis2threejs plugin for QGIS3 that causes a python error when features have data defined properties, such as color by elevation using zmin and zmax variables (second option above). The error looks like
AttributeError:'QgsSimpleLineSymbolLayer' object has no attribute 'dataDefinedProperty'(The problem doesn't arise if features are colored by ranges as in the first option above.)
A workaround is as follows. First add zmin and zmax variables into the layer properties (bring up the Properties window and go to the Variables tab): use the green '+' button to add two new variables then click on Apply and OK. Choose values suited to the data set of interest (as above), for example for the DowProv case they can be set to 320 and 400 respectively (elevation in metres ODN). Second, make sure in the main QGIS map window the features in the layer of interest (eg legs) use only a simple style with a fixed colour (this is the default). Third, in the Qgis2threejs Exporter window, double click on the layer of interest (eg legs) to bring up the layer properties, and in the Style panel select Color → Expression. Paste the following into the Expression box.
ramp_color('Spectral',scale_linear("ELEVATION",@zmin,@zmax,1,0))If all is well the lines in the Qgis2threejs Exporter preview window should change to be colored by elevation.
Note that if you encountered the python error the plugin may not function correctly any more. It may have to reloaded (which can be done if you have installed the 'Plugin Reloader' plugin); or QGIS3 restarted.
A real-life example data set which implements the above protocols for
the Dow Cave - Providence Pot system (including Dowbergill Passage) is
contained in the DowProv directory. A precompiled .3d file
georeferenced to EPSG:7405
can be found there as DowProv.3d.
v1.3.1 (current) - fix to handle how survex embeds EPSG numbers in the .3d file.
v1.3 - upload to QGIS3 plugin repository
v1.2 - fixed CRS import methods
v1.1 - minor updates, tagged for packaging
v1.0 - migrated and updated from QGIS 2.18 plugin
There is a QGIS2 version of this plugin but it is no longer being maintained.
Code in this repository is licensed under GPL v2:
This program is free software: you can redistribute it and / or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 2 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with this program. If not, see http://www.gnu.org/licenses/.
Copyright © (2018-2023) Patrick B Warren.
The .3d file parser is based on a GPL v2 library to handle Survex 3D
files (*.3d), copyright © 2008-2012 Thomas Holder,
http://sf.net/users/speleo3/; see
https://github.com/speleo3/inkscape-speleo.