Closed jmaeding closed 5 years ago
zsiki
commented
5 years ago Dear James,
thanks for your feedback. I'm ready to improve my plug-in. Can you send me a sample dataset to reproduce the error message? What type of geodatabase do you use (ESRI, PostGIS, ...)?
Best regards, Zoltan
jmaeding
commented
5 years ago Hi There, thanks for replying. The attached zip has a polygon layer that causes it. Please do not forward that data to anyone though, its project specific and I am not allowed to distribute. Anyway, so I looked up another plugin that does not batch, but does not catch on the types. Its from the MMQGIS plugin. I started to try to steal their code for the csv write, but am a c# person, not python so struggling through it. In the code from attached .txt (was.py), it loops through the features and does all the things to output csv. I love how all these plugins are not compiled, so easy to learn from. See what you come up with. thx
James Maeding Project Engineer Direct: 949-458-5448 Cell: 949-279-1894 Email: Jmaeding@hunsaker.commailto:Jmaeding@hunsaker.com
3 Hughes Irvine, Ca 92618 Main: 949-583-1010 Website: www.hunsaker.comhttp://www.hunsaker.com/
From: Zoltan Siki [mailto:notifications@github.com] Sent: Monday, September 9, 2019 1:12 PM To: zsiki/bulkvectorexport bulkvectorexport@noreply.github.com Cc: James Maeding JMaeding@hunsaker.com; Author author@noreply.github.com Subject: Re: [zsiki/bulkvectorexport] Multiple Feature geometry not imported (OGR error: Unsupported WKB type 1167) errors (#5)
Dear James,
thanks for your feedback. I'm ready to improve my plug-in. Can you send me a sample dataset to reproduce the error message? What type of geodatabase do you use (ESRI, PostGIS, ...)?
Best regards, Zoltan
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/zsiki/bulkvectorexport/issues/5?email_source=notifications&email_token=AA5Y44JOHFGSHMEBHXYI4UDQI2UYNA5CNFSM4IU6S3FKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6I4AMA#issuecomment-529645616, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AA5Y44LWGTZ2TMAQZ3LHXGDQI2UYNANCNFSM4IU6S3FA.
#
#
import io import re import csv import sys import time import json import math import locale import random import os.path import operator import tempfile import urllib.parse import urllib.request import xml.etree.ElementTree
from qgis.core import from PyQt5.QtCore import from PyQt5.QtGui import *
from math import *
def mmqgis_animate_lines(print_layout, input_layer, fixed_speed, frame_count, frame_directory, status_callback = None):
# Error Checks
if not print_layout:
return "Invalid print layout"
exporter = QgsLayoutExporter(print_layout)
if not exporter:
return "No exporter found for print layout " + print_layout.name()
if not input_layer:
return "Invalid map layer"
if (input_layer.wkbType() in [QgsWkbTypes.LineString25D, QgsWkbTypes.MultiLineString25D]):
return "2.5-D not supported. Modify -> Convert Geometry Type to line to animate"
if (input_layer.type() != QgsMapLayer.VectorLayer) or\
(input_layer.wkbType() not in [QgsWkbTypes.LineString, QgsWkbTypes.LineString25D, \
QgsWkbTypes.MultiLineString, QgsWkbTypes.MultiLineString25D]):
return "Input needs to be a line layer"
if frame_count <= 0:
return "Invalid number of frames specified: " + str(frame_count)
if not os.path.isdir(frame_directory):
return "Invalid output directory: " + str(frame_directory)
# Convert features to lists of points
points = []
length = []
for feature_index, feature in enumerate(input_layer.getFeatures()):
# This doesn't actually work for 2.5D geometries = waiting for API to support them 2/22/2017
fpoints = []
if (feature.geometry().wkbType() == QgsWkbTypes.LineString) or \
(feature.geometry().wkbType() == QgsWkbTypes.LineString25D):
fpoints = feature.geometry().asPolyline()
elif (feature.geometry().wkbType() == QgsWkbTypes.MultiLineString) or \
(feature.geometry().wkbType() == QgsWkbTypes.MultiLineString25D):
for line in feature.geometry().asMultiPolyline():
fpoints.extend(line)
else:
return "Invalid geometry type " + str(feature.geometry().wkbType())
points.append(fpoints)
#print str(feature_index) + " = " + str(len(points)) + " = " + str(len(fpoints)) + \
# " = " + str(feature.geometry().wkbType()) + " = " + str(type(points))
# Calculate total shape length
# Can't use length() function because it does not consider circuity
flength = 0
for point in range(0,len(fpoints) - 1):
flength = flength + \
sqrt(pow(fpoints[point].x() - fpoints[point + 1].x(), 2) + \
pow(fpoints[point].y() - fpoints[point + 1].y(), 2))
length.append(flength)
if len(length) <= 0:
return "No features in layer to animate"
max_length = max(length)
# Iterate Frames
for frame in range(frame_count + 1):
if status_callback:
if status_callback(100 * frame / frame_count, "Rendering frame " + str(frame)):
return "Animate lines cancelled on frame " + str(frame)
input_layer.startEditing()
for feature_index, feature in enumerate(input_layer.getFeatures()):
fpoints = points[feature_index]
if (len(fpoints) <= 0):
continue
if fixed_speed:
visible_length = min([length[feature_index], max_length * frame / frame_count])
else:
visible_length = length[feature_index] * frame / frame_count
total_length = 0
visible = [fpoints[0], fpoints[0]]
for z in range(1, len(fpoints)):
segment_length = pow(pow(fpoints[z].x() - fpoints[z - 1].x(), 2) + \
pow(fpoints[z].y() - fpoints[z - 1].y(), 2), 0.5)
# print " " + str(total_length) + " + " + str(segment_length)
if (total_length >= visible_length):
break
elif (total_length + segment_length) <= visible_length:
visible.append(fpoints[z])
total_length = total_length + segment_length
else: # only display part of line segment
fraction = (visible_length - total_length) / segment_length
x = fpoints[z - 1].x() + ((fpoints[z].x() - fpoints[z - 1].x()) * fraction)
y = fpoints[z - 1].y() + ((fpoints[z].y() - fpoints[z - 1].y()) * fraction)
visible.append(QgsPointXY(x, y))
break
# print str(visible_length) + ", " + str(len(visible)) + ", " + \
# str(total_length) + ", " + str(max_length)
# This doesn't actually work for 2.5D geometries = waiting for API to support them 2/22/2017
input_layer.changeGeometry(feature.id(), QgsGeometry.fromPolylineXY(visible))
# Write Frame
framefile = frame_directory + "/frame" + format(frame, "06d") + ".png"
exporter.exportToImage(framefile, QgsLayoutExporter.ImageExportSettings())
# Clean up: Constantly starting and stopping editing is slow,
# but leaving editing open and accumulating successive changes
# seems to be even slower
input_layer.rollBack()
if status_callback:
status_callback(100, str(frame_count) + " frames exported")
return Nonedef mmqgis_animate_location(print_layout, source_layer, source_key_field, \ destination_layer, destination_key_field, frame_count, \ frame_directory, status_callback = None):
# Error Checks
if not print_layout:
return "Invalid print layout"
exporter = QgsLayoutExporter(print_layout)
if not exporter:
return "No exporter found for print layout " + print_layout.name()
if (not source_layer) or (source_layer.type() != QgsMapLayer.VectorLayer) or (source_layer.featureCount() <= 0):
return "Invalid source layer"
if (not destination_layer) or (destination_layer.type() != QgsMapLayer.VectorLayer) or \
(destination_layer.featureCount() <= 0):
return "Invalid destination layer"
source_key_index = source_layer.dataProvider().fieldNameIndex(source_key_field)
if (source_key_index < 0):
return "Invalid source key field: " + str(long_col)
destination_key_index = destination_layer.dataProvider().fieldNameIndex(destination_key_field)
if (destination_key_index < 0):
return "Invalid destination key field: " + str(long_col)
if frame_count <= 0:
return "Invalid number of frames specified: " + str(frame_count)
if not os.path.isdir(frame_directory):
return "Invalid output directory: " + str(frame_directory)
transform = QgsCoordinateTransform(destination_layer.crs(), source_layer.crs(), QgsProject.instance())
# Find each feature's differential change with each frame
xdifferential = {}
ydifferential = {}
animated_features = 0
for feature in source_layer.getFeatures():
x_start = feature.geometry().centroid().asPoint().x()
y_start = feature.geometry().centroid().asPoint().y()
source_key = str(feature.attributes()[source_key_index])
expression = "\"" + str(destination_key_field) + "\" = '" + source_key + "'"
for destination in destination_layer.getFeatures(expression):
geometry = destination.geometry()
geometry.transform(transform)
x_end = geometry.centroid().asPoint().x()
y_end = geometry.centroid().asPoint().y()
xdifferential[feature.id()] = (x_end - x_start) / frame_count
ydifferential[feature.id()] = (y_end - y_start) / frame_count
#print("Feature " + str(feature.id()) + " " + str(feature.attributes()[2]) + \
# ": " + str(xdifferential[feature.id()]) + ", " + str(ydifferential[feature.id()]))
animated_features = animated_features + 1
break
# Iterate Frames
for frame in range(frame_count + 1):
if status_callback:
if status_callback(100 * frame / frame_count,
str(animated_features) + " features, frame " + str(frame) + " of " + str(frame_count)):
return "Animate columns cancelled on frame " + str(frame)
# Translate (move) shapes
source_layer.startEditing()
for feature in source_layer.getFeatures():
if feature.id() in xdifferential:
x_shift = xdifferential[feature.id()] * frame
y_shift = ydifferential[feature.id()] * frame
#print(str(feature.id()) + ": " + str(x_shift) + ", " + str(y_shift))
source_layer.translateFeature(feature.id(), x_shift, y_shift)
# Write Frame
frame_file = frame_directory + "/frame" + format(frame, "06d") + ".png"
exporter.exportToImage(frame_file, QgsLayoutExporter.ImageExportSettings())
# Clean up: Constantly starting and stopping editing is slow,
# but leaving editing open and accumulating successive changes
# seems to be even slower (6/5/2014)
source_layer.rollBack()
if status_callback:
status_callback(100, str(animated_features) + " features, " + str(frame_count) + " frames")
return Nonedef mmqgis_animate_sequence(print_layout, layers, cumulative, frame_directory, status_callback = None):
# Error Checks
if not print_layout:
return "Invalid print layout"
exporter = QgsLayoutExporter(print_layout)
if not exporter:
return "No exporter found for print layout " + print_layout.name()
frame_count = 0
for layer in layers:
if frame_count < layer.featureCount():
frame_count = layer.featureCount()
if not layer.startEditing():
return "A layer must be editable to animate"
layer.rollBack()
if frame_count <= 1:
return "At least one animated layer must have more than one feature"
if not os.path.isdir(frame_directory):
return "Invalid output directory: " + str(frame_directory)
# Store geometries in a list and delete them from features
geometries = [[] * len(layers)]
feature_ids = [None] * len(layers)
for layer_index, layer in enumerate(layers):
layer.startEditing()
feature_ids[layer_index] = layer.allFeatureIds()
for feature_index, feature in enumerate(layer.getFeatures()):
geometries[layer_index].append(QgsGeometry(feature.geometry()))
layer.changeGeometry(feature_ids[layer_index][feature_index], QgsGeometry())
#feature.setGeometry(QgsGeometry(QgsGeometry.fromPoint(QgsPoint(0,0))))
# Iterate frames
# qgis.mapCanvas().renderComplete.connect(temp_render_complete)
#qgis.mapCanvas().setParallelRenderingEnabled(False)
#qgis.mapCanvas().setCachingEnabled(False)
for frame in range(int(frame_count + 1)):
if status_callback:
if status_callback(100 * frame / frame_count, "Rendering frame " + str(frame)):
return "Animate rows cancelled on frame " + str(frame)
for layer_index, layer in enumerate(layers):
if frame < len(geometries[layer_index]):
layer.changeGeometry(feature_ids[layer_index][frame], \
QgsGeometry(geometries[layer_index][frame]))
if (frame > 0) and (not cumulative):
layer.changeGeometry(feature_ids[layer_index][frame - 1], QgsGeometry())
# qgis.mapCanvas().refresh()
#pixmap = QPixmap(qgis.mapCanvas().mapSettings().outputSize().width(),
# qgis.mapCanvas().mapSettings().outputSize().height())
framefile = frame_directory + "/frame" + format(frame, "06d") + ".png"
# qgis.mapCanvas().saveAsImage(framefile, pixmap)
exporter.exportToImage(framefile, QgsLayoutExporter.ImageExportSettings())
# print("Saved " + str(frame))
# return
# Restore geometries
for layer_index, layer in enumerate(layers):
for frame in range(int(frame_count + 1)):
if frame < len(geometries[layer_index]):
layer.changeGeometry(feature_ids[layer_index][frame], \
QgsGeometry(geometries[layer_index][frame]))
layer.rollBack()
if status_callback:
status_callback(100, str(frame_count) + " frames exported")
return Nonedef mmqgis_animate_zoom(print_layout, start_lat, start_long, start_zoom, \ end_lat, end_long, end_zoom, frame_count, frame_directory, status_callback = None):
# Error checks
if (not print_layout) or (type(print_layout) != QgsPrintLayout):
return "Invalid print layout"
if (type(start_lat) not in [int, float]) or (start_lat < -90) or (start_lat > 90) or \
(type(start_long) not in [int, float]) or (start_long < -180) or (start_long > 180):
return "Start location out of range"
if (type(end_lat) not in [int, float]) or (end_lat < -90) or (end_lat > 90) or \
(type(end_long) not in [int, float]) or (end_long < -180) or (end_long > 180):
return "End location out of range"
if (type(start_zoom) not in [int, float]) or (start_zoom < 1) or (start_zoom > 20):
return "Start zoom out of range"
if (type(end_zoom) not in [int, float]) or (end_zoom < 1) or (end_zoom > 20):
return "End zoom out of range"
if (type(frame_count) != int) or (frame_count <= 0):
return "Frame count must be positive"
if (not frame_directory) or (not os.path.isdir(frame_directory)):
return "Invalid frame directory"
map_item_count = 0
for page_number in range(print_layout.pageCollection().pageCount()):
for item in print_layout.pageCollection().itemsOnPage(page_number):
if item.type() == QgsLayoutItemRegistry.LayoutMap:
map_item_count = map_item_count + 1
if not map_item_count:
return "No map items in the print layout"
wgs84 = QgsCoordinateReferenceSystem("PROJ4:+proj=longlat +datum=WGS84 +no_defs")
# Export frame images
exporter = QgsLayoutExporter(print_layout)
for frame in range(frame_count + 1):
if status_callback:
if status_callback(100 * frame / float(frame_count), "Frame " + str(frame)):
return "Cancelled at frame " + str(frame) + " of " + str(frame_count)
# Calculate the smoothed zoom, latitude, and longitude
ratio = (1 - math.cos((frame / frame_count) * math.pi)) / 2
zoom = start_zoom + ((end_zoom - start_zoom) * ratio)
degree_height = 360 * math.pow(0.5, zoom)
center_lat = start_lat + (ratio * (end_lat - start_lat))
center_long = start_long + (ratio * (end_long - start_long))
# print("Ratio " + str(ratio) + " = " + str(zoom) + " = " + str(degree_height) + " degrees")
# Zoom in/out
old_extents = []
for page_number in range(print_layout.pageCollection().pageCount()):
for item in print_layout.pageCollection().itemsOnPage(page_number):
if item.type() == QgsLayoutItemRegistry.LayoutMap:
# Save the old extent for restoration after printing the fram
old_extent = item.extent()
old_extents.append(old_extent)
# Find the extent top/bottom based on zoom level and center
top = QgsPointXY(center_long, center_lat + (degree_height / 2))
bottom = QgsPointXY(center_long, center_lat - (degree_height / 2))
transform = QgsCoordinateTransform(wgs84, item.crs(), QgsProject.instance())
top = transform.transform(top)
bottom = transform.transform(bottom)
# Find the new width based on the aspect ratio of the old extent
new_width = old_extent.width() * (top.x() - bottom.x()) / old_extent.height()
new_extent = QgsRectangle(bottom.x() - (new_width / 2), bottom.y(),
bottom.x() + (new_width / 2), top.y())
# print(str(new_extent))
item.zoomToExtent(new_extent)
# Print the frame image
file_name = frame_directory + "/frame%04d.png" % frame
exporter.exportToImage(file_name, exporter.ImageExportSettings())
# Restore old extents
for page_number in range(print_layout.pageCollection().pageCount()):
for item in print_layout.pageCollection().itemsOnPage(page_number):
if item.type() == QgsLayoutItemRegistry.LayoutMap:
if len(old_extents) > 0:
item.zoomToExtent(old_extents[0])
old_extents.remove(old_extents[0])
if status_callback:
status_callback(100, str(frame_count) + " frames exported")
return Nonedef mmqgis_attribute_export(input_layer, attribute_names, output_csv_name, \ field_delimiter = ",", line_terminator = "\n", decimal_mark = ".", \ status_callback = None):
# Error checks
if (not input_layer) or (input_layer.type() != QgsMapLayer.VectorLayer) or (input_layer.featureCount() <= 0):
return "Invalid layer"
if not attribute_names:
return "No attributes specified for export"
# CSV Options
layer_options = []
if line_terminator == "\r\n":
layer_options.append("LINEFORMAT=CRLF")
else:
layer_options.append("LINEFORMAT=LF")
if field_delimiter == ";":
layer_options.append("SEPARATOR=SEMICOLON")
elif field_delimiter == "\t":
layer_options.append("SEPARATOR=TAB")
elif field_delimiter == " ":
layer_options.append("SEPARATOR=SPACE")
else:
layer_options.append("SEPARATOR=COMMA")
if not decimal_mark:
decimal_mark = "."
# Build field list
fields = QgsFields()
attribute_indices = []
for attribute in attribute_names:
found = False
for index, field in enumerate(input_layer.fields()):
if field.name() == attribute:
fields.append(field)
attribute_indices.append(index)
found = True
break
if not found:
return "Invalid attribute name: " + attribute
if not attribute_indices:
return "No valid attributes specified"
# Create file writer
outfile = QgsVectorFileWriter(output_csv_name, "utf-8", fields, \
QgsWkbTypes.Unknown, driverName = "CSV", layerOptions = layer_options)
if (outfile.hasError() != QgsVectorFileWriter.NoError):
return "Failure creating output file: " + str(outfile.errorMessage())
# Iterate through each feature in the source layer
for index, feature in enumerate(input_layer.getFeatures()):
if ((index % 50) == 0) and status_callback:
if status_callback(100 * index / input_layer.featureCount(), \
"Exporting " + str(index) + " of " + str(input_layer.featureCount())):
return "Export attributes cancelled on feature " + str(index)
attributes = []
for x in attribute_indices:
attributes.append(feature.attributes()[x])
newfeature = QgsFeature()
newfeature.setAttributes(attributes)
outfile.addFeature(newfeature)
del outfile
if status_callback:
status_callback(100, str(input_layer.featureCount()) + " records exported")
return Nonedef mmqgis_attribute_join(input_layer, input_layer_attribute, input_csv_name, input_csv_attribute, \ output_file_name, status_callback = None):
# Error checks
try:
if (input_layer.type() != QgsMapLayer.VectorLayer):
return "Invalid layer type " + str(input_layer.type()) + " for attribute export"
except Exception as e:
return "Invalid layer: " + str(e)
if (input_layer.wkbType() == None) or (input_layer.wkbType() == QgsWkbTypes.NoGeometry):
return "Layer has no geometries"
join_index = input_layer.dataProvider().fieldNameIndex(input_layer_attribute)
if join_index < 0:
return "Invalid CSV field name: " + str(input_layer_attribute)
if not output_file_name:
return "No output file name given"
file_formats = { ".shp":"ESRI Shapefile", ".geojson":"GeoJSON", ".kml":"KML", ".sqlite":"SQLite", ".gpkg":"GPKG" }
if os.path.splitext(output_file_name)[1] not in file_formats:
return "Unsupported output file format: " + str(output_file_name)
output_file_format = file_formats[os.path.splitext(output_file_name)[1]]
# Open CSV file and read the header
if not input_csv_name:
return "No input CSV file name given"
input_csv = QgsVectorLayer(input_csv_name)
if (not input_csv) or (input_csv.featureCount() <= 0) or (len(input_csv.fields()) <= 0):
return "Failure opening input file: " + str(input_csv_name)
header = [x.name() for x in input_csv.fields()]
# Read shapefile field names
# 12/27/2016: Real numbers imported from shapefiles have a precision of
# zero, which causes them to be written as integers, which causes
# loss of decimal points and total loss of value when values exceed MAXINT.
# This kludge sets the precision to an arbitrary value, which causes
# the OGR writer to consider them floating point.
newfields = QgsFields()
for field in input_layer.fields():
if field.type() == QVariant.Double:
newfields.append(QgsField(field.name(), field.type(), field.typeName(), 12, 4))
else:
newfields.append(QgsField(field.name(), field.type(), field.typeName(),
field.length(), field.precision()))
# Create a combined list of fields with shapefile-safe (<= 10 char) unique names
csv_index = -1
for index in range(0, len(header)):
if header[index].strip().lower() == input_csv_attribute.strip().lower():
csv_index = index
else:
# Shapefile-safe = 10 characters or less
fieldname = header[index].strip()[0:10]
# Rename fields that have duplicate names
suffix = 1
while (newfields.indexFromName(fieldname) >= 0):
suffix = suffix + 1
if (suffix <= 9):
fieldname = fieldname[0:9] + str(suffix)
else:
fieldname = fieldname[0:8] + str(suffix)
# 12/27/2016: String length of 254 is used to prevent a warning thrown
# when the default 255 exceeds the 254 char limit
newfields.append(QgsField(fieldname, QVariant.String, "String", 254))
if csv_index < 0:
return "Field " + str(input_csv_attribute) + " not found in " + str(input_csv_name)
# Create the output shapefile
outfile = QgsVectorFileWriter(output_file_name, "utf-8", newfields, \
input_layer.wkbType(), input_layer.crs(), output_file_format)
if (outfile.hasError() != QgsVectorFileWriter.NoError):
return "Failure creating output file: " + str(outfile.errorMessage())
# Iterate through each feature in the source layer
matched_count = 0
for feature_index, feature in enumerate(input_layer.getFeatures()):
if status_callback and ((feature_index % 50) == 0):
if status_callback(100 * feature_index / input_layer.featureCount(), \
"Feature " + str(feature_index) + " of " + str(input_layer.featureCount()) + \
" (" + str(matched_count) + " matched)"):
return "Cancelled on feature " + str(feature.id()) + " of " + str(input_layer.featureCount())
if feature.geometry() == None:
return "No geometry in on feature " + str(feature_index)
attributes = feature.attributes()
key = attributes[join_index].lower().strip()
for row_index, row in enumerate(input_csv.getFeatures()):
if row.attribute(csv_index).strip().lower() == key:
# print(key + " --------------")
newattributes = []
for value in attributes:
newattributes.append(value)
for combine_index in range(len(row.attributes())):
if combine_index != csv_index:
newattributes.append(row.attribute(combine_index))
newfeature = QgsFeature()
newfeature.setAttributes(newattributes)
newfeature.setGeometry(feature.geometry())
outfile.addFeature(newfeature)
matched_count += 1
del outfile
if matched_count <= 0:
return "No matching records found"
if status_callback:
status_callback(100, str(input_layer.featureCount()) + " layer + " \
+ str(input_csv.featureCount()) + " CSV = " + str(matched_count) + " features")
return Nonedef mmqgis_bearing(start, end):
# http://www.movable-type.co.uk/scripts/latlong.html
start_lon = start.x() * pi / 180
start_lat = start.y() * pi / 180
end_lon = end.x() * pi / 180
end_lat = end.y() * pi / 180
return atan2(sin(end_lon - start_lon) * cos(end_lat), \
(cos(start_lat) * sin(end_lat)) - \
(sin(start_lat) * cos(end_lat) * cos(end_lon - start_lon))) \
* 180 / pidef mmqgis_endpoint(start, distance, degrees):
# bearing in degrees with north = 0, east = 90, west = -90
# Uses the haversine formula for calculation:
# http://www.movable-type.co.uk/scripts/latlong.html
radius = 6378137.0 # meters
start_lon = start.x() * pi / 180
start_lat = start.y() * pi / 180
bearing = degrees * pi / 180
end_lat = asin((sin(start_lat) * cos(distance / radius)) +
(cos(start_lat) * sin(distance / radius) * cos(bearing)))
end_lon = start_lon + atan2( \
sin(bearing) * sin(distance / radius) * cos(start_lat),
cos(distance / radius) - (sin(start_lat) * sin(end_lat)))
return QgsPointXY(end_lon * 180 / pi, end_lat * 180 / pi)def mmqgis_buffer_geometry(geometry, meters): if meters <= 0: return None
# To approximate meaningful meter distances independent of the original CRS,
# the geometry is transformed to an azimuthal equidistant projection
# with the center of the polygon as the origin. After buffer creation,
# the buffer is transformed to WGS 84 and returned. While this may introduce
# some deviation from the original CRS, buffering is assumed in practice
# to be a fairly inexact operation that can tolerate such deviation
wgs84 = QgsCoordinateReferenceSystem("PROJ4:+proj=longlat +datum=WGS84 +no_defs")
latitude = str(geometry.centroid().asPoint().y())
longitude = str(geometry.centroid().asPoint().x())
#proj4 = "+proj=aeqd +lat_0=" + str(geometry.centroid().asPoint().y()) + \
# " +lon_0=" + str(geometry.centroid().asPoint().x()) + \
# " +x_0=0 +y_0=0 +datum=WGS84 +units=m +no_defs"
# For some reason, Azimuthal Equidistant transformation noticed to not be
# working on 10 July 2014. World Equidistant Conic works, but there may be errors.
proj4 = "+proj=eqdc +lat_0=0 +lon_0=0 +lat_1=60 +lat_2=60 " + \
"+x_0=0 +y_0=0 +ellps=WGS84 +datum=WGS84 +units=m +no_defs"
azimuthal_equidistant = QgsCoordinateReferenceSystem()
azimuthal_equidistant.createFromProj4(proj4)
transform = QgsCoordinateTransform(wgs84, azimuthal_equidistant, QgsProject.instance())
geometry.transform(transform)
newgeometry = geometry.buffer(meters, 7)
wgs84 = QgsCoordinateReferenceSystem()
wgs84.createFromProj4("+proj=longlat +datum=WGS84 +no_defs")
transform = QgsCoordinateTransform(azimuthal_equidistant, wgs84, QgsProject.instance())
newgeometry.transform(transform)
return newgeometrydef mmqgis_buffer_point(point, meters, edges, rotation_degrees): if (meters <= 0) or (edges < 3): return None
# Points are treated separately from other geometries so that discrete
# edges can be supplied for non-circular buffers that are not supported
# by the QgsGeometry.buffer() function
wgs84 = QgsCoordinateReferenceSystem()
wgs84.createFromProj4("+proj=longlat +datum=WGS84 +no_defs")
# print "Point " + str(point.x()) + ", " + str(point.y()) + " meters " + str(meters)
polyline = []
for edge in range(0, edges + 1):
degrees = ((float(edge) * 360.0 / float(edges)) + rotation_degrees) % 360
polyline.append(mmqgis_endpoint(QgsPointXY(point), meters, degrees))
return QgsGeometry.fromPolygonXY([polyline])def mmqgis_buffer_line_side(geometry, width, direction):
# direction should be 0 for north side, 90 for east, 180 for south, 270 for west
# print "\nmmqgis_buffer_line_side(" + str(direction) + ")"
if (geometry.wkbType() == QgsWkbTypes.MultiLineString) or \
(geometry.wkbType() == QgsWkbTypes.MultiLineString25D):
multipolygon = None
for line in geometry.asMultiPolyline():
segment = mmqgis_buffer_line_side(QgsGeometry.fromPolylineXY(line), width, direction)
if multipolygon == None:
multipolygon = segment
else:
multipolygon = multipolygon.combine(segment)
# print " Build multipolygon " + str(multipolygon.isGeosValid())
# Multiline always has multipolygon buffer even if buffers merge into one polygon
if multipolygon.wkbType() == QgsWkbTypes.Polygon:
multipolygon = QgsGeometry.fromMultiPolygonXY([multipolygon.asPolygon()])
# print "Final Multipolygon " + str(multipolygon.isGeosValid())
return multipolygon
if (geometry.wkbType() != QgsWkbTypes.LineString) and \
(geometry.wkbType() != QgsWkbTypes.LineString25D):
return geometry
points = geometry.asPolyline()
line_bearing = mmqgis_bearing(points[0], points[-1]) % 360
# Determine side of line to buffer based on angle from start point to end point
# "bearing" will be 90 for right side buffer, -90 for left side buffer
direction = round((direction % 360) / 90) * 90
if (direction == 0): # North
if (line_bearing >= 180):
bearing = 90 # Right
else:
bearing = -90 # Left
elif (direction == 90): # East
if (line_bearing >= 270) or (line_bearing < 90):
bearing = 90 # Right
else:
bearing = -90 # Left
elif (direction == 180): # South
if (line_bearing < 180):
bearing = 90 # Right
else:
bearing = -90 # Left
else: # West
if (line_bearing >= 90) and (line_bearing < 270):
bearing = 90 # Right
else:
bearing = -90 # Left
# Buffer individual segments
polygon = None
for z in range(0, len(points) - 1):
b1 = mmqgis_bearing(points[z], points[z + 1]) % 360
# Form rectangle beside line
# 2% offset mitigates topology floating-point errors
linestring = [QgsPointXY(points[z])]
if (z == 0):
linestring.append(mmqgis_endpoint(points[z], width, b1 + bearing))
else:
linestring.append(mmqgis_endpoint(points[z], width, b1 + (1.02 * bearing)))
linestring.append(mmqgis_endpoint(points[z + 1], width, b1 + bearing))
# Determine if rounded convex elbow is needed
if (z < (len(points) - 2)):
b2 = mmqgis_bearing(points[z + 1], points[z + 2]) % 360
elbow = b2 - b1
if (elbow < -180):
elbow = elbow + 360
elif (elbow > 180):
elbow = elbow - 360
# print str(b1) + ", " + str(b2) + " = " + str(elbow)
# 8-step interpolation of arc
if (((bearing > 0) and (elbow < 0)) or \
((bearing < 0) and (elbow > 0))):
for a in range(1,8):
b = b1 + (elbow * a / 8.0) + bearing
linestring.append(mmqgis_endpoint(points[z + 1], width, b))
# print " arc: " + str(b)
linestring.append(mmqgis_endpoint(points[z + 1], width, b2 + bearing))
# Close polygon
linestring.append(QgsPointXY(points[z + 1]))
linestring.append(QgsPointXY(points[z]))
segment = QgsGeometry.fromPolygonXY([linestring])
# print linestring
# print " Line to polygon " + str(segment.isGeosValid())
if (polygon == None):
polygon = segment
else:
polygon = polygon.combine(segment)
#print " Polygon build " + str(polygon.isGeosValid())
#if not polygon.isGeosValid():
# print polygon.asPolygon()
# print " Final polygon " + str(polygon.isGeosValid())
return polygondef mmqgis_buffers(input_layer, selected_only, radius_attribute, radius, radius_unit, \ edge_attribute, edge_count, rotation_attribute, rotation_degrees, \ output_file_name, status_callback = None):
# Error checking
try:
if (input_layer.type() != QgsMapLayer.VectorLayer):
return "Invalid layer type for buffering: " + str(input_layer.type())
except Exception as e:
return "Invalid layer: " + str(e)
# Radius
radius_attribute_index = -1
if radius_attribute:
radius_attribute_index = input_layer.dataProvider().fieldNameIndex(radius_attribute)
if (radius_attribute_index < 0):
return "Invalid radius attribute name: " + str(radius_attribute)
else:
try:
radius = float(radius)
except Exception as e:
return "Invalid radius: " + str(radius)
if (radius <= 0):
return "Radius must be greater than zero (" + str(radius) + ")"
# Edges
edge_attribute_index = -1
if (input_layer.wkbType() in [QgsWkbTypes.Point, QgsWkbTypes.Point25D, \
QgsWkbTypes.MultiPoint, QgsWkbTypes.MultiPoint25D]):
if edge_attribute:
edge_attribute_index = input_layer.dataProvider().fieldNameIndex(edge_attribute)
if (edge_attribute_index < 0):
return "Invalid edge attribute name: " + str(edge_attribute)
else:
try:
edge_count = int(edge_count)
except Exception as e:
return "Invalid edge count: " + str(edge_count)
if (edge_count <= 0):
return "Number of edges must be greater than zero (" + str(edge_count) + ")"
# Rotation
rotation_attribute_index = -1
if rotation_attribute:
rotation_attribute_index = input_layer.dataProvider().fieldNameIndex(rotation_attribute)
if (rotation_attribute_index < 0):
return "Invalid rotation attribute name: " + str(rotation_attribute)
else:
try:
rotation_degrees = float(rotation_degrees)
except Exception as e:
return "Invalid rotation degrees: " + str(rotation_degrees)
# Create the output file
wgs84 = QgsCoordinateReferenceSystem()
wgs84.createFromProj4("+proj=longlat +datum=WGS84 +no_defs")
transform = QgsCoordinateTransform(input_layer.crs(), wgs84, QgsProject.instance())
# print layer.crs().toProj4() + " -> " + wgs84.toProj4()
if not output_file_name:
return "No output file name given"
file_formats = { ".shp":"ESRI Shapefile", ".geojson":"GeoJSON", ".kml":"KML", ".sqlite":"SQLite", ".gpkg":"GPKG" }
if os.path.splitext(output_file_name)[1] not in file_formats:
return "Unsupported output file format: " + str(output_file_name)
output_file_format = file_formats[os.path.splitext(output_file_name)[1]]
outfile = QgsVectorFileWriter(output_file_name, "utf-8", input_layer.fields(), \
QgsWkbTypes.Polygon, wgs84, output_file_format)
if (outfile.hasError() != QgsVectorFileWriter.NoError):
return str(outfile.errorMessage())
# Create buffers for each feature
buffercount = 0
feature_count = input_layer.featureCount();
if selected_only:
feature_list = input_layer.selectedFeatures()
else:
feature_list = input_layer.getFeatures()
for feature_index, feature in enumerate(feature_list):
if status_callback:
if status_callback(100 * feature.id() / feature_count, \
"Feature " + str(feature.id()) + " of " + str(feature_count)):
return "Buffering cancelled on feature " + str(feature.id()) + " of " + str(feature_count)
if radius_attribute_index < 0:
feature_radius = radius
else:
try:
feature_radius = float(feature.attributes()[radius_attribute_index])
except:
feature_radius = 0.0
if feature_radius <= 0:
continue
# Buffer radii are always in meters
if radius_unit == "Kilometers":
feature_radius = feature_radius * 1000
elif radius_unit == "Feet":
feature_radius = feature_radius / 3.2808399
elif radius_unit == "Miles":
feature_radius = feature_radius * 1609.344
elif radius_unit == "Nautical Miles":
feature_radius = feature_radius * 1852
if feature_radius <= 0:
continue
if edge_attribute_index < 0:
feature_edges = edge_count
else:
try:
feature_edges = int(feature.attributes()[edge_attribute_index])
except:
feature_edges = 32 # default to circle
if rotation_attribute_index < 0:
feature_rotation = rotation_degrees
else:
try:
feature_rotation = float(feature.attributes()[rotation_attribute_index])
except:
feature_rotation = 0.0
geometry = feature.geometry()
geometry.transform(transform) # Needs to be WGS 84 to use Haversine distance calculation
# print "Transform " + str(x) + ": " + str(geometry.centroid().asPoint().x())
if (geometry.wkbType() in [QgsWkbTypes.Point, QgsWkbTypes.Point25D, \
QgsWkbTypes.MultiPoint, QgsWkbTypes.MultiPoint25D]):
newgeometry = mmqgis_buffer_point(geometry.asPoint(), feature_radius, feature_edges, feature_rotation)
elif (geometry.wkbType() in [QgsWkbTypes.LineString, QgsWkbTypes.LineString25D, \
QgsWkbTypes.MultiLineString, QgsWkbTypes.MultiLineString25D]):
if (edge_attribute == "Flat End"):
# newgeometry = mmqgis_buffer_line_flat_end(geometry, feature_radius)
north = mmqgis_buffer_line_side(QgsGeometry(geometry), feature_radius, 0)
south = mmqgis_buffer_line_side(QgsGeometry(geometry), feature_radius, 180)
newgeometry = north.combine(south)
elif (edge_attribute == "North Side"):
newgeometry = mmqgis_buffer_line_side(geometry, feature_radius, 0)
elif (edge_attribute == "East Side"):
newgeometry = mmqgis_buffer_line_side(geometry, feature_radius, 90)
elif (edge_attribute == "South Side"):
newgeometry = mmqgis_buffer_line_side(geometry, feature_radius, 180)
elif (edge_attribute == "West Side"):
newgeometry = mmqgis_buffer_line_side(geometry, feature_radius, 270)
else: # "Rounded"
newgeometry = mmqgis_buffer_geometry(geometry, feature_radius)
else:
newgeometry = mmqgis_buffer_geometry(geometry, feature_radius)
if newgeometry == None:
return "Failure converting geometry for feature " + str(buffercount)
else:
newfeature = QgsFeature()
newfeature.setGeometry(newgeometry)
newfeature.setAttributes(feature.attributes())
outfile.addFeature(newfeature)
buffercount = buffercount + 1
del outfile
if status_callback:
status_callback(100, str(buffercount) + " buffers created for " + str(feature_count) + " features")
return Nonedef mmqgis_change_projection(input_layer, new_crs, output_file_name, status_callback = None):
# Error checks
if type(input_layer) not in [ QgsMapLayer, QgsVectorLayer ]:
return "Invalid layer type for modification: " + str(type(input_layer))
if type(new_crs) != QgsCoordinateReferenceSystem:
return "Invalid CRS"
if not output_file_name:
return "No output file name given"
file_formats = { ".shp":"ESRI Shapefile", ".geojson":"GeoJSON", ".kml":"KML", ".sqlite":"SQLite", ".gpkg":"GPKG" }
if os.path.splitext(output_file_name)[1] not in file_formats:
return "Unsupported output file format: " + str(output_file_name)
output_file_format = file_formats[os.path.splitext(output_file_name)[1]]
outfile = QgsVectorFileWriter(output_file_name, "utf-8", input_layer.fields(), \
input_layer.wkbType(), new_crs, output_file_format)
if (outfile.hasError() != QgsVectorFileWriter.NoError):
return str(outfile.errorMessage())
transform = QgsCoordinateTransform(input_layer.crs(), new_crs, QgsProject.instance())
for index, feature in enumerate(input_layer.getFeatures()):
if status_callback and ((index % 50) == 0):
if status_callback(100 * index / input_layer.featureCount(), "Feature " + str(index)):
return "Cancelled on feature " + str(index) + " of " + str(input_layer.featureCount())
new_feature = QgsFeature()
new_feature.setAttributes(feature.attributes())
new_geometry = feature.geometry()
try:
new_geometry.transform(transform)
except Exception as e:
return "Feature " + str(index) + " could not be transformed to the new projection: " + str(e)
new_feature.setGeometry(new_geometry)
outfile.addFeature(new_feature)
if status_callback:
status_callback(100, "Changed projection for " + str(input_layer.featureCount()) + " features")
return Nonedef mmqgis_delete_duplicate_geometries(input_layer, output_file_name, status_callback = None):
# Error checks
try:
if (input_layer.type() != QgsMapLayer.VectorLayer):
return "Invalid layer type for modification: " + str(input_layer.type())
except Exception as e:
return "Invalid layer: " + str(e)
if not output_file_name:
return "No output file name given"
file_formats = { ".shp":"ESRI Shapefile", ".geojson":"GeoJSON", ".kml":"KML", ".sqlite":"SQLite", ".gpkg":"GPKG" }
if os.path.splitext(output_file_name)[1] not in file_formats:
return "Unsupported output file format: " + str(output_file_name)
output_file_format = file_formats[os.path.splitext(output_file_name)[1]]
outfile = QgsVectorFileWriter(output_file_name, "utf-8", input_layer.fields(), \
input_layer.wkbType(), input_layer.crs(), output_file_format)
if (outfile.hasError() != QgsVectorFileWriter.NoError):
return str(outfile.errorMessage())
# Read geometries into an array
# Have to save as WKT because saving geometries causes segfault
# when they are used with equal() later
geometries = []
for feature in input_layer.getFeatures():
geometries.append(feature.geometry().asWkt())
# NULL duplicate geometries
for x in range(0, len(geometries) - 1):
if geometries[x] == None:
continue
if status_callback and ((x % 50) == 0):
if status_callback(100 * x / len(geometries), "Feature " + str(x)):
return "Cancelled on feature " + str(x) + " of " + str(len(geometries))
for y in range(x + 1, len(geometries)):
#print "Comparing " + str(x) + ", " + str(y)
if geometries[x] == geometries[y]:
#print "None " + str(x)
geometries[y] = None
writecount = 0
for index, feature in enumerate(input_layer.getFeatures()):
if geometries[index] != None:
writecount += 1
outfile.addFeature(feature)
del outfile
if status_callback:
status_callback(100, str(writecount) + " unique of " + str(input_layer.featureCount()))
return Nonedef mmqgis_float_to_text(input_layer, attributes, separator, \ decimals, multiplier, prefix, suffix, \ output_file_name, status_callback = None):
# Error checks
try:
if input_layer.type() != QgsMapLayer.VectorLayer:
return "Input layer must be a vector layer"
except:
return "Invalid input layer"
if decimals < 0:
return "Invalid number of decimals: " + str(decimals)
if not multiplier:
return "Invalid multiplier: " + str(multiplier)
if not separator:
separator = ""
if not prefix:
prefix = ""
if not suffix:
suffix = ""
# Build dictionary of fields with selected fields for conversion to floating point
changecount = 0
fieldchanged = []
destfields = QgsFields();
for index, field in enumerate(input_layer.fields()):
if field.name() in attributes:
if not (field.type() in [QVariant.Double, QVariant.Int, QVariant.UInt, \
QVariant.LongLong, QVariant.ULongLong]):
return "Cannot convert non-numeric field: " + str(field.name())
changecount += 1
fieldchanged.append(True)
destfields.append(QgsField (field.name(), QVariant.String, field.typeName(), \
20, 0, field.comment()))
else:
fieldchanged.append(False)
destfields.append(QgsField (field.name(), field.type(), field.typeName(), \
field.length(), field.precision(), field.comment()))
if (changecount <= 0):
return "No numeric fields selected for conversion"
# Create the output file
if not output_file_name:
return "No output file name given"
file_formats = { ".shp":"ESRI Shapefile", ".geojson":"GeoJSON", ".kml":"KML", ".sqlite":"SQLite", ".gpkg":"GPKG" }
if os.path.splitext(output_file_name)[1] not in file_formats:
return "Unsupported output file format: " + str(output_file_name)
output_file_format = file_formats[os.path.splitext(output_file_name)[1]]
outfile = QgsVectorFileWriter(output_file_name, "utf-8", destfields, \
input_layer.wkbType(), input_layer.crs(), output_file_format)
if (outfile.hasError() != QgsVectorFileWriter.NoError):
return str(outfile.errorMessage())
# Write the features with modified attributes
feature_count = input_layer.featureCount();
for feature_index, feature in enumerate(input_layer.getFeatures()):
if status_callback and ((feature_index % 50) == 0):
if status_callback(100 * feature_index / feature_count, \
"Feature " + str(feature_index) + " of " + str(feature_count)):
return "Cancelled on feature " + str(feature_index) + " of " + str(feature_count)
attributes = feature.attributes()
for index, field in enumerate(input_layer.fields()):
if fieldchanged[index]:
# floatvalue, test = attributes[index].toDouble()
try:
floatvalue = multiplier * float(attributes[index])
except:
floatvalue = 0
value = ("{:,." + str(decimals) + "f}").format(floatvalue)
if (separator == ' ') or (separator == '.'):
# European-style numbers: 1.203.347,42
value = value.replace(".", "dec")
value = value.replace(",", separator)
value = value.replace("dec", ",")
elif separator == "":
value = value.replace(",", "")
else:
value = value.replace(",", separator)
attributes[index] = str(prefix) + str(value) + str(suffix)
feature.setAttributes(attributes)
outfile.addFeature(feature)
del outfile
if status_callback:
status_callback(100, str(len(attributes)) + " fields, " + str(input_layer.featureCount()) + " features")
return Nonedef mmqgis_proxy_settings():
try:
settings = QSettings()
proxyEnabled = settings.value("proxy/proxyEnabled", "")
proxyType = settings.value("proxy/proxyType", "" )
proxyHost = settings.value("proxy/proxyHost", "" )
proxyPort = settings.value("proxy/proxyPort", "" )
proxyUser = settings.value("proxy/proxyUser", "" )
proxyPassword = settings.value("proxy/proxyPassword", "" )
# http://stackoverflow.com/questions/1450132/proxy-with-urllib2
if proxyEnabled == "true":
if proxyUser:
proxy = urllib.request.ProxyHandler({'http': 'http://' + proxyUser + ':' +
proxyPassword + '@' + proxyHost + ':' + proxyPort})
else:
proxy = urllib.request.ProxyHandler({'http': 'http://' + proxyHost + ':' + proxyPort})
opener = urllib.request.build_opener(proxy)
urllib.request.install_opener(opener)
except:
passdef mmqgis_geocode_reverse(input_layer, web_service, api_key, use_first, output_file_name, status_callback = None):
# Error checks
web_services = ["Google", "OpenStreetMap / Nominatim"]
if web_service not in web_services:
return "Invalid web service name: " + str(web_service)
if (web_service == "Google") and (not api_key):
return "A Google Maps API key is required\n" + \
"https://developers.google.com/maps/documentation/javascript/get-api-key"
# Create the output file
try:
fields = input_layer.fields()
except:
return "Invalid layer"
if web_service == "Google":
for field_name in ["result_num", "status", "formatted_address", "place_id", \
"location_type", "latlong"]:
fields.append(QgsField (field_name, QVariant.String))
elif web_service == "OpenStreetMap / Nominatim":
for field_name in ["result_num", "osm_id", "display_name", "category", "type", "latlong"]:
fields.append(QgsField (field_name, QVariant.String))
if not output_file_name:
return "No output file name given"
file_formats = { ".shp":"ESRI Shapefile", ".geojson":"GeoJSON", ".kml":"KML", ".sqlite":"SQLite", ".gpkg":"GPKG" }
if os.path.splitext(output_file_name)[1] not in file_formats:
return "Unsupported output file format: " + str(output_file_name)
output_file_format = file_formats[os.path.splitext(output_file_name)[1]]
outfile = QgsVectorFileWriter(output_file_name, "utf-8", fields, \
input_layer.wkbType(), input_layer.crs(), output_file_format)
if (outfile.hasError() != QgsVectorFileWriter.NoError):
return str(outfile.errorMessage())
# HTTP(S) proxy settings from qgis options settings
mmqgis_proxy_settings()
# Coordinates to the web services assumed to be WGS 84 latitude/longitude
wgs84 = QgsCoordinateReferenceSystem()
wgs84.createFromProj4("+proj=longlat +datum=WGS84 +no_defs")
transform = QgsCoordinateTransform(input_layer.crs(), wgs84, QgsProject.instance())
# Iterate through each feature in the source layer
feature_count = input_layer.featureCount()
result_count = 0
for feature_index, feature in enumerate(input_layer.getFeatures()):
if status_callback and ((feature_index % 3) == 0):
if status_callback(100 * feature_index / feature_count,
"Feature " + str(feature_index) + " of " + str(feature_count)):
return "Cancelled on feature " + str(feature_index)
# Use the centroid as the latitude and longitude
point = feature.geometry().centroid().asPoint()
point = transform.transform(point)
latitude = point.y()
longitude = point.x()
# API URL
if web_service == "Google":
url = "https://maps.googleapis.com/maps/api/geocode/json?latlng=" + \
str(latitude) + "," + str(longitude) + "&key=" + api_key
else:
url = "https://nominatim.openstreetmap.org/reverse?&format=geojson&lat=" + \
str(latitude) + "&lon=" + str(longitude)
# Query the API
max_attempts = 5
for attempt in range(1, max_attempts + 1):
try:
# https://operations.osmfoundation.org/policies/nominatim/
user_agent = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_3)"
request = urllib.request.Request(url, headers={'User-Agent': user_agent})
json_string = urllib.request.urlopen(request).read()
break
except Exception as e:
if (attempt >= max_attempts):
return "Failure connecting to API: " + str(e)
# Wait a second and try again
time.sleep(1)
new_feature = QgsFeature()
new_feature.setGeometry(feature.geometry())
try:
json_array = json.loads(json_string.decode("utf-8"))
except Exception as e:
attributes = feature.attributes()
attributes.append(str(-1))
attributes.append(str(e))
new_feature.setAttributes(feature.attributes())
outfile.addFeature(new_feature)
continue
if web_service == "Google":
# Check status
# https://developers.google.com/maps/documentation/geocoding/intro#GeocodingRequests
if "status" in json_array:
status = json_array["status"]
else:
status = "No status"
if "error_meessage" in json_array:
error_messsage = json_array["error_message"]
else:
error_message = ""
if (status == "REQUEST_DENIED") or (status == "OVER_QUERY_LIMIT"):
return status + ": " + error_message
if status != "OK":
attributes = feature.attributes()
attributes.append(str(-1))
attributes.append(status)
attributes.append(error_message)
new_feature.setAttributes(attributes)
outfile.addFeature(new_feature)
continue
# No results found (shouldn't happen?)
if (not "results" in json_array) or (len(json_array["results"]) <= 0):
attributes = feature.attributes()
attributes.append(str(-1))
attributes.append("No results")
new_feature.setAttributes(attributes)
outfile.addFeature(new_feature)
continue
# Iterate through results
for result_num, result in enumerate(json_array['results']):
# Collect attributes
attributes = feature.attributes()
attributes.append(str(result_num))
attributes.append(status)
if "formatted_address" in result:
attributes.append(str(result["formatted_address"]))
else:
attributes.append("")
if "place_id" in result:
attributes.append(str(result["place_id"]))
else:
attributes.append("")
try:
latitude = float(result["geometry"]["location"]["lat"])
longitude = float(result["geometry"]["location"]["lng"])
attributes.append(str(result["geometry"]["location_type"]))
attributes.append(str(latitude) + "," + str(longitude))
except Exception as e:
attributes.append("")
attributes.append("")
# Add feature
new_feature.setAttributes(attributes)
outfile.addFeature(new_feature)
result_count = result_count + 1
if use_first:
break
else: # OSM/Nominatim
# https://nominatim.org/release-docs/develop/api/Reverse/
if "error" in json_array:
if "message" in json_array["error"]:
error_message = json_array["error"]["message"]
else:
error_message = "Undefined Error"
attributes = feature.attributes()
attributes.append(str(-1))
attributes.append(error_message)
new_feature.setAttributes(attributes)
outfile.addFeature(new_feature)
continue
if (not "features" in json_array) or (len(json_array["features"]) <= 0):
attributes = feature.attributes()
attributes.append(str(-1))
attributes.append("No results")
new_feature.setAttributes(attributes)
outfile.addFeature(new_feature)
continue
for result_num, result in enumerate(json_array['features']):
# Collect attributes
attributes = feature.attributes()
attributes.append(str(result_num))
for property_name in ["osm_id", "display_name", "category", "type"]:
if ("properties" in result) and (property_name in result["properties"]):
attributes.append(str(result["properties"][property_name]))
else:
attributes.append("")
try:
latitude = float(result["geometry"]["coordinates"][1])
longitude = float(result["geometry"]["coordinates"][0])
attributes.append(str(latitude) + "," + str(longitude))
except Exception as e:
attributes.append("")
# Add feature
new_feature.setAttributes(attributes)
outfile.addFeature(new_feature)
result_count = result_count + 1
if use_first:
break
del outfile
if status_callback:
status_callback(100, str(result_count) + " of " + str(input_layer.featureCount()) + " reverse geocoded")
return Nonedef mmqgis_searchable_streetname(name):
if not name:
return ""
# name = str(name).strip().lower()
name = name.strip().lower()
name = name.replace(".", "")
name = name.replace(" street", " st")
name = name.replace(" avenue", " av")
name = name.replace(" plaza", " plz")
name = name.replace(" drive", " dr")
name = name.replace("saint ", "st ")
name = name.replace("fort ", "ft ")
name = name.replace(" ave", " av")
name = name.replace("east", "e")
name = name.replace("west", "w")
name = name.replace("north", "n")
name = name.replace("south", "s")
name = name.replace("1st", "1")
name = name.replace("2nd", "2")
name = name.replace("3rd", "3")
name = name.replace("4th", "4")
name = name.replace("5th", "5")
name = name.replace("6th", "6")
name = name.replace("7th", "7")
name = name.replace("8th", "8")
name = name.replace("9th", "9")
name = name.replace("0th", "0")
name = name.replace("1th", "1")
name = name.replace("2th", "2")
name = name.replace("3th", "3")
name = name.replace("first", "1")
name = name.replace("second", "2")
name = name.replace("third", "3")
name = name.replace("fourth", "4")
name = name.replace("fifth", "5")
name = name.replace("sixth", "6")
name = name.replace("seventh", "7")
name = name.replace("eighth", "8")
name = name.replace("ninth", "9")
name = name.replace("tenth", "10")
return namedef mmqgis_geocode_street_layer(input_csv_name, number_column, street_name_column, zip_column, \ input_layer, street_name_attr, left_from_attr, left_to_attr, left_zip_attr, \ right_from_attr, right_to_attr, right_zip_attr, \ from_x_attr, from_y_attr, to_x_attr, to_y_attr, setback, \ output_file_name, not_found_file, status_callback = None):
# Error checks
try:
input_layer.featureCount()
except:
return "Invalid input street layer"
if (input_layer.wkbType() != QgsWkbTypes.LineString) and \
(input_layer.wkbType() != QgsWkbTypes.LineString25D) and \
(input_layer.wkbType() != QgsWkbTypes.MultiLineString) and \
(input_layer.wkbType() != QgsWkbTypes.MultiLineString25D):
return "Street layer must be lines or multilines (WKB Type " + str(input_layer.wkbType()) + ")"
if (not street_name_attr) or (input_layer.dataProvider().fieldNameIndex(str(street_name_attr)) < 0):
return "Invalid street name attribute: " + str(street_name_attr)
if (not left_from_attr) or (input_layer.dataProvider().fieldNameIndex(str(left_from_attr)) < 0):
return "Invalid left from attribute: " + str(left_from_attr)
if (not left_to_attr) or (input_layer.dataProvider().fieldNameIndex(str(left_to_attr)) < 0):
return "Invalid left to attribute: " + str(left_to_attr)
if left_zip_attr and (input_layer.dataProvider().fieldNameIndex(str(left_zip_attr)) < 0):
return "Invalid left ZIP Code attribute: " + str(left_zip_attr)
if (not right_from_attr) or (input_layer.dataProvider().fieldNameIndex(str(right_from_attr)) < 0):
return "Invalid right from attribute: " + str(right_from_attr)
if (not right_to_attr) or (input_layer.dataProvider().fieldNameIndex(str(right_to_attr)) < 0):
return "Invalid right to attribute: " + str(right_to_attr)
if right_zip_attr and (input_layer.dataProvider().fieldNameIndex(str(right_zip_attr)) < 0):
return "Invalid right ZIP Code attribute: " + str(right_zip_attr)
if from_x_attr and (input_layer.dataProvider().fieldNameIndex(str(from_x_attr)) < 0):
return "Invalid from x attribute: " + str(from_x_attr)
if from_y_attr and (input_layer.dataProvider().fieldNameIndey(str(from_y_attr)) < 0):
return "Invalid from y attribute: " + str(from_y_attr)
if to_x_attr and (input_layer.dataProvider().fieldNameIndex(str(to_x_attr)) < 0):
return "Invalid to x attribute: " + str(to_x_attr)
if to_y_attr and (input_layer.dataProvider().fieldNameIndey(str(to_y_attr)) < 0):
return "Invalid to y attribute: " + str(to_y_attr)
try:
setback = float(setback)
except Exception as e:
return "Invalid setback value: " + str(e)
# Open CSV file and validate fields
if status_callback:
status_callback(0, "Opening Files")
if not input_csv_name:
return "No input CSV file name given"
input_csv = QgsVectorLayer(input_csv_name)
if (not input_csv) or (input_csv.featureCount() <= 0) or (len(input_csv.fields()) <= 0):
return "Failure opening input file: " + str(input_csv_name)
if input_csv.dataProvider().fieldNameIndex(str(number_column)) < 0:
return "Invalid CSV number column: " + str(number_column)
if input_csv.dataProvider().fieldNameIndex(str(street_name_column)) < 0:
return "Invalid CSV street name column: " + str(street_name_column)
if zip_column and (input_csv.dataProvider().fieldNameIndex(str(zip_column)) < 0):
return "Invalid CSV ZIP Code column: " + str(zip_column)
# Create the not found file for addresses that were not valid or not found
not_found = QgsVectorFileWriter(not_found_file, "utf-8", input_csv.fields(), \
QgsWkbTypes.Unknown, driverName = "CSV")
if (not_found.hasError() != QgsVectorFileWriter.NoError):
return "Failure creating not found CSV file: " + str(not_found.errorMessage())
# Create the output shapefile
if not output_file_name:
return "No output file name given"
file_formats = { ".shp":"ESRI Shapefile", ".geojson":"GeoJSON", ".kml":"KML", ".sqlite":"SQLite", ".gpkg":"GPKG" }
if os.path.splitext(output_file_name)[1] not in file_formats:
return "Unsupported output file format: " + str(output_file_name)
output_file_format = file_formats[os.path.splitext(output_file_name)[1]]
new_fields = input_csv.fields()
new_fields.append(QgsField("Longitude", QVariant.Double, "real", 24, 16))
new_fields.append(QgsField("Latitude", QVariant.Double, "real", 24, 16))
new_fields.append(QgsField("Side", QVariant.String))
outfile = QgsVectorFileWriter(output_file_name, "utf-8", new_fields, QgsWkbTypes.Point, \
input_layer.crs(), output_file_format)
if (outfile.hasError() != QgsVectorFileWriter.NoError):
return "Failure creating output file: " + str(outfile.errorMessage())
# Iterate through each CSV row
matched_count = 0
for address_index, address in enumerate(input_csv.getFeatures()):
if status_callback and ((address_index % 2) == 0):
if status_callback(100 * address_index / input_csv.featureCount(),
str(matched_count) + " of " + str(address_index) + " matched"):
return "Cancelled geocode at address " + \
str(address_index) + " of " + str(input_csv.featureCount())
# Find parts of this address
street = mmqgis_searchable_streetname(str(address.attribute(street_name_column)))
if not street:
new_feature = QgsFeature()
new_feature.setAttributes(address.attributes())
not_found.addFeature(new_feature)
continue
try:
number = int(address.attribute(number_column))
except:
number = 0
zip_code = None
if zip_column:
zip_code = str(address.attribute(zip_column))
# Iterate through each feature in the street layer
found = False
for feature_index, feature in enumerate(input_layer.getFeatures()):
# Compare street names
feature_street = mmqgis_searchable_streetname(str(feature.attribute(street_name_attr)))
if (not feature_street) or (feature_street != street):
continue # Not on this street
# Compare street numbers and find distance along the side
try:
left_to_number = int(feature.attribute(left_to_attr))
left_from_number = int(feature.attribute(left_from_attr))
right_to_number = int(feature.attribute(right_to_attr))
right_from_number = int(feature.attribute(right_from_attr))
except:
left_to_number = 0
left_from_number = 0
right_to_number = 0
right_from_number = 0
left_side = False
right_side = False
if ((left_from_number < left_to_number) and \
(number >= left_from_number) and (number <= left_to_number)) or \
((number <= left_from_number) and (number >= left_to_number)):
left_side = True
if left_from_number == left_to_number:
distance_ratio = 0
else:
distance_ratio = (number - left_from_number) / (left_to_number - left_from_number)
if ((right_from_number < right_to_number) and \
(number >= right_from_number) and (number <= right_to_number)) or \
((number <= right_from_number) and (number >= right_to_number)):
# Check odd/even numbering match
if (not left_side) or ((number % 2) == (right_from_number % 2)):
left_side = False
right_side = True
if right_from_number == right_to_number:
distance_ratio = 0
else:
distance_ratio = (number - right_from_number) / \
(right_to_number - right_from_number)
# print("Street match " + str(address.attribute("Name")) + " number " + \
# str(number) + " right " + str(right_from_number) + " - " + str(right_to_number) + \
# ", left " + str(left_from_number) + " - " + str(left_to_numbHey there, I cobbled together code to do what I wanted, but it does not yet honor the "selected only" checkbox. The attached .py file (rename to get rid of .txt) had the edits. I took a function from the mmqgis tools and modified to do all fields. I changed the statement at top to "from qgis.core import *" I don't know if that loads more than needed and slows things. I'm super happy this seems to work though. I wrote my own "table grouper" program in .net to crunch the table rows so I can get counts and lengths for things. We use the GIS data like a catalog of things we want to build, then want a quantity list at the end. Wish I know how to tell python to start a .net program and send it params. I guess I could compile mine as a console app, and have python call it and send command line params, or a filename with params. I'm super new at the qgis python api so need to spend about a week getting a feel for python. thx
James Maeding Project Engineer Direct: 949-458-5448 Cell: 949-279-1894 Email: Jmaeding@hunsaker.commailto:Jmaeding@hunsaker.com
3 Hughes Irvine, Ca 92618 Main: 949-583-1010 Website: www.hunsaker.comhttp://www.hunsaker.com/
From: Zoltan Siki [mailto:notifications@github.com] Sent: Monday, September 9, 2019 1:12 PM To: zsiki/bulkvectorexport bulkvectorexport@noreply.github.com Cc: James Maeding JMaeding@hunsaker.com; Author author@noreply.github.com Subject: Re: [zsiki/bulkvectorexport] Multiple Feature geometry not imported (OGR error: Unsupported WKB type 1167) errors (#5)
Dear James,
thanks for your feedback. I'm ready to improve my plug-in. Can you send me a sample dataset to reproduce the error message? What type of geodatabase do you use (ESRI, PostGIS, ...)?
Best regards, Zoltan
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/zsiki/bulkvectorexport/issues/5?email_source=notifications&email_token=AA5Y44JOHFGSHMEBHXYI4UDQI2UYNA5CNFSM4IU6S3FKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6I4AMA#issuecomment-529645616, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AA5Y44LWGTZ2TMAQZ3LHXGDQI2UYNANCNFSM4IU6S3FA.
begin : 2018-12-08
git sha : $Format:%H$
copyright : (C) 2018 by Zoltan Siki
email : siki1958@gmail.com***/
/***
from .resources import *
from .bulkvectorexport_dialog import BulkVectorExportDialog
class BulkVectorExport: """QGIS Plugin Implementation."""
def __init__(self, iface):
"""Constructor.
:param iface: An interface instance that will be passed to this class
which provides the hook by which you can manipulate the QGIS
application at run time.
:type iface: QgsInterface
"""
# Save reference to the QGIS interface
self.iface = iface
# initialize plugin directory
self.plugin_dir = os.path.dirname(__file__)
# initialize locale
locale = QSettings().value('locale/userLocale')[0:2]
locale_path = os.path.join(self.plugin_dir, 'i18n',
'BulkVectorExport_{}.qm'.format(locale))
if os.path.exists(locale_path):
self.translator = QTranslator()
self.translator.load(locale_path)
if qVersion() > '4.3.3':
QCoreApplication.installTranslator(self.translator)
# Create the dialog (after translation) and keep reference
self.dlg = BulkVectorExportDialog()
self.dlg.setModal(True)
# Declare instance attributes
self.actions = []
self.menu = self.tr(u'&bulkvectorexport')
# TODO: We are going to let the user set this up in a future iteration
self.toolbar = self.iface.addToolBar(u'BulkVectorExport')
self.toolbar.setObjectName(u'BulkVectorExport')
# noinspection PyMethodMayBeStatic
def tr(self, message):
"""Get the translation for a string using Qt translation API.
We implement this ourselves since we do not inherit QObject.
:param message: String for translation.
:type message: str, QString
:returns: Translated version of message.
:rtype: QString
"""
# noinspection PyTypeChecker,PyArgumentList,PyCallByClass
return QCoreApplication.translate('BulkVectorExport', message)
def add_action(self, icon_path, text, callback, enabled_flag=True,
add_to_menu=True, add_to_toolbar=True, status_tip=None,
whats_this=None, parent=None):
"""Add a toolbar icon to the toolbar.
:param icon_path: Path to the icon for this action. Can be a resource
path (e.g. ':/plugins/foo/bar.png') or a normal file system path.
:type icon_path: str
:param text: Text that should be shown in menu items for this action.
:type text: str
:param callback: Function to be called when the action is triggered.
:type callback: function
:param enabled_flag: A flag indicating if the action should be enabled
by default. Defaults to True.
:type enabled_flag: bool
:param add_to_menu: Flag indicating whether the action should also
be added to the menu. Defaults to True.
:type add_to_menu: bool
:param add_to_toolbar: Flag indicating whether the action should also
be added to the toolbar. Defaults to True.
:type add_to_toolbar: bool
:param status_tip: Optional text to show in a popup when mouse pointer
hovers over the action.
:type status_tip: str
:param parent: Parent widget for the new action. Defaults None.
:type parent: QWidget
:param whats_this: Optional text to show in the status bar when the
mouse pointer hovers over the action.
:returns: The action that was created. Note that the action is also
added to self.actions list.
:rtype: QAction
"""
icon = QIcon(icon_path)
action = QAction(icon, text, parent)
action.triggered.connect(callback)
action.setEnabled(enabled_flag)
if status_tip is not None:
action.setStatusTip(status_tip)
if whats_this is not None:
action.setWhatsThis(whats_this)
if add_to_toolbar:
self.toolbar.addAction(action)
if add_to_menu:
self.iface.addPluginToVectorMenu(
self.menu,
action)
self.actions.append(action)
return action
def initGui(self):
"""Create the menu entries and toolbar icons inside the QGIS GUI."""
icon_path = ':/plugins/bulkvectorexport/icon.png'
self.add_action(
icon_path,
text=self.tr(u'Bulk Vector Export'),
callback=self.run,
parent=self.iface.mainWindow())
def unload(self):
"""Removes the plugin menu item and icon from QGIS GUI."""
for action in self.actions:
self.iface.removePluginVectorMenu(
self.tr(u'&bulkvectorexport'),
action)
self.iface.removeToolBarIcon(action)
# remove the toolbar
del self.toolbar
def run(self):
"""Run method that performs all the real work"""
# show the dialog
self.dlg.show()
# Run the dialog event loop
result = self.dlg.exec_()
# See if OK was pressed
if result:
# get directry name
dirName = self.dlg.dirEdit.text().strip()
# get ogr driver name
ogr_driver_name = self.dlg.formatBox.currentText()
exportOnlySelected = self.dlg.onlySelected.isChecked()
print('Selected: ', exportOnlySelected)
sldExport = self.dlg.sldExport.isChecked()
qlrExport = self.dlg.qlrExport.isChecked()
crs = QgsProject.instance().crs()
for layer in self.iface.mapCanvas().layers():
if layer.type() == QgsMapLayer.VectorLayer:
print('Writing:' + layer.name())
layer_filename = os.path.join(dirName, layer.name())
if self.dlg.layerCrsButton.isChecked():
crs = layer.crs()
print('CRS selected: ', crs.description())
#export using library function
message = mmqgis_attribute_export_all(layer, layer_filename, ",", "\r\n", ".")
if message != None:
QMessageBox.critical(self.iface.mainWindow(), "Attribute Export", message)
else:
if sldExport:
# export SLD if layer was exported with success
layer.saveSldStyle(layer_filename + '.sld')
print('SLD: ' + layer_filename)
if qlrExport:
# export QGIS layer definition (qlr)
node = QgsProject.instance().layerTreeRoot().findLayer(layer.id())
QgsLayerDefinition.exportLayerDefinition(
layer_filename, [node])
print('QLR: ' + layer_filename)
# Thijs Brentjens (https://github.com/thijsbrentjens/)
# add option for exporting only selected featuresdef mmqgis_attribute_export_all(input_layer, output_csv_name, \ field_delimiter = ",", line_terminator = "\n", decimal_mark = ".", \ status_callback = None):
# Error checks
if (not input_layer) or (input_layer.type() != QgsMapLayer.VectorLayer) or (input_layer.featureCount() <= 0):
return "Invalid layer"
# CSV Options
layer_options = []
if line_terminator == "\r\n":
layer_options.append("LINEFORMAT=CRLF")
else:
layer_options.append("LINEFORMAT=LF")
if field_delimiter == ";":
layer_options.append("SEPARATOR=SEMICOLON")
elif field_delimiter == "\t":
layer_options.append("SEPARATOR=TAB")
elif field_delimiter == " ":
layer_options.append("SEPARATOR=SPACE")
else:
layer_options.append("SEPARATOR=COMMA")
if not decimal_mark:
decimal_mark = "."
# Build field list
fields = QgsFields()
attribute_indices = []
for index, field in enumerate(input_layer.fields()):
fields.append(field)
attribute_indices.append(index)
# Create file writer
outfile = QgsVectorFileWriter(output_csv_name, "utf-8", fields, \
QgsWkbTypes.Unknown, driverName = "CSV", layerOptions = layer_options)
if (outfile.hasError() != QgsVectorFileWriter.NoError):
return "Failure creating output file: " + str(outfile.errorMessage())
# Iterate through each feature in the source layer
for index, feature in enumerate(input_layer.getFeatures()):
if ((index % 50) == 0) and status_callback:
if status_callback(100 * index / input_layer.featureCount(), \
"Exporting " + str(index) + " of " + str(input_layer.featureCount())):
return "Export attributes cancelled on feature " + str(index)
attributes = []
for x in attribute_indices:
attributes.append(feature.attributes()[x])
newfeature = QgsFeature()
newfeature.setAttributes(attributes)
outfile.addFeature(newfeature)
del outfile
if status_callback:
status_callback(100, str(input_layer.featureCount()) + " records exported")
return NoneI found my last code to run quite slow, and realized I could add all layer features at once, using outfile.addFeatures(input_layer.getFeatures()) and outfile.addFeatures(input_layer.selectedFeatures()) That ran as fast as your original way.
I keep thinking we could add a param to the writeAsVectorFormat method you used like this: result2 = QgsVectorFileWriter.writeAsVectorFormat(layer, layer_filename, layer.dataProvider().encoding(), crs, ogr_driver_name, exportOnlySelected, overrideGeometryType = QgsWkbTypes.Unknown) but it did not work. I have not gotten my new code to fail yet. I attached the latest. Note that if you set format to geopackage, you still get the errors like original code. I'm sure the api authors could say why in 2 seconds…I have no idea though.
James Maeding Project Engineer Direct: 949-458-5448 Cell: 949-279-1894 Email: Jmaeding@hunsaker.commailto:Jmaeding@hunsaker.com
3 Hughes Irvine, Ca 92618 Main: 949-583-1010 Website: www.hunsaker.comhttp://www.hunsaker.com/
From: Zoltan Siki [mailto:notifications@github.com] Sent: Monday, September 9, 2019 1:12 PM To: zsiki/bulkvectorexport bulkvectorexport@noreply.github.com Cc: James Maeding JMaeding@hunsaker.com; Author author@noreply.github.com Subject: Re: [zsiki/bulkvectorexport] Multiple Feature geometry not imported (OGR error: Unsupported WKB type 1167) errors (#5)
Dear James,
thanks for your feedback. I'm ready to improve my plug-in. Can you send me a sample dataset to reproduce the error message? What type of geodatabase do you use (ESRI, PostGIS, ...)?
Best regards, Zoltan
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/zsiki/bulkvectorexport/issues/5?email_source=notifications&email_token=AA5Y44JOHFGSHMEBHXYI4UDQI2UYNA5CNFSM4IU6S3FKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6I4AMA#issuecomment-529645616, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AA5Y44LWGTZ2TMAQZ3LHXGDQI2UYNANCNFSM4IU6S3FA.
begin : 2018-12-08
git sha : $Format:%H$
copyright : (C) 2018 by Zoltan Siki
email : siki1958@gmail.com***/
/***
from .resources import *
from .bulkvectorexport_dialog import BulkVectorExportDialog
class BulkVectorExport: """QGIS Plugin Implementation."""
def __init__(self, iface):
"""Constructor.
:param iface: An interface instance that will be passed to this class
which provides the hook by which you can manipulate the QGIS
application at run time.
:type iface: QgsInterface
"""
# Save reference to the QGIS interface
self.iface = iface
# initialize plugin directory
self.plugin_dir = os.path.dirname(__file__)
# initialize locale
locale = QSettings().value('locale/userLocale')[0:2]
locale_path = os.path.join(self.plugin_dir, 'i18n',
'BulkVectorExport_{}.qm'.format(locale))
if os.path.exists(locale_path):
self.translator = QTranslator()
self.translator.load(locale_path)
if qVersion() > '4.3.3':
QCoreApplication.installTranslator(self.translator)
# Create the dialog (after translation) and keep reference
self.dlg = BulkVectorExportDialog()
self.dlg.setModal(True)
# Declare instance attributes
self.actions = []
self.menu = self.tr(u'&bulkvectorexport')
# TODO: We are going to let the user set this up in a future iteration
self.toolbar = self.iface.addToolBar(u'BulkVectorExport')
self.toolbar.setObjectName(u'BulkVectorExport')
# noinspection PyMethodMayBeStatic
def tr(self, message):
"""Get the translation for a string using Qt translation API.
We implement this ourselves since we do not inherit QObject.
:param message: String for translation.
:type message: str, QString
:returns: Translated version of message.
:rtype: QString
"""
# noinspection PyTypeChecker,PyArgumentList,PyCallByClass
return QCoreApplication.translate('BulkVectorExport', message)
def add_action(self, icon_path, text, callback, enabled_flag=True,
add_to_menu=True, add_to_toolbar=True, status_tip=None,
whats_this=None, parent=None):
"""Add a toolbar icon to the toolbar.
:param icon_path: Path to the icon for this action. Can be a resource
path (e.g. ':/plugins/foo/bar.png') or a normal file system path.
:type icon_path: str
:param text: Text that should be shown in menu items for this action.
:type text: str
:param callback: Function to be called when the action is triggered.
:type callback: function
:param enabled_flag: A flag indicating if the action should be enabled
by default. Defaults to True.
:type enabled_flag: bool
:param add_to_menu: Flag indicating whether the action should also
be added to the menu. Defaults to True.
:type add_to_menu: bool
:param add_to_toolbar: Flag indicating whether the action should also
be added to the toolbar. Defaults to True.
:type add_to_toolbar: bool
:param status_tip: Optional text to show in a popup when mouse pointer
hovers over the action.
:type status_tip: str
:param parent: Parent widget for the new action. Defaults None.
:type parent: QWidget
:param whats_this: Optional text to show in the status bar when the
mouse pointer hovers over the action.
:returns: The action that was created. Note that the action is also
added to self.actions list.
:rtype: QAction
"""
icon = QIcon(icon_path)
action = QAction(icon, text, parent)
action.triggered.connect(callback)
action.setEnabled(enabled_flag)
if status_tip is not None:
action.setStatusTip(status_tip)
if whats_this is not None:
action.setWhatsThis(whats_this)
if add_to_toolbar:
self.toolbar.addAction(action)
if add_to_menu:
self.iface.addPluginToVectorMenu(
self.menu,
action)
self.actions.append(action)
return action
def initGui(self):
"""Create the menu entries and toolbar icons inside the QGIS GUI."""
icon_path = ':/plugins/bulkvectorexport/icon.png'
self.add_action(
icon_path,
text=self.tr(u'Bulk Vector Export'),
callback=self.run,
parent=self.iface.mainWindow())
def unload(self):
"""Removes the plugin menu item and icon from QGIS GUI."""
for action in self.actions:
self.iface.removePluginVectorMenu(
self.tr(u'&bulkvectorexport'),
action)
self.iface.removeToolBarIcon(action)
# remove the toolbar
del self.toolbar
def run(self):
"""Run method that performs all the real work"""
# show the dialog
self.dlg.show()
# Run the dialog event loop
result = self.dlg.exec_()
# See if OK was pressed
if result:
# get directry name
dirName = self.dlg.dirEdit.text().strip()
# get ogr driver name
ogr_driver_name = self.dlg.formatBox.currentText()
exportOnlySelected = self.dlg.onlySelected.isChecked()
print('Selected: ', exportOnlySelected)
sldExport = self.dlg.sldExport.isChecked()
qlrExport = self.dlg.qlrExport.isChecked()
crs = QgsProject.instance().crs()
for layer in self.iface.mapCanvas().layers():
if layer.type() == QgsMapLayer.VectorLayer:
print('Writing:' + layer.name())
layer_filename = os.path.join(dirName, layer.name())
if self.dlg.layerCrsButton.isChecked():
crs = layer.crs()
print('CRS selected: ', crs.description())
# Thijs Brentjens (https://github.com/thijsbrentjens/)
# add option for exporting only selected features
result2 = QgsVectorFileWriter.writeAsVectorFormat(layer,
layer_filename, layer.dataProvider().encoding(), crs,
ogr_driver_name, exportOnlySelected)
if result2[0]:
QMessageBox.warning(self.dlg, "BulkVectorExport",\
"Failed to export: " + layer.name() + \
" Status: " + str(result2))
#export using library function else:
if sldExport:
# export SLD if layer was exported with success
layer.saveSldStyle(layer_filename + '.sld')
print('SLD: ' + layer_filename)
if qlrExport:
# export QGIS layer definition (qlr)
node = QgsProject.instance().layerTreeRoot().findLayer(layer.id())
QgsLayerDefinition.exportLayerDefinition(
layer_filename, [node])
print('QLR: ' + layer_filename)def mmqgis_attribute_export_all(input_layer, output_csv_name, doSelected, drivername, \ field_delimiter = ",", line_terminator = "\r\n", decimal_mark = ".", \ status_callback = None):
# Error checks
if (not input_layer) or (input_layer.type() != QgsMapLayer.VectorLayer) or (input_layer.featureCount() <= 0):
return "Invalid layer"
# CSV Options
layer_options = []
if line_terminator == "\r\n":
layer_options.append("LINEFORMAT=CRLF")
else:
layer_options.append("LINEFORMAT=LF")
if field_delimiter == ";":
layer_options.append("SEPARATOR=SEMICOLON")
elif field_delimiter == "\t":
layer_options.append("SEPARATOR=TAB")
elif field_delimiter == " ":
layer_options.append("SEPARATOR=SPACE")
else:
layer_options.append("SEPARATOR=COMMA")
if not decimal_mark:
decimal_mark = "."
# Create file writer
outfile = QgsVectorFileWriter(output_csv_name, "utf-8", input_layer.fields(), \
QgsWkbTypes.Unknown, driverName = drivername, layerOptions = layer_options)
if (outfile.hasError() != QgsVectorFileWriter.NoError):
return "Failure creating output file: " + str(outfile.errorMessage())
if doSelected:
outfile.addFeatures(input_layer.selectedFeatures())
else:
outfile.addFeatures(input_layer.getFeatures())
# Iterate through each feature in the source layerdel outfile
if status_callback:
status_callback(100, str(input_layer.featureCount()) + " records exported")
return NoneShoot, caught a typo in last update, use this instead.
James Maeding Project Engineer Direct: 949-458-5448 Cell: 949-279-1894 Email: Jmaeding@hunsaker.commailto:Jmaeding@hunsaker.com
3 Hughes Irvine, Ca 92618 Main: 949-583-1010 Website: www.hunsaker.comhttp://www.hunsaker.com/
From: Zoltan Siki [mailto:notifications@github.com] Sent: Monday, September 9, 2019 1:12 PM To: zsiki/bulkvectorexport bulkvectorexport@noreply.github.com Cc: James Maeding JMaeding@hunsaker.com; Author author@noreply.github.com Subject: Re: [zsiki/bulkvectorexport] Multiple Feature geometry not imported (OGR error: Unsupported WKB type 1167) errors (#5)
Dear James,
thanks for your feedback. I'm ready to improve my plug-in. Can you send me a sample dataset to reproduce the error message? What type of geodatabase do you use (ESRI, PostGIS, ...)?
Best regards, Zoltan
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/zsiki/bulkvectorexport/issues/5?email_source=notifications&email_token=AA5Y44JOHFGSHMEBHXYI4UDQI2UYNA5CNFSM4IU6S3FKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6I4AMA#issuecomment-529645616, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AA5Y44LWGTZ2TMAQZ3LHXGDQI2UYNANCNFSM4IU6S3FA.
begin : 2018-12-08
git sha : $Format:%H$
copyright : (C) 2018 by Zoltan Siki
email : siki1958@gmail.com***/
/***
from .resources import *
from .bulkvectorexport_dialog import BulkVectorExportDialog
class BulkVectorExport: """QGIS Plugin Implementation."""
def __init__(self, iface):
"""Constructor.
:param iface: An interface instance that will be passed to this class
which provides the hook by which you can manipulate the QGIS
application at run time.
:type iface: QgsInterface
"""
# Save reference to the QGIS interface
self.iface = iface
# initialize plugin directory
self.plugin_dir = os.path.dirname(__file__)
# initialize locale
locale = QSettings().value('locale/userLocale')[0:2]
locale_path = os.path.join(self.plugin_dir, 'i18n',
'BulkVectorExport_{}.qm'.format(locale))
if os.path.exists(locale_path):
self.translator = QTranslator()
self.translator.load(locale_path)
if qVersion() > '4.3.3':
QCoreApplication.installTranslator(self.translator)
# Create the dialog (after translation) and keep reference
self.dlg = BulkVectorExportDialog()
self.dlg.setModal(True)
# Declare instance attributes
self.actions = []
self.menu = self.tr(u'&bulkvectorexport')
# TODO: We are going to let the user set this up in a future iteration
self.toolbar = self.iface.addToolBar(u'BulkVectorExport')
self.toolbar.setObjectName(u'BulkVectorExport')
# noinspection PyMethodMayBeStatic
def tr(self, message):
"""Get the translation for a string using Qt translation API.
We implement this ourselves since we do not inherit QObject.
:param message: String for translation.
:type message: str, QString
:returns: Translated version of message.
:rtype: QString
"""
# noinspection PyTypeChecker,PyArgumentList,PyCallByClass
return QCoreApplication.translate('BulkVectorExport', message)
def add_action(self, icon_path, text, callback, enabled_flag=True,
add_to_menu=True, add_to_toolbar=True, status_tip=None,
whats_this=None, parent=None):
"""Add a toolbar icon to the toolbar.
:param icon_path: Path to the icon for this action. Can be a resource
path (e.g. ':/plugins/foo/bar.png') or a normal file system path.
:type icon_path: str
:param text: Text that should be shown in menu items for this action.
:type text: str
:param callback: Function to be called when the action is triggered.
:type callback: function
:param enabled_flag: A flag indicating if the action should be enabled
by default. Defaults to True.
:type enabled_flag: bool
:param add_to_menu: Flag indicating whether the action should also
be added to the menu. Defaults to True.
:type add_to_menu: bool
:param add_to_toolbar: Flag indicating whether the action should also
be added to the toolbar. Defaults to True.
:type add_to_toolbar: bool
:param status_tip: Optional text to show in a popup when mouse pointer
hovers over the action.
:type status_tip: str
:param parent: Parent widget for the new action. Defaults None.
:type parent: QWidget
:param whats_this: Optional text to show in the status bar when the
mouse pointer hovers over the action.
:returns: The action that was created. Note that the action is also
added to self.actions list.
:rtype: QAction
"""
icon = QIcon(icon_path)
action = QAction(icon, text, parent)
action.triggered.connect(callback)
action.setEnabled(enabled_flag)
if status_tip is not None:
action.setStatusTip(status_tip)
if whats_this is not None:
action.setWhatsThis(whats_this)
if add_to_toolbar:
self.toolbar.addAction(action)
if add_to_menu:
self.iface.addPluginToVectorMenu(
self.menu,
action)
self.actions.append(action)
return action
def initGui(self):
"""Create the menu entries and toolbar icons inside the QGIS GUI."""
icon_path = ':/plugins/bulkvectorexport/icon.png'
self.add_action(
icon_path,
text=self.tr(u'Bulk Vector Export'),
callback=self.run,
parent=self.iface.mainWindow())
def unload(self):
"""Removes the plugin menu item and icon from QGIS GUI."""
for action in self.actions:
self.iface.removePluginVectorMenu(
self.tr(u'&bulkvectorexport'),
action)
self.iface.removeToolBarIcon(action)
# remove the toolbar
del self.toolbar
def run(self):
"""Run method that performs all the real work"""
# show the dialog
self.dlg.show()
# Run the dialog event loop
result = self.dlg.exec_()
# See if OK was pressed
if result:
# get directry name
dirName = self.dlg.dirEdit.text().strip()
# get ogr driver name
ogr_driver_name = self.dlg.formatBox.currentText()
exportOnlySelected = self.dlg.onlySelected.isChecked()
print('Selected: ', exportOnlySelected)
sldExport = self.dlg.sldExport.isChecked()
qlrExport = self.dlg.qlrExport.isChecked()
crs = QgsProject.instance().crs()
for layer in self.iface.mapCanvas().layers():
if layer.type() == QgsMapLayer.VectorLayer:
print('Writing:' + layer.name())
layer_filename = os.path.join(dirName, layer.name())
if self.dlg.layerCrsButton.isChecked():
crs = layer.crs()
print('CRS selected: ', crs.description())
# Thijs Brentjens (https://github.com/thijsbrentjens/)
# add option for exporting only selected features #export using library function
message = mmqgis_attribute_export_all(layer, layer_filename, exportOnlySelected, ogr_driver_name, ",", "\r\n", ".")
if message != None:
QMessageBox.critical(self.iface.mainWindow(), "Attribute Export", message)
else:
if sldExport:
# export SLD if layer was exported with success
layer.saveSldStyle(layer_filename + '.sld')
print('SLD: ' + layer_filename)
if qlrExport:
# export QGIS layer definition (qlr)
node = QgsProject.instance().layerTreeRoot().findLayer(layer.id())
QgsLayerDefinition.exportLayerDefinition(
layer_filename, [node])
print('QLR: ' + layer_filename)def mmqgis_attribute_export_all(input_layer, output_csv_name, doSelected, drivername, \ field_delimiter = ",", line_terminator = "\r\n", decimal_mark = ".", \ status_callback = None):
# Error checks
if (not input_layer) or (input_layer.type() != QgsMapLayer.VectorLayer) or (input_layer.featureCount() <= 0):
return "Invalid layer"
# CSV Options
layer_options = []
if line_terminator == "\r\n":
layer_options.append("LINEFORMAT=CRLF")
else:
layer_options.append("LINEFORMAT=LF")
if field_delimiter == ";":
layer_options.append("SEPARATOR=SEMICOLON")
elif field_delimiter == "\t":
layer_options.append("SEPARATOR=TAB")
elif field_delimiter == " ":
layer_options.append("SEPARATOR=SPACE")
else:
layer_options.append("SEPARATOR=COMMA")
if not decimal_mark:
decimal_mark = "."
# Create file writer
outfile = QgsVectorFileWriter(output_csv_name, "utf-8", input_layer.fields(), \
QgsWkbTypes.Unknown, driverName = drivername, layerOptions = layer_options)
if (outfile.hasError() != QgsVectorFileWriter.NoError):
return "Failure creating output file: " + str(outfile.errorMessage())
if doSelected:
outfile.addFeatures(input_layer.selectedFeatures())
else:
outfile.addFeatures(input_layer.getFeatures())
# Iterate through each feature in the source layerdel outfile
if status_callback:
status_callback(100, str(input_layer.featureCount()) + " records exported")
return NoneDear James, I can't see your attached sample data. Could you send it tome by email? Your code looks useful for csv export only. It is not good for other formats which are supported by bulkvectorexport plug-in :(
jmaeding
commented
5 years ago Oh, then use this link to grab the gdb: https://dn.hunsaker.com/?linkid=KZi4zr6VWWXk/lxbtNw+kJRtHku9vGai/4DODS6ncAGbVD1ezlNyYA
I think I realized what is going on. I tried choosing the other formats like dxf and shp with my code, and I get the Unsupported Wkb errors. Its because the add method avoids projecting or converting anything if I tell it type.unknown and the driver is csv.
The moment I say to do dxf or shp, it gives the errors, as the mechanism to do that must load the items into ogr containers. I seem to have found a very limiting loophole for dealing with my data, that does not work for too many other things as it avoids the target coord system param. Its more code too, which is more maintenance and so on. I don't think I would change anything yet if I were you. This is more a case of cleaning the incoming data so I need to run down what is so special about a few of the polygons in the gdb. I did not make the gdb btw. I noticed it has many polygons of area less than 0.01 sq ft so that may be playing in here. thx
James Maeding Project Engineer Direct: 949-458-5448 Cell: 949-279-1894 Email: Jmaeding@hunsaker.commailto:Jmaeding@hunsaker.com
3 Hughes Irvine, Ca 92618 Main: 949-583-1010 Website: www.hunsaker.comhttp://www.hunsaker.com/
From: Zoltan Siki [mailto:notifications@github.com] Sent: Tuesday, September 10, 2019 6:38 AM To: zsiki/bulkvectorexport bulkvectorexport@noreply.github.com Cc: James Maeding JMaeding@hunsaker.com; Author author@noreply.github.com Subject: Re: [zsiki/bulkvectorexport] Multiple Feature geometry not imported (OGR error: Unsupported WKB type 1167) errors (#5)
Dear James, I can't see your attached sample data. Could you send it tome by email? Your code looks useful for csv export only. It is not good for other formats which are supported by bulkvectorexport plug-in :(
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/zsiki/bulkvectorexport/issues/5?email_source=notifications&email_token=AA5Y44NUSS3HBNVISKTN6XTQI6PMLA5CNFSM4IU6S3FKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6LDU4A#issuecomment-529939056, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AA5Y44IH7WK6BBPMIZ2HSETQI6PMLANCNFSM4IU6S3FA.
jmaeding
commented
5 years ago Hi Zoltan, I spent some time running down what items were causing the errors. It is "multipart polygons". I have no idea how those were made, as most were not multipart. The solution was to run Explode Multipart to single part in ArcMap, as QGIS cannot edit gdb - or at least I have not figured out how to make it. The exploded data worked perfect, so it was just one issue. I bet this will not happen with other formats, like geopackage, as those are already in OGR compatible form. So, of course, I had the special gdb data format, and the problem polygons at the same time, go figure… thx
James Maeding Project Engineer Direct: 949-458-5448 Cell: 949-279-1894 Email: Jmaeding@hunsaker.commailto:Jmaeding@hunsaker.com
3 Hughes Irvine, Ca 92618 Main: 949-583-1010 Website: www.hunsaker.comhttp://www.hunsaker.com/
From: Zoltan Siki [mailto:notifications@github.com] Sent: Tuesday, September 10, 2019 6:38 AM To: zsiki/bulkvectorexport bulkvectorexport@noreply.github.com Cc: James Maeding JMaeding@hunsaker.com; Author author@noreply.github.com Subject: Re: [zsiki/bulkvectorexport] Multiple Feature geometry not imported (OGR error: Unsupported WKB type 1167) errors (#5)
Dear James, I can't see your attached sample data. Could you send it tome by email? Your code looks useful for csv export only. It is not good for other formats which are supported by bulkvectorexport plug-in :(
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/zsiki/bulkvectorexport/issues/5?email_source=notifications&email_token=AA5Y44NUSS3HBNVISKTN6XTQI6PMLA5CNFSM4IU6S3FKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6LDU4A#issuecomment-529939056, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AA5Y44IH7WK6BBPMIZ2HSETQI6PMLANCNFSM4IU6S3FA.
jmaeding
commented
5 years ago Another update…I uninstalled qgis and reinstalled from OSGeo4W64 so I could get the FileGDB driver which has gdb edit ability. When I ran on the original data with multipart polygons, I get no errors. So now I can export the data without cleaning, and I can edit in QGIS. Whoo-hooo! I can likely drop my Arcgis desktop advanced seat now and save thousands every year. I think I earned my salary for the day, time to take a nap.
James Maeding Project Engineer Direct: 949-458-5448 Cell: 949-279-1894 Email: Jmaeding@hunsaker.commailto:Jmaeding@hunsaker.com
3 Hughes Irvine, Ca 92618 Main: 949-583-1010 Website: www.hunsaker.comhttp://www.hunsaker.com/
From: Zoltan Siki [mailto:notifications@github.com] Sent: Monday, September 9, 2019 1:12 PM To: zsiki/bulkvectorexport bulkvectorexport@noreply.github.com Cc: James Maeding JMaeding@hunsaker.com; Author author@noreply.github.com Subject: Re: [zsiki/bulkvectorexport] Multiple Feature geometry not imported (OGR error: Unsupported WKB type 1167) errors (#5)
Dear James,
thanks for your feedback. I'm ready to improve my plug-in. Can you send me a sample dataset to reproduce the error message? What type of geodatabase do you use (ESRI, PostGIS, ...)?
Best regards, Zoltan
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/zsiki/bulkvectorexport/issues/5?email_source=notifications&email_token=AA5Y44JOHFGSHMEBHXYI4UDQI2UYNA5CNFSM4IU6S3FKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6I4AMA#issuecomment-529645616, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AA5Y44LWGTZ2TMAQZ3LHXGDQI2UYNANCNFSM4IU6S3FA.
Hi there, I have a polygon vector source coming from a geodatabase, that is giving lots of errors saying "Feature geometry not imported (OGR error: Unsupported WKB type 1167)". It skips those items. Its not just type 1167, it has 0, 1182, 217191512...and some other numbers. I looked at the soruce and see you are using the QgsVectorFileWriter.writeAsVectorFormat method. So I know that is coming from c++ and even looked at the QgsVectorFileWriter code. I could not figure out what WKB type 0 and the others are though. They must be Ogr types to be supported. Now I can export the layer to csv one at a time using normal qgis export command. That does not have a problem with any of the polygons and I get a full output listing in the csv. I need to export 30 tables over and over though so that is tedious. My thought is to write my own iterator to write to the csv. You may want to do that to make your plugin more robust, or you may prefer to leave it as is. Its a fairly unique tool though so thanks for talking the time to create and share it.