zskarte / zskarte-client

Zivilschutz-Karte allows to draw situation maps for disaster management
https://www.zskarte.ch/
MIT License
13 stars 8 forks source link

feat(csv layer): add option for layers generated from csv files #426

Open swerder opened 1 month ago

swerder commented 1 month ago

Add an option to directly load csv data with coordinates as layer with point features. Also allowing filtering the the data from csv before internally create the features.

As this change is based on the geojson logic and settings you can also activate search for the generated features here.

This function was designed to have a possibility for an offline search for places, and use the "swissNAMES3D_PLY.csv" from https://www.swisstopo.admin.ch/en/landscape-model-swissnames3d (with filtering for "OBJEKTKLASSE_TLM" = "TLM_SIEDLUNGSNAME")

It is based on geojson update https://github.com/zskarte/zskarte-client/pull/417

some words to CSV parsing

Most used CSV parser (also @fast-csv/parse, was already in source) are for node runtime and would need pollyfill for node-stream to work in web. Therefore I add uDSV. It's not as much used, but is small and work as expected on my tests. If you planned to add stream pollyfill anyway. The parse logic in GeoJSONService.fetchCsvData could be changed to:

import { parse, Row } from '@fast-csv/parse';

        const csvLines: Row[] = [];
        const stream = parse({ headers: true, ignoreEmpty: true })
          .transform((data) => ({
            ...data,
            [feature.fieldX]: parseFloat(data[feature.fieldX]),
            [feature.fieldY]: parseFloat(data[feature.fieldY]),
          }))
          .on('error', (error) => console.error(error))
          .on('data', (row) => {
            if (isNaN(row[feature.fieldX]) || isNaN(row[feature.fieldY]) || (row[feature.fieldX] === 0 && row[feature.fieldY] === 0)) {
              return;
            }

            if (regexPatterns?.length) {
              const matches = regexPatterns
                .map((pattern) => row[pattern.field].match(pattern.regex))
                .filter((match) => match) as RegExpMatchArray[];
              if (!matches || matches.length !== regexPatterns.length) {
                return;
              }
            }
            csvLines.push(row);
          })
          .on('end', (rowCount: number) => console.log(`Parsed ${rowCount} rows`));
        stream.write(csvContent);
        stream.end();