.INP cannot be created from .shp files (Points and lines problem)

[IDoytchinov]

Dear all, I'm experiencing a problem while trying to use QGISRed for some Machine Learning related application study. I'm trying to create a custom .inp network for analysis. Unfortunately, it does not work - the function in QGISRed does not extract the correct points/lines that I provide as shapefiles. As a result instead of having the correct .inp I have wired scares results:

What I get:

How my .inp net should look like:

I'm not sure if the problem is related to the pipe file (Split_from_merged_5m_Erronorous.shp) that I created from a Merged.shp file from the Pipes of the original network/ .inp.

I have applied all the shapefiles, the QGISRed project and the .inp of my project here: https://drive.google.com/drive/folders/1mY_5nnLIHEiU1OrMTwzcK_B5SSo-DXya?usp=sharing

If anyone has any idea how to fix this problem will be really grateful!

Iordan Doytchinov, Ph.D.

Postdoctoral researcher and scientific collaborator

Ecole Polytechnique Fédérale de Lausanne (EPFL) EPFL – TOPO Station 18 – Bâtiment GC C2 398

CH–1015 Lausanne

Office telephone: +41216939832 Personal mobile: +33699850592

iordan.doytchinov@epfl.ch http://topo.epfl.ch/

[neslerel]

Hi @IDoytchinov ,

I have no problems generating the INP file. Please note that only the layers you can see in the legend of the following image are considered for building the INP file. Epanet:

[IDoytchinov]

Hello @neslerel, unfortunately, this is not the correct .inp! I get the same erroneous inp with wrong nodes and missing pipes! If you have a look here at the .shp files for the nodes and pipes, the inp should have this dense shape as shown below. And as you as well showed the .inp creator does not manage for some reason and gives this space results that you showed. Any ideas why this might be?

[neslerel]

Yes, as I told you before, QGISRed only use a specific shps files (not all the layers opened in QGis) to build the inp file. The shapefiles used are: Network_Reservoirs Network_Tanks Network_Pumps Network_Valves Network_Junctions Network_Pipes

In this sense, if the nodes you shown in the previous image you want to include in the INP, these elements should be included in Network_Junctions. You can create another QGISRed project and import elements from your personal shp files.

[IDoytchinov]

Hello @neslerel . Thanks for noting. In fact, this is what I'm doing. I'm importing the .shp files within QGISRed in a new project via the Add data via shapefiles function:

The Network QGISRed shapefile structure is created but not in an expected manner (as a function of the .shp nodes and split pipes but in some other random manner). Not sure if there is something wrong with my .shp files in the first place or if there is another bug that could be causing this? :/

All the best,

Iordan D.

[neslerel]

Ok! Please, when you select the fields to import, do not select the Id Field in pipes and also in junctions layers: Then, you obtain: It occurrs because there are some ids duplicated and in the importation process the repeated ids are removed... I noticed that in your random points layer there are 443 points. However, the total junctions needed for this shape of pipes is 680. The difference are created automatically.

Try to do this change and I hope you can obtain a good INP file ;)

[IDoytchinov]

Thanks a lot @neslerel ! I finally got to this stage. However, that additional new junction created to ruin the whole model we want to create : ( ( We specifically wanted to have a minimum distance of 5 m between nodes around our geometry (As otherwise, the transient simulation timestep we do blows out of proportion as it is linked to the wave-speed and pipe-length). Do you think there would be a way to create the .inp in such a way that we can still have the minimum 5 m distance between the nodes? Not sure if there is another way than just spending a couple of days manually simplifying/editing the Networks? : /...

[neslerel]

Sorry, I tried to find some solution on Google, but I didn't found anything to fix it. The QGis tool split lines by a maximum length. However, you want to split by minimum length, in order you have segements of 5m and only some segments betwen 5-10 m, is it?

[IDoytchinov]

Yes, @neslerel exactly. We want to keep the topology of the network (pipe, diameters, constants etc) however we wish to descriptive the network as you say on many nodes with a minimum pipe length of 5 m maximum 10 m. If it is less than 5 m the transient time constant starts to become quite small... This is why we wanted to create a simplified representative network with nodes minimum length of 5 m and no less...

[neslerel]

Hi @IDoytchinov, did you get any solution for your problem?

Could we close this issue?

[IDoytchinov]

Hello! One of my colleagues found a solution now I'm awaiting his feedback on the steps in order to update the git! I will update it next week! best, Iordan

---

From: neslerel @.***> Sent: 08 November 2021 16:01:27 To: neslerel/QGISRed Cc: Doytchinov Iordan; Mention Subject: Re: [neslerel/QGISRed] .INP cannot be created from .shp files (Points and lines problem) (#53)

Hi @IDoytchinovhttps://github.com/IDoytchinov, did you get any solution for your problem?

Could we close this issue?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/neslerel/QGISRed/issues/53#issuecomment-963244781, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ATIZSIDMB7WSOF3JLUNH6ADUK7Q4NANCNFSM45UN2MPQ. Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[IDoytchinov]

objective: create a simplified EPANET network from shapefile with nodes every 5m.

Cleaning the shapefile Make sure that pipes that should not be connected together are not overlapping or very close to each other. If it is the case, toggle the editing on for the layer, and use the vertex editing tool to move them apart. this will prevent unwanted connections when running the next tool. Use the v.clean tool from the GRASS toolbox. Set the cleaning tools to snap, break, rmdupl, and rmsa with threshold values of 0, 5, 0, 0, and run it. The first cleaning tool snap all lines with a length less than 5m while the other tools will topologically clean the layer by removing lines with zero length, duplicates, small angles... The file contains multiple successive pipes having the same diameter. those pipes need to be merged together and then split again at 5m intervals. to merge the pipes use the Dissolve tool in the vector geometry toolbox and set the dissolve fields to Diameter. this will merge all pipes of the same diameter. The next step is to split the pipes that are not connected to each other. To do so, go to Vector, Geometry Tools, Multiparts to Singleparts and run the tool on the dissolved shapefile. the following shapefile is obtained (colour representing pipes of different diameters).

Generating the nodes Use the Extract specific vertices tool from the Vector geometry toolbox and set the vertex indices to 0, -1. this will generate a shapefile with points at the extremities of the pipes. However, this file contains duplicates. to remove them, run the Delete duplicate geometries tool on the point file to get rid of the unwanted points. then use the split lines at points tool and give it the pipe file as lines and the point file as split features. Use the Points along geometry tool to generate points along the lines. give the tool a distance of 5m and start and end offsets of 5m. merge the two-point files (the results of points along geometry and delete duplicate geometries) using merge vector layers. use split lines at points to split the pipes at the newly created nodes. Right click on the created layer and open the attribute table. Open the field calculator (Ctrl + I) and create a new field with the following name: PIPE_ID. in the expression textbox, write the following: @row_number. This will generate a unique ID for each pipe section. Load a DSM of the area and use sample raster values to get the elevation of the nodes. Similarly to the pipes generate unique IDs for the nodes.

Creating the epanet file install the QGISRed plugin. create a new project (arrow next to first icon). Click on the arrow next to the second icon and add data. go to the pipes tab and add the pipe shapefile with the correct settings (note: set the length to none). do the same for the junctions (make sure that the elevations are set correctly). If shapefiles with the valves, pumps, reservoirs, or tanks are available add them in their respective tabs. Then click on the arrow next to the second icon and click on export to epanet. A few pipes might be lost in the export and will have to be added manually in epanet