Set origin (displace) of the 5km grid?

[mnhn-paul]

Hello, I hope this is the right place to post my issue. I only yesterday discovered this plugin (Biological-Recording-Tools) and immediately loved it, as it does things that I was trying to figure out "manually" for some time. I mainly want to do species distribution atlas maps for Luxembourg, with a 5km grid. As Luxembourg is, compared to other country relatively small we usually "displace" the origin of our 5km grid to better cover the surface of Luxembourg. We do for example place a grid cell on 53.000E 54.000N etc. Now your plugin places the 5km grid automatically on 55.000E 55.000N etc. If only using the 1km grid this is not a problem, however choosing any other bigger grid size in the plugin, this does not work for Luxembourg. I tried to illustrate this "problem" in the attached map.

Is there any way of changing the origin of the 5km grid cells? Or any chance that this feature will be added in a future version of the plugin. If I'm able to solve this small "problem" I would know a series of people whom I could recommend this plugin to here in Luxembourg.

Hope someone can help Kind regards Paul

Showing the grid used typically by us (black lines) and the grid cells placed by the plugin (blue)

[burkmarr]

Hello Paul,

That's fascinating and a beautiful map to illustrate the problem. Without looking at the code my first thoughts are that this could be quite straight-forward to implement and wouldn't require complicating the interface. If it can be done simply, I will enable you to specify the offsets in Plugins->Tombio Tools->Environment options. I'll probably have a look at this either tomorrow or over the weekend. I am hoping to make a release to fix a bug in handling species names with special characters, so I can include this enhancement with that.

One question, what CRS are you using?

Rich

[mnhn-paul]

Hi Rich,

Thanks for your quick response! I would appreciate this a lot if it would work.

The CRS that I'm mainly using is: Luxembourg 1930 / Gauss EPSG:2169 (was used to make the map above, so it works with your plugin) Luxembourg 1929 IGNF:LUXGAUSSK

I did set the grid size manually (not the predefined 5km squares etc.) to 5000 map units.

hope to hear from you soon Best wishes Paul

[burkmarr]

Hello Paul,

I've made an enhancement that addresses your issue. As we discussed, I've added the ability to set offsets for grid maps of biological records. At the same time I also added an enhancement to the tool which creates grids (we call it the OSGR tool) to read the same offsets.

To try out biological record enhancement you will need to replace a single file - projection.py - in your plugin folder which will be something like this if you are using Windows: C:\Users\fsc.qgis2\python\plugins\QGIS-Biological-Recording-Tools (fsc is my username on my computer so yours will be different).

The new version of this file is here: https://raw.githubusercontent.com/burkmarr/QGIS-Biological-Recording-Tools/issue-4-fix/projection.py

If you want to try out the enhancement to the grid generation tool, do the same with this file: https://raw.githubusercontent.com/burkmarr/QGIS-Biological-Recording-Tools/issue-4-fix/osgrLayer.py

You need to save them to the plugin folder, replacing those that are there. In Google Chrome, just enter the above address in your addressbar then right-click your mouse on the page and choose 'Save as'. If Chrome tries to put add a '.txt' extension to the filename, remove it. Once you've done that you will need to restart QGIS if it's already running. Then you should be able to try it out.

Once the file(s) are in your plugin folder, you can apply the offsets by using these environment variables for the plugin:

biorec.xGridOffset: 3000 (you could use -2000 instead here if you wanted - makes no difference) biorec.yGridOffset: 4000 (you could use -1000 instead here if you wanted - makes no difference)

The attached image shows Luxembourg with CRS EPSG:2169 and a 5 km grid which I generated with the enhanced OSGR tool that matches the grid you use I think. I also used it to generate the 500 m and 1000 m grids shown in the bottom left. This is a really handy tool for generating grids on the fly - you might find it useful to be able to do that aligned to the grid you use.

The point layer is 30 points that I generated randomly and the the dark squares were created by the biorec tool gridding them to a user-defined grid of 5000 m with the same offsets applied.

I intend to make a maintenance release soon that will include this enhancement but tit would be good to know first if it works for you. Let me know whether or not you can try this out and, if so, how you get on.

Best wishes,

Rich

[mnhn-paul]

Hi,

This looks promising and great. I will try it as soon as possible, probably only on Monday when I'm back at work, maybe earlier. I will tell you if it works for me or if I have any problems. Have a nice weekend Paul

[burkmarr]

That would be great Paul. Have a good one yourself.

Rich

[mnhn-paul]

I did a quick test at home, seems to work fine :) I'll try with some "real" data on Monday and will upload the result here and give you my impressions, Thanks Paul

[burkmarr]

Great - I'll await the results of that before making a release. (BTW subsequent to providing you with the update I found a problem with the grid tool - OSGB Tool - in that it didn't automatically pick up changes made in the environment file to the offset parameters until the plugin was reloaded. That will be fixed in the proper release - so changes to the offset parameters will be picked up as soon as they are made.)

[mnhn-paul]

Hi,

as promised the first thing in the morning I did was test the changes at work. And of course it works perfectly as you can see. I relay have to thank you a lot. This all went much faster then I initially thought. Thank you! I will transmit this information to other people who might find it usefull.

Have a nice day Paul

[burkmarr]

Okay that's great. I'll include this enhancement in a new release (2.7.0).

Best wishes,

Rich

[krisboers]

Hi Rich I installed version 2.7.4 today, use Belgian coordinate system (EPSG: 31370) and have the same problem. Do I do something wrong, since you mention to include the solution in the 'new release 2.7.0'. (but that was only a few days ago I see - so I suppose that must be 2.8.0) I'm a Mac user, and I can't really find the location of the projection.py-fyle, so I can't try if it works if I do what you explain :-) Kind regards Kris

[burkmarr]

The enhancement to offset grids is included in version 2.7.4 Kris. What are you trying to do exactly?

[krisboers]

When I use the plugin to visualise the point/species, it works fine and the symbols are at the right place when I want km-squares, I have the same problem as Paul had in Luxembourg

The only thin I do different with the youtube-manuals is that I first open the csv-files in Qgis because in Belgium our csv files are ";" separeted, and not "," separated. That's because "," for decimal numbers. But it doesn't give any problems as long was we want to display the exact location instead of the UTM-square

[burkmarr]

Hi Kris. I think that this is a different problem to Paul's - I can see from your illustration that the squares and the grid actually have different projections. You need to make sure that you are using the same CRS when you generate the squares. If you send me a sample point file, I will demonstrate for you.

Rich

[krisboers]

export_40079_2014-03-01_2017-03-27.xlsx

[burkmarr]

Hi Kris. Below are some pictures which I think illustrate what you are after. I'll explain how I produced them and, hopefully, you can work out from there what you're doing wrong.

I sourced a shapefile showing (pretty roughly) the boundary of Belgium. I used the TomBio OSGR tool to create a 5 km grid over this aligned to the Belgian coordinate system (EPSG:31370). To do this I used the OSGR settings shown below. Note particularly that this is a user-defined grid size (all the others are geared to the UK grid). The CRS of the map view (i.e. project CRS) must first be set to EPSG:31370. With the correct user-defined grid size set, I made my boundary of Belgium the active layer and then selected the feature in the map view. Then I clicked the outlined in red at the bottom.

Next I prepared the Excel spreadsheet you sent me by deleting the first row (contained some title text) and then saving as CSV. Then I opened this with the BioRec tool (I called it 'Sample.csv') and used the settings shown below to generate a single 5 km atlas map of all species.

Note that I used a user-defined ataas size again and specified the CRS as EPSG:31370. This produced a map layer shown in the images below.

So it's just the case of using the right matching projections when the grid and the atlas records are created. Paul's problem was different. In Luxembourg the biological records are apparently often aggregated to a grid that is offset from the standard grid representing the projection they work to.

I hope this helps. Let me know if anything is unclear.

[krisboers]

Hi Rich Thanks - very clear But this way, I create a 'new' grid for Belgium/Flanders, that looks like

The grid we usually use, looks like this

They're both 5x5 km

[burkmarr]

Are you saying the two grids are different - it's hard to tell from those maps. If they are, can you create an image that shows the two grids superimposed?

Even better - can you send me a shapefile with the grid that you normally use?

(Also, if you were following Paul post, you may have tried to set offsets in the environment file - but I don't think you need to offset your grid.)

Rich

[krisboers]

yes, the two grids are different for one or another reason, our UTM-raster isn't straight :-) you can find it here: https://we.tl/1EfOVkTNMl

[burkmarr]

Okay Kris I see the problem now. The grid you sent me has a user-defined CRS - it's not in EPSG:31370 as I thought you said it was. I will have to look into this.

[krisboers]

As far as I know, there is no difference between the user-defined CRS and EPSG:31370. Thanks for the effort!

[burkmarr]

Hi Kris. I compared the 'proj4' strings of EPSG:31370 and that user-defined CRS and they are different - so I'm not sure what's going on there.

However I now think that wasn't the real issue. The actual issue is that the 1 km grid that you sent me (shapefile utm1_vl), isn't a grid generated on either of these two CRSs - it's a grid generated from a UTM CRS - I think UTM Zone 31M (EPSG:32361). That's why it appears non-square when projected on EPSG:31370.

So what you actually need to do is deal with X & Y coordinates in your CSV file which are encoded to EPSG:31370, but generate atlas maps from these based on EPSG:32361. In other words the input and ouput CRSs are different.

This images below show the Biological Records Tool set up to do this and the resultant map output. I created a 5 km atlas map and this overlays the 1 km grid you sent me. As you can see they line up. The map view CRS is EPSG:31370 which is why the grid doesn't appear square to the view.

[krisboers]

thanks a lot! (I was almost at the same point)

[burkmarr]

You're welcome.

[IwanLewylle]

Hi Rich,

as a ex-colleague of Kris (see above) I've got a similar question ;-)

I can't seem to produce a 1km atlas (monads) or other similar grids as I get this message "Info: Only points or user-defined grid for input data that are not OSGB (EPSG:27700)"

What I don't get is that the CRS I'm using is WGS 84 - EPSG 4326? I'm guessing that I need a conversion between latitude/longitude and grid references?

greetings Iwan

[burkmarr]

Hi Iwan,

If you want to generate a 1 km grid, you need to do that with reference to some other CRS that WGS 84 since the map units for that CRS are in degrees, not meters. So what CRS do you want the 1 km squares to be referenced to? Are you working on data for the UK or some other area? (It doesn't matter that your data are captured in WGS 84 - that's a separate issue to the CRS you want to generate your aggregated squares for.) It is perfectly doable - I will be able to help you once I know what geographical area you are working in (and therefore what likely CRSs you can use).

Rich