Initial plot importer warning test observations

[andrewvanbreda]

It has been a while since I used the importer and am currently looking into rechecking it so I know what it does as I couldn't remember. This will be available to test on the dev site shortly. I have tested the importer using identical code on my machine, the only real difference is I altered Oli's test data to use the taxa that are available on my own machine. My observations are: The Good:

• Importer is stable (no crashing, exceptions observed)
• I was able to importer all but 700 approx records (there were nearly 7000 in the file so this isn't too bad a result to start with)
• Most of the rejections were due to genuinely invalid spatial references

The Bad

• The changes discussed after this version of the code was released have not been implemented yet. There are still sample groups which is something we were going to get rid of. Also the importer is going to change quite a lot with the occurrence failure detection coming earlier, also another big change is the sample IDs/names are going to be part of the generic importer that plant portal will use, so this bit is not working properly yet, the observations might end up attached to the wrong samples. So these are big changes that aren't available yet.
• I saw the system complaining about the South-Yorkshire location during import, I think this bit should be working, I do not know why it is complaining about that.

Nested in #46

[andrewvanbreda]

I know at least part of the problem with the failing South-east Yorkshire rows is that the importer is expecting them in a separate Vice County column. Can you confirm if you are expecting the vice county to be in a separate column? I think we discussed it would be like that? Also in the case of an invalid spatial reference currently we just flag it and not import the row. However we could just generate a valid one from the vice-county or country, currently the importer attempts to only do that if the spatial reference is missing entirely.

[andrewvanbreda]

Note about the Vice County problem. I have now seen this working on my machine. The issues were

• Vice County Column required - Details Above
• There was a space on the Vice County column header I created that caused the code to not recognise the column. This is now trimmed by the code. I have seen the spatial reference for South-east Yorkshire now working in the sample on my machine. The dev site importer won't be up to date in this regard, but this fix is very very simple->1 line ->so if causes issues with your testing a correction is easy.

[andrewvanbreda]

As agreed, I have removed the Sample Group functionality. This seems to be stable on my machine without too much bother even though it involved removing quite a lot of code. I main observation about this is that is does make the importer far easier to understand, particular from the user point of view. Code is not currently on the BRC dev site but can be made as soon as required as it is committed

[sacrevert]

Regarding the vice-county item. One Dec 3rd under #5 we said the following:

If the spatial reference is missing check the vice county and use this as the grid reference. If the vice-county is missing then check the country and use a grid reference for this as the grid reference. Fine. AVB: Fine

...

I do still have a couple of questions if the above is right:

Do you want the vice county/country information retained, or is it just purely for calculating a missing grid reference and discarded by the importer. OLP: I think retain and store in the database. I think all plots should have a stored VC association. Presumably this will make queries against VC much quicker (e.g. for users interested in plots in their area?) AVB: Fine. The only issue is that if you are wanted to do things with the location I will need to do some coding because this would probably need to be stored as an sample attribute (as the location field is in use by the plot) and I don't think attributes currently have location facilities. Should be possible though. In theory we could still have the grid reference/vice county/country in one column if you prefer, but I think it would greatly aid simplicity if we could still get the user to tell the importer what the field contains, maybe change the options to "corner", "centroid", "vague spatial reference", "vague from vice-county", "vague from country" (or words to that effect). I think it would be problematic, particularly for performance to work out what the user had entered in the grid ref/county/country column manually. Or we can stick with separate columns we originally planned. I think your first idea of checking the columns in sequence until something intelligible is found is better than complicating the labelling of a single column, particularly as this would not work for datasets containing data of mixed accuracy anyway. AVB: Fine

[andrewvanbreda]

Hi, Yes I think what is described here is the way the code works. It is expecting the Vice County/Country to be in a separate column, so those rows in the example import file won't currently import without alteration. Currently the list of Vice Counties/Countries is saved as configuration at both the Drupal and Warehouse end so I don't have to keep going to the database. I have seen the lookup of the grid reference working if the grid reference is missing. This is all good.

The Vice County and Country are also already retained in the code in attributes. I am sure I can write a search where the user code types a country for instance and it looks up samples in that country. However I have just though of a weakness of that approach, if we use the stored country and allow a used to search on it, what happens to rows which don't have a Vice County/Country specified, so I might need to find the vice county/country automatically using the spatial reference only at search time. Or those fields could be auto-filled at import stage. Either way there will be a hit on performance, whether that is a fatal hit I don't know.

[sacrevert]

It would be ideal to have all plots having a stored vice-county and country attribute, irrespective of whether that is provided by the user on import. I can see that there are costs to calculating this on import, or when running a query. One solution might be not to do either, but to fill in the missing attributes occasionally by running a code overnight that calculates the missing attributes and stores them in the relevant field, that way, when a query is run, the information is there. Not sure how realistic this is budget-wise?

[andrewvanbreda]

I like the idea of doing it afterworlds. Perhaps even immediately after the import invisibly in the background. It is hard to say how much it would add to the budget, it would probably involve going to the database which is what I was trying to minimise, so querying the database and then actioning based on that query is quite different to what it does now. I think the truth of the matter is we either do it or don't do it. So we could code it for now not doing it, then as we hit any issues then we think of a solution. I would apprehensive at this start about spending too much time on this issue, when the disadvantages of not including this functionality are slightly unclear. We could end up coding something that is not the optimal solution given the problems we may or may not face in this area. I would say let's keep this in mind for future.

[sacrevert]

OK, fine. I think it is essential in the long term, but I agree that it is better to wait rather than doing it inefficiently now.

[andrewvanbreda]

OK thanks, will keep in mind that it is essential

[DavidRoy]

The existing scheduled task assigns all samples, and occurrences, to a VC. The samples and occurrence cache tables therefore provide the link to VCs. I therefore suggest no specific coding needed for the PlantPortal

[sacrevert]

Can you just clarify what you mean - which scheduled task is this? Do you mean existing Indicia core code? Are the cache tables being built automatically anyway then?

[DavidRoy]

Sorry for being vague. Yes, an automatic task that runs in the background on the Indicia Data Warehouse, every 15 minutes. It does various processing tasks, including an automatic assignment of sample locations to VCs. This leads to an update to the cache tables that are then used for reporting.

This does pose the question of what to base reports on - the user-supplied VC or automatic assignment, which might not always match

[sacrevert]

Presumably they would have to use the automatic assignment, given that one of the main reasons for using the cache tables will be to catch those samples/occurrences where the user-supplied VC is missing.

[andrewvanbreda]

Hi David, Do you know how the vice county link is stored in the database? Is it the location_id? because that could already be in use? I don't see an attribute to hold it.

For reporting I would go with the automatically generated one, it should be more reliable in the case of a clash. If the code is correct the auto-generating one should always be correct. Note we still need the code I have already done to auto-generate a spatial reference as required if it is missing, we just might not need any extra coding beyond that for this (i.e. I don't think any time has been wasted in this area)

[johnvanbreda]

@andrewvanbreda I presume the vice county link is the one created by the spatial index builder. Look in cache_occurrences_functional.location_id_vice_county as a starting point.

[andrewvanbreda]

Ah ok, I think what has happened is I don't have that field on my machine because that particular module is not enabled/installed. I think the importer is already be working correctly in this Vice County/Country area with the code I have already written. In summary: the general Indicia code should generate the long term Vice Country link automatically already, and my code handles an auto-generated grid reference based on what the user has entered for the Vice County/County if the grid reference is missing. The auto-generated one should be used for reporting, but seeing an I am using attributes during the import process, we can also store what the user entered VC/Country for reference.

[andrewvanbreda]

I have configured it on the Plant Portal Dev site, however what has happened is the update to the warehouse has overwritten the config file, I will need to get Biren to reinstate it (I think he is back tomorrow).

If you like you can use the "Normal Importer" on the Plant Portal dev site now to see changes relating to that importer which should be working in full (http://test.brc.ac.uk/plantportal/normal-importer), or you can have a little browse through the screens on the main plant portal importer (http://test.brc.ac.uk/plantportal/import-records-from-file) being aware that you won't be able to click the Import button as the warehouse bit freezes due to the config file being missing. But you can do things like see the re-upload functionality, the warnings screen.

You might see the odd error message on screen, I know of one place this is happening, it only happens if there is a column in the input file that you don't map to a database field on the mappings seen. You probably won't have that in practice much so shouldn't be too much of an issue when demonstrating the site to you. (I noticed actually the site isn't quite running my latest code so I think I might of already patched that issue) It is up to date enough to demonstrate anyway.

[sacrevert]

Thanks @andrewvanbreda - so, I should try and upload one of the files (or similar) that I previously sent to you for testing and development, yes?

[andrewvanbreda]

Hi @sacrevert The actually upload itself won't work until that config file I mentioned on the warehouse is fixed, hopefully tomorrow. However you can still have a look around the screens on the importer before the actual final upload button is clicked. I would maybe just create a file with a couple of lines in it, then you could put a deliberate mistake in it, and try the re-upload facility...perhaps you could leave out a spatial reference, and try it with or without a vice county and see what happens.

When mapping the columns on the mappings screen, the Sample Identifier needs to map to Sample External Key and the plot maps to the Location lookup if I remember off the top of my head. The others should be more obvious. I will change the column names at some point so that Location says Plot instead.

Also there are the enhancements to the normal importer, that should be fully working, so you can upload anything you like with that screen.

Basically Normal Importer = Do what you like with Plant Portal Plot Importer = Have a fiddle with and look around but expect issues for now until that config is fixed at least

[andrewvanbreda]

One further note on this, as a general rule (on any project) I wouldn't do a huge import first, because if it doesn't go how you expect you won't be able to see what happened because there will be too much data,

[andrewvanbreda]

@sacrevert, I will now upload some files that you can use to see different bits of functionality. For now leave the general Spatial Reference System drop-down at the very start of the Wizard OFF as we will supply the ref system on each row separately. You will know the drop-down I am talking about as it has a page of its own.

Take note of my previous note above about mapping the Sample ID to the Sample External Key and the Plot to the Location

I am finding the importer less stable than on my own box, am now sure why that is right now

No to worry for now, I can demonstrate some things to you.

[andrewvanbreda]

This first file is quite a complex situation (even though there are only 3 rows) In the data rows 2 and 3 have the same sample ID (so should go into the same sample). However row 2 is having its spatial reference taken from the Vice County but row 3 is from the country as the vice county is missing. These means their spatial references are different and the system is flagging this as they should be the same for the same sample. It prompts the user to re-upload the file

You will need to change the file extension from .csv to .txt as it isn't letting me upload .csv

[andrewvanbreda]

Test File for Oli 1.txt

OLP response: The logic seems fine, although there are some elements of the system's response that I don't quite follow:

The following rows have been found to have inconsistancies: Daisy,2010,3984,52.62865N 1.46538W,4326,Test plot name,HL10,,England (row number 3)

Where is the system getting the "52.62865N 1.46538" from? This row has no spatial ref at all, so how is it reporting a lat long in this error message? @andrewvanbreda

AVB response: : Perhaps the importer is confusing in this scenario (although it is correct I believe) In the file, rows 2 and 3 have the same sample ID, this obviously means they should go into the same sample and therefore have the same spatial reference. In this case both spatial references are missing but the first one can be collected from the vice county, however the other row has its vice Ccounty missing so the spatial reference is collected from the country. This means that both rows have spatial references that the system has collected, but because one is from the Vice County, and one is from the country they are different and the system is saying they should be the same because the sample id is the same. Perhaps the importer needs rewording. I will raise an issue to double-check error messages are clear. I am not even sure my explanation is clear!

[andrewvanbreda]

Test file 2's situation is simpler but the user is again warned that they need to re-upload the file. Rows 1 and 3 share the same sample id (external key) so should go into the same sample. The user is warned rows for the same sample need to be place on consecutive rows in the file Test File for Oli 2.txt

OLP response: Logic seems fine, but the error message is a bit opaque:

The following rows have been found to have matching sample external keys but are separated from earlier rows with the same external key inside the import file. Rows which you need to go into the same sample should be placed on consecutive rows. If the sample external key is not present for the row, then the row needs to be placed next to other rows that also don't have a sample external key.

I suppose this is just a simple matter of making terminology a bit more user friendly (e.g. sample external key might be better called unique sample identifier). @andrewvanbreda

AVB response: I will raise a separate issue where you can list terminology you would like to change. A lot of the text in the importer is already configurable (although I have a feeling the above message is one of the few places that isn't configurable, although we can easily change the code)

[andrewvanbreda]

File 3's row 3 is missing a spatial reference but also a vice county and country to fill it in with. The user is warned this must be corrected before an upload can continue. Not also on the warnings screen that row 2 has its spatial reference generated by the vice county and the spatial reference system is present even though it was missing in the original file Test File for Oli 3.txt

OLP response: This is fine, although I'm not overly keen on the lat/long centroid(?) appearing in the SPATIAL_REF column. I presume that this is the spatial ref auto-generated by the vice-county? My issue with this is that it seems to me like it may result in the spurious association of the occurrence with a highly precise location (the VC centroid), rather than a proper attribution of the occurrence to a vague area (i.e. anywhere within the VC polygon). I realise this might be a feature of the system, but can you think of downstream issues that might arise through including a precise centroid in the SPATIAL_REF column? Perhaps it is standard and there is nothing to worry about.

Also, the error reporting here is better laid out than for Test Files 1 and 2. Is this something that will be unified? The table layout looks really nice. @andrewvanbreda

AVB response: I will address the unification of error messages first. The answer is they can’t easily be unified, as one screen are warnings specific to Plant Portal, the other screen is now part of the standard importer and the warnings are generated by talking to the warehouse first. This extra warnings screen basically avoids the problem of the warehouse reporting errors but importing other rows OK. Now it won’t import unless there are no errors are all for Plant Portal So the warning types are different and the screens are placed in completely different places in the code, having said that there is no reason whatsoever why you can’t make suggestions on making any of the warnings screens look better.

In response to the spatial references, the sample has to have a spatial reference. It is only a Lat/Long as that is the default I chose for the Vice Counties/Countries. We could make the spatial reference vaguer if you wanted, like a low precision British National Grid reference? The spatial reference doesn’t need to even be the same as what is in the warehouse, I use a separate lookup list (much faster) stored in the page configuration so the spatial references could all be on the moon if you liked. Or we could have the spatial reference in the background and not show the user.

[andrewvanbreda]

File 4 is a simple valid file that would allow import. Note that the user still has the option to re-upload if they wish. Unfortunately until the warehouse config is corrected I can't show you it picking up existing Plot Groups and Plots. I think I needed to do something when committing the code to stop it being overwritten Test File for Oli 4.txt

That will do to get you started anyway

OLP comment: Seems fine, although perhaps an enhancement would be to remove the column mapping option for the spatial reference system if the user has already blanket applied this on the first screen. I could see that the column to map to disappeared, but the sref_system column from the file was still there to be mapped.

I presume the fact that the option to actually continue with the submission is missing is still to do with the config options on the test warehouse, or something like that? @andrewvanbreda

AVB response: I think the system is behaving correctly for the spatial reference system column. If the column hasn't been mapped then the system doesn't know that it is the spatial reference system column, it is just a column in the file that happens to be called spatial reference system. There is logic actually that looks at the column and try to auto-map it, we could try and apply that same logic and instead of auto-mapping hide the column. The only problem is if it gets it wrong and the column is hidden. Personally I think this issue is best left unless I am misunderstanding.

The continue button should of been available though, there should of been some radio buttons which said (something like) "Continue" or "Re-upload". Once continue is selected the appropriate button is displayed. If the warehouse code is wrong I would only expect a failure once the progress bar has appeared....in fact it might appear to work when in the background it hasn't (it might even actually add two of everything if the warehouse code is wrong). Anyway, the continue option should of been there.

[andrewvanbreda]

Hi @sacrevert, I just tried running your original importer file on the dev site, completely untouched so I didn't try to change the formatting. The importer has detected and correctly warned the user of inconsistencies in the file, so it has behaved correctly (so far) as far as I can tell. You can try it yourself. It warns the user of lots of different samples that appear to have the same sample id but the spatial references are different. So there are two possibilities

1. The file is wrong
2. My understanding is wrong.

Either needs to be discussed.

OLP response: The file is wrong. Either I was not thinking when i made it, or I was thinking about something that we have since dropped. Not sure. Shall I amend it? @andrewvanbreda

AVB response: Yes feel free to amend and try again if you want to

[andrewvanbreda]

This is the file HLS_test_data_HL10.txt

[andrewvanbreda]

I should further add that we can have occurrences at different spatial references to the main sample by using sub-samples, that is a built-in indicia function. However I have not tried/tested/coded for plant portal, it would add an extra layer of complexity

[sacrevert]

@andrewvanbreda Just to say that I am going through these tests now.

In relation to the above comment, I still don't see that we need anything as complex as sub-samples at this time. It may be useful in the future for 'nested' sampling (e.g. surveyors record a large plot, say 10 x 10 m, and then a smaller plot that is spatially nested within it (e.g. a 1 x 1 m plot)), but this is not likely to be a frequent situation, and we should not worry about it at this point.

[andrewvanbreda]

@sacrevert yes a simple solution is better but we need a way of overcoming a problem with your test file where we had samples with the same sample ID but different spatial references (if I remember correctly). The importer will warn you during import so you will see it. @BirenRathod has the dev warehouse been switched back to the dev branch? In which case Oli I don't think the importer will work properly for an actual import until the warehouse is switch back to the plant portal branch. Oli you might find that most of your tests that create warnings using the test files I supplied might still work as they don't write to the warehouse, but an actual import of data might fail [OLP: yes, that is what I found]

[BirenRathod]

@andrewvanbreda , yes it has switched to develop branch. last week.

[sacrevert]

@andrewvanbreda

yes a simple solution is better but we need a way of overcoming a problem with your test file where we had samples with the same sample ID but different spatial references (if I remember correctly). The importer will warn you during import so you will see it.

This was my mistake with the test file. Occurrences with the same sample ID should have the same spatial ref, as they should all relate to the same plot. Sorry about that error.

[andrewvanbreda]

Biren has just confirmed the warehouse is not running the Plant Portal branch anymore. So feel free to continue testing, but if it does something you aren't expecting let me know and I will (might) be able to tell you if it is a bug or just the warehouse code branch being wrong

[sacrevert]

I'm probably going to move on to finishing the TVK coding for the NVC data now (lichens were outstanding), so feel free to look at my edited comments relating to the 4 test files above. Regarding further dev work, David's email seems to me to confirm that we can work on the following (from his email):

It sounds like the next steps are:

1. Oli tests the import of PlantAtt
2. Oli/Biren imports the NVC Floristic data into the Dev warehouse. Oli to test the import
3. Biren/Andy ensure the codebase is uptodate for the plot-data importer >(https://github.com/BiologicalRecordsCentre/BSBI-Card-and-PlantPortal/issues/47). Biren - can you let me and Andy know a good time for you to work on this?

My impression was that it was OK to work on these items, but not to commit to additional new development.

[BirenRathod]

I'm working most of days like Tuesday,Wednesday,Thursday in this month. Off may be for long weekends. So no problem on those days.

[andrewvanbreda]

OK that is fine then (in response to Oli's point). I will need to work out where we are with these things I can't remember where we are off the top of my head. Oli have you tried the plot importer yet, if not you might want to get Biren to switch the warehouse branch for you

[sacrevert]

Do you mean the 'normal importer'? http://test.brc.ac.uk/plantportal/normal-importer I'm not sure I recall exactly what I should be testing here, or why I am testing it. Under what circumstances would this be used for Plant Portal imports? (i.e. I understand that the normal importer has been enhanced by the addition of code developed for the plant portal, but, given that I didn't use the normal importer before the change, I'm not sure whether you are asking me to test this for (i) general errors or (ii) specifically for people using it for the PP project. If (ii), then refer back to my earlier question of when this would be used over the PP specific importer. Apologies for being dense.

[andrewvanbreda]

No, the importer you are interested in is this one http://test.brc.ac.uk/plantportal/import-records-from-file

The "normal importer" has been placed on the site so anyone who is interest (You/John/David) can see how the Plant Portal importer affects the standard Indicia importer. It seems sensible to place this page on the site because that is the only place that is up to date with the Plant Portal code so I couldn't put it elsewhere. You don't have to test the normal importer, it is just there for the curious. The other "import-records-from-file" importer is the Plant Portal importer, you don't need to test it as such, it is for your benefit for you to see it running (hopefully it a similar way that you envisioned it) Don't feel you have to do anything you don't want to do, the exercise is purely for your benefit.

[sacrevert]

OK, I understand. I have in fact already tested this with your 4 files, that it what was confusing me. Perhaps you did not see that I have edited all of your posts above concerning these files with my initial feedback. I could make additional minor comments on labelling etc., but for the moment I have restricted feedback to what I saw as the more substantial issues. Sorry, I thought that by editing and tagging the edits, you would be alerted to the fact.

[andrewvanbreda]

Just to clarify the "normal importer" page will not be on the final Plant Portal site and doesn't need testing, it is just there for anyone to see

[andrewvanbreda]

Ah OK, no I didn't see your comments I do not know why as I usually get emails, maybe it doesn't email me if you edit a comment I made originally. I will take a look at what you have said, feel free to edit your test file and retry it if you wish, but I would advise getting Biren to switch branch on the warehouse first. You can go ahead without doing that if you wish, as long as you are happy that there will be issues.

[sacrevert]

Regarding the results from the test files above:

1. Some work needed on labelling and wording, but in most cases this should be straightforward (see #58).
2. Contents of spatial ref field when using 'vague' VC/country boundary. My concern here is that the grid ref field is ending up with a precise grid reference, even though it is actually vague. Your reply didn’t really address the issue of whether holding a precise centroid in this field is sensible (unless there is another column that should be consulted in tandem to clarify that it is a centroid, and not an informative point location). I agree with your point that hiding it from the user might be useful (although I don't consider this urgent given our lack of funds); however, this doesn't quite ease my mind on the previous point.
3. I'm happy to leave the item regarding auto-hiding columns for mapping (where they have already been mapped). It doesn't sound like it is sensible to do this, and it is probably not worth the hassle.

[andrewvanbreda]

That is true actually, on my machine when I was programming the importer there was a column to indicate whether this was a vague spatial reference or precise one. I seem to have forgotten to put this on the dev site, probably because the code didn't really do anything with it. I could change it so this is set to vague automatically if a Country or Vice County is used.

As for whether it is sensible to hold a precise spatial reference. Are there disadvantages considering the column I neglected to use? It is hard for me to say, as I am the programmer so I didn't have the original vision of how the system would work. I suppose the main disadvantage I can think of is will the data ever be exported out of the Plant Portal system? and thereby into another system that doesn't understand the column that indicates if it vague. It might also be an issue displaying on maps, the map might zoom in on the precise spatial reference. That is what I can think of off the top of my head.

[sacrevert]

OK, I am happy if their is a field indicating that it is a 'vague' reference. As long as that exists that I am happy that future coding could take account of this in mapping and exporting to other systems. For example, if the reference if flagged as vague, then a map might know to use a boundary rather than a point etc. Regarding hiding these vague points from the user, I think this is desirable in the longer term, but not whilst we have limited funding. I will create a new issue to this effect.

[andrewvanbreda]

OK, I will leave it off my To Do list for now. I will create the column I mentioned as it is just an attribute and will be easy to add to the dev site

[andrewvanbreda]

Just to summarise all the discussion from yesterday. I will assume the terminology and error message wording issues will be dealt with gradually over time as these now have separate threads and do not require any immediate action. I think I am correct in understand that we are holding off on the hiding of the precise spatial reference from the user right now until more funding is confirmed (again has a separate thread now)

What would be good is to eventually be in a situation where you have seen your data import correctly, so we can be sure you are happy with that actual import rather than just the error messages. That will require the file to have its spatial references corrected and Biren also to switch the warehouse branch to the Plant Portal one. Those would be the bare minimum requirements to start that test...no urgency at my end, just as urgent as you want it to be.

Given all that discussion I think there is only one item to actively add to my To Do list right now and that is to put the attribute that holds whether the spatial reference is precise or vague into the importer, that will be a quick task.

@sacrevert Let me know if I have missed something

I am almost tempted to change the name of this thread to "Initial plot importer warning test observations" and there close it and start afresh....perhaps I could make the attribute change and then close it?

[andrewvanbreda]

Hi @sacrevert I just checked, I was talked rubbish actually, I thought I checked this yesterday but must of missed it. That attribute I was referring to was always there on the site It is called "Plant Portal spatial reference type" and it currently accepts the terms "Accurate" and "Vague". It was always on the mapping screen in the mapping drop-down next to each column it can be found under the "Sample custom attributes" heading and it is the "Plant Portal spatial reference type (from controlled termlist)" option Note that the names don't really matter, they are easy to change. Currently this is just set by the user, but one of my suggestions was that this could be auto-set by the system if it has to use a Vice County or Country to get the spatial reference if you like

[sacrevert]

Thanks Andy. I agree with your summary, and with your suggestion to close this thread and start a new one for the new import tests with the data that I previously created (not real data, but closer to reality).

@BirenRathod Can you do the 'branch switching' thing that Andy mentions?

[sacrevert]

Yes, I think the "Plant Portal spatial reference type" definitely needs to be set automatically.