Results from detailed test of altered HLS_test_data_HL10_v2.csv

[andrewvanbreda]

@sacrevert Yesterday's test was mainly dealing with making sure the importer showed the correct warnings and I was able to the proceed after corrections. Today I have conducted a detailed test of the database state after an import of the same file, I also tried a second import of the same file (as this situation is different as there is existing data). Not,e as previously mentioned I altered the original file to remove the deliberate mistakes to allow import.

To my surprise almost everything past, I was expecting a few niggles, but there is one issue to deal with that I detected, no doubt things will crop up further down the line with other files

Passes are as follows: First import

• Correct number of occurrences created
• Correct number of plots created
• Correct number of plot groups created (1)
• Each plot correctly linked to plot group
• User linked to plot group
• Correct number of occurrence attribute values created (stratum, cover)

Second pass

• User correctly warned that we will be using existing plots, and is allowed to continue
• Correctly created same number of occurrences
• No new plots created as using existing ones
• No new plot groups created as using existing ones
• No plot group links created between user and plots as ones already exist
• Correct number of occurrence attribute values created (stratum, cover)

There is only one failure that I spotted. There were too many samples, I thought there would be 20, but it is 25, this is odd as I would expect there is be a link between the expected and actual numbers, such as twice as many or something like that. Perhaps 25 is right and I made a mistake counting. The number of sample attribute values (vice counties and countries) was also wrong, again the number appearing in the database compared to the expected result was somewhat random.

So that is what I will aim to fix going forward.

Test conducted on my own box at the minute as dev site not up to date, although if you wanted to see it working I am available for Skype demo

[sacrevert]

@andrewvanbreda sounds great, please carry on. No need for a demo at this point. Cheers!

[andrewvanbreda]

The samples were correct. I miscounted. However there is a problem with the sample attribute values. Far too few vice counties have been saved, but the counties are OK. Very strange. @sacrevert By the way, I just remembered something I was going to ask you, here you have used IDs for the vice counties, was this for test purposes? I had assumed the real import files would use a real names

[sacrevert]

@andrewvanbreda I can't honestly remember whether I did it for test purposes (I thought I had included names and number examples, but clearly not). It would be nice if the import could cope with numbers, which some recorders will use. The reason being that using names then throws up another thing that the user must check (e.g. spelling mistakes, capitalisation, some VCs also have name variants), and we will have to provide the standard list that we are expecting them to conform to somewhere.

AVB Response: @sacrevert Yes actually you are right, what you say about the IDs being less liable to error. Um having thought about it, we actually almost support using IDs already. The Vice Counties (and countries) are looked up against a custom list stored in the importer configuration, it doesn't actually talk to the warehouse for that, all I would need to do is extend this list to include IDs as well as Vice Counties. I thought doing it that way might have a use eventually This list would effectively just have two entries for each vice country, an ID and a name, but it could match either. I could do that very easily without further code change. The only question is that it would be we would be storing the ID into the Vice County field on the sample attribute that stores the Vice County. We could actually leave it like that as that is the entry the user actually made. I think perhaps the answer to this is that we don't need to do anything for this at all at the moment and it will probably crop up again later, I will make a separate issue so I can keep a note of my idea of extending the list otherwise I might forget and code something when I don't need

[sacrevert]

I prob should have included a grid ref / VC mismatch in that test as well, perhaps you can alter one grid ref next time you run a test. @andrewvanbreda

AVB Response: Actually the importer wouldn't care about a mismatch (whether that is right or not is another question). It just would say, OK there is a spatial reference already, we'll use that and I will store the Vice County name as a sample attribute. If the spatial reference is missing it would try to determine one from the vice county but it doesn't attempt to validate the current one. I personally don't think it should either, otherwise at what point do we stop......do we check the vice county is in the correct country etc, we have to stop somewhere and trust the data

[andrewvanbreda]

AVB Response: The sample attribute values issue appears to behave the same way with the standard Indicia importer. I will raise this separately as it may still be caused by Plant Portal changes in that area. The Vice County/Country issue has already been raised separately. Closing this now