HGStorageImplementation interface test

[GoogleCodeExporter]

We need full comprehensive coverage of this interface and all storage related 
interfaces. Especially now where multiple storage engines are being 
implemented...

Original issue reported on code.google.com by borislav...@gmail.com on 25 Jun 2012 at 4:44

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

There is one simpler test suite that is complete, and one more sophisticated 
test suite under development. Both are written in scala. Neither of them uses a 
Testing frameworks, since Unit tests, imho, are not well suited here, imho. 
Also, TestNG makes it really hard to debug failed tests, the stacktrace is 
ultra long etc. I don't see the advantage of a test framework for my purposes. 
But I'd willing to help link it up with the existing hgtest package.
Both tests dont test by single data input / test, but on large scale by using 
collections filled with autogenerated testdata of tuneable size, that 
correspond to the tested component. For example for HGStoreImplementation, 
which is split in to data (handle => byte array), links (handle => handle 
array) and incidence Set (handle => HGRandomResultSet). 
Another feature of both frameworks is permutation of important configuration 
options are made and all tests are run for each config-permutation.
Both tests also have some simple benchmarking, with possibility to include a 
warmup phase by simply let one (or each) configuration run once more.

Simple test located at: org.hypergraphdb.storage.hazelstore.testing.BasicTests
This is the entry point. In there are separate tests for storage and indices. 
Each test can also be run separately. All above implies, but code is more 
difficult to read and maintain.

The other test is supposed to make the tests more powerful, better integrated 
final reports, easier to maintain and extend. It was done by factoring out some 
common pattern: basically any test consists generation of testdata, mutations 
on the tested component using that testdata, and validations with respect to 
whether mutation was an addition or a removal. 
In each mutation, some function is performed on some testdata input that either 
adds to or removes (so, mutates) from the tested component. After each 
mutation, the whole set of validations are run systematically. In BasicTests 
suite this is coded for each case on its own.
HazelTestCenter is the entry point and runs the tests on config permutations. 
The logic for all described above is factored out in the HazelTest trait. The 
tests themselves are now much shorter and only consist of a bunch of functions.
Implemented but currently not yet working: 
- after removals, two things should be checked: has the removal succeeded? and 
is the rest intact? 
- reporting

Original comment by Ingvar.Bogdahn on 15 May 2013 at 10:15

[GoogleCodeExporter]

I added a bunch of test cases for 'org.hypergraphdb.storage.bje' package (75% 
Line Coverage, 66% Branch Coverage according to Cobertura's report). All of 
them are pure unit test. I developed them from scratch. 

Each class is tested in isolation (as far as possible) from other classes. Each 
method is tested on small data input. Also there are test cases for abnormal 
scenarios (like occurring exception in method which is under test). 

This 'pure unit test approach' has pros and cons.

From my perspective Pros are:

1) Isolation – when something goes wrong in the particular class/method it is 
not so hard to identify 'broken' code.

2) Clearness – each test case is an example of using functionality of 
particular class.

3) Modularity – there are separate test cases for each feature (including 
test cases for particular bugs after they become fixed).

And the Cons are:

1) Not all behavior is checked (thread locking, etc.), too less input data for 
covering all cases (especially for handles and links).

2) This test cases are difficult to maintain due to code duplication.

But as I see these test cases are necessary. But for comprehensive coverage 
more sophisticated tests are required. I am going to make cleanup of these unit 
test cases and keep them in the project. Then I am going to refactor existing 
package ('testcore' module) taking advantages from the tests located in 
'hazelstore' module. 

Please, tell me about your suggestions for cleaning-up this new unit tests and 
improving 'testcore'.
Or maybe it is better to focus on another issues (for example Issue 78: Unit 
tests for subgraphs)?

Original comment by YuriySec...@gmail.com on 16 Feb 2015 at 10:35

[GoogleCodeExporter]

Hi Yuriy,

This is great!!

My main question is: how specific to BJE are those tests. I can see that
some of them apply only to BJE, but probably a lot can be placed into
'testcore' so that different storage implementations can be tested.

In terms of refactoring and cleaning up, feel free to propose what you feel
is right. There are two main things that I would like to see: (1) that
independent storage layer tests be placed in testcore so one can run the
test suite with an implementation of one's choosing and (2) get rid of
TestNG and switch to jUnit.  The reason for (2) is that while TestNG was a
good choice at the time, jUnit is now much more comprehensive and much more
readily available in IDEs.

Otherwise, if you would like to contribute to the main code, let me know
what part interests you?

Cheers,
Boris

Original comment by borislav...@gmail.com on 18 Feb 2015 at 5:15

[GoogleCodeExporter]

Hi, Boris!

The independent storage layer tests can be placed into 'testcore' module (test 
cases for methods which are inherited from interfaces and abstract classes 
coming from 'core' module, like HGSortIndex.findLT() or 
HGStorageImplementation.getData() ).

The difference is only in code for starting up and shutting down appropriate 
underlying storage engine. But many test cases (for abnormal scenarios) need 
more specific initialization. So when we have to move common test cases we also 
should move these specific test cases. Otherwise we get tests for the same 
methods in different modules. As for me it is not good.

But it will be nice to have common test cases simultaneously with the specific 
test cases (which are organized into small chunks of code). I will look for the 
appropriate Java library.

BR,
Yuriy

Original comment by YuriySec...@gmail.com on 18 Feb 2015 at 7:46

[GoogleCodeExporter]

Hi!

I tried to apply the following logic both to BJEStorageImplementation and 
BDBStorageImplementation:

    final HGPersistentHandle handle = new UUIDPersistentHandle();
    storage.store(handle, new byte[] {});
    final byte[] retrieved = storage.getData(handle);

In case of BJE I'v got empty array retrieved, but in case of BDB I'v got null. 
So which behaviour is correct?

Original comment by YuriySec...@gmail.com on 18 Apr 2015 at 4:32

[GoogleCodeExporter]

I would say returning an empty array is the correct behavior since that is what 
was stored. 

Original comment by borislav...@gmail.com on 18 Apr 2015 at 4:36

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Thanks. I've fixed that. Now BJEStorageImplementation and 
BDBStorageImplementation return empty array.

Original comment by YuriySec...@gmail.com on 19 Apr 2015 at 10:16