jeffh
commented
9 years ago Hey Brian,
Currently Fox state machine generators don't provide a convenient way to have a customizations of transition generation. Technically, frequency, setup, and teardowns are possible on the parallel state machine branch (which is mostly complete, except a more deterministic PRNG is needed/research). Ideally, common setup and teardown should not be part of the state machine.
Otherwise, most of that seems correct. Fox state machine generation is highly affected by nextState and transitions. This means the testing model and subject under test greatly affect its run time (in that order).
I haven't gotten a chance to look at your code in full. But I'll try to in the next few days.
— Sent from my iPhone
On Sat, Feb 21, 2015 at 6:24 PM, Brian King notifications@github.com wrote:
Hey guys, Excited about Fox. I'm working on a project that composes index spaces from NSFetchedResultsController or static arrays into an tree structure that can be indexed by NSIndexPath and back UITableView / UICollectionView's. Whenever any underlying data source changes, change sets are percolated up to update the backing view. Any index space can be transformed via a filter with a few more transformations coming (Like sort and map). I've written a number of tests to aggrevate a few scenarios, but I'm very interested in writing tests that will aggrevate more scenarios. The majority of test scenarios involve starting a batch update, performing a number of inserts and removals, and then closing the update and confirming the state of the change set.
I wrote a state machine test that had 4 transitions, open, insert, remove, close. I could ensure that the transitions got called in the right order, but it wasn't very ideal. I ran into a few issues, and I was wondering if you had any input: 1) It took a long time to generate a valid test cases. 2) It executed a lot of test cases that did not close the batch update. Closing the batch is the test scenario that I'm most interested in testing. 3) I'm not sure how to control the number of transitions occur. It appears to be only generating X transitions, 1 for every defined transition. Ideally, we would open, perform a large number of random insert or remove operations, then close the batch. 4) Ideally I would have a generator define the initial state of the subject factory. Not sure how that would work. Here's the test for reference: https://github.com/KingOfBrian/RZAssemblage/blob/testing/fox/RZAssemblageTests/RZAssemblagePropertyTestCase.m#L203 I'm not sure that the state machine approach is the right direction. Maybe I should use a custom generator instead? Anyway, I get this is a pretty open ended question. Any insight would be great!Brian
Reply to this email directly or view it on GitHub: https://github.com/jeffh/Fox/issues/40
KingOfBrian
Hey guys, Excited about Fox. I'm working on a project that composes index spaces from NSFetchedResultsController or static arrays into an tree structure that can be indexed by NSIndexPath and back UITableView / UICollectionView's. Whenever any underlying data source changes, change sets are percolated up to update the backing view. Any index space can be transformed via a filter with a few more transformations coming (Like sort and map).
I've written a number of tests to aggrevate a few scenarios, but I'm very interested in writing tests that will aggrevate more scenarios. The majority of test scenarios involve starting a batch update, performing a number of inserts and removals, and then closing the update and confirming the state of the change set.
I wrote a state machine test that had 4 transitions, open, insert, remove, close. I could ensure that the transitions got called in the right order, but it wasn't very ideal. I ran into a few issues, and I was wondering if you had any input:
1) It took a long time to generate a valid test cases. 2) It executed a lot of test cases that did not close the batch update. Closing the batch is the test scenario that I'm most interested in testing. 3) I'm not sure how to control the number of transitions that occur. It appears to be only generating X transitions, 1 for every defined transition. Ideally, the test would open, perform a large number of random insert or remove operations, then close the batch. 4) I don't know how to define a generator to vary the initial state of the subject factory. Not sure how that would work.
Here's the test for reference:
https://github.com/KingOfBrian/RZAssemblage/blob/testing/fox/RZAssemblageTests/RZAssemblagePropertyTestCase.m#L203
I'm not sure that the state machine approach is the right direction. Maybe I should use a custom generator instead? Anyway, I get this is a pretty open ended question. Any insight would be great!
Brian