Questions on Current state

[Lythom]

Hello !

I'm strongly interested in using Barometer for automated tests for my mod (https://github.com/Lythom/capsule). I'm currently updating the mod for latests 1.10, 1.11 and 1.12 version of forge and would like to test automatically compatibility with other mods.

Is Barometer usable as is for latest 1.10.2 of forge ? How much effort would it take to make it compatible with 1.11 and 1.12 ? (An estimate of the time it would take me on a fork to get it work).

More questions on capabilities : Can barometer simulate player logging / actions / interactions ? Would it be done only by simulating network packets ? I saw in the code : "TODO Make a nice empty world for tests". Did you have some progress on the feature ? What king of difficulty should I expect to put it in ?

Thank you for sharing your work on the topic of automated tests !

[jjtParadox]

Thanks for your interest! I'll do my best to answer your questions.

Is Barometer usable as is for latest 1.10.2 of forge ?

Unless something has changed in Forge's 1.10.2 API since this was last updated, this should still work with the latest forge. Otherwise it'll probably take less than 15 minutes of work to update it.

How much effort would it take to make it compatible with 1.11 and 1.12 ?

For 1.12, a branch already exists for it, but it still might need some minor updates. Updating from 1.10.2 to 1.12 didn't take much effort at all, so making something for 1.11 should be pretty simple.

I saw in the code : "TODO Make a nice empty world for tests". Did you have some progress on the feature ?

An empty world for tests can be accomplished by setting level-type to FLAT and generator-settings to 3;minecraft:air;127; in server.properties found in the test run dir (you might need to run the tests once for the server to create the files).

Can barometer simulate player logging / actions / interactions ? Would it be done only by simulating network packets ? ... [I] would like to test automatically compatibility with other mods.

I haven't experimented with network packets at all, so I have no clue what could or could not be accomplished with those, tho replicating the player login handshake might be a major challenge.

Unfortunately, player interaction and mod compatibility might be outside of the scope of what unit tests (and thus Barometer) can do. Barometer's basic function is to make sure individual tiny pieces of code work (e.g. chest placement code, furnace smelting code, etc). You might be able to fake player interactions by mocking a Player object and then calling the correct functions using the mocked object, but that will probably run into snags. Try looking into Java mocking frameworks (e.g. mockito, easymock) and seeing what you can accomplish with those.

For mod compatibility, you'll have to do a lot of testing to see if it's feasible. The CodeChickenLib mod might be able to load ordinary mods in a dev environment (just stick it and any mods in the test server's mods folder), so it might allow for some amount of testing, but I have no clue what will and will not break when being run with Barometer.

Good luck with your tests, and let me know if you discover anything useful!

(I'll close this issue in a week or two if there aren't any updates)

[Lythom]

Thank you for the detailed answer, you covered all my interrogations and I'll open new issues for specific points (or maybe PR for useful discoveries :smiley:), so I close this one.

Player interaction would be a nice to have, I can get equivalent tests for my need by spawning items and blocks, but I'll still look into it since it would allow to cover more test cases. Adding mods to check compatibility is a must have though, I'll let you know how it goes when I get something !

FYI, I plan to manually test my mod a little bit more then release a first update early next week to get some feedback from players, then get into automated tests to be able to reproduce reported bugs so that their status can be tracked automatically.