slub / ocrd_kitodo

Docker integration of Kitodo.Production and OCR-D
MIT License
9 stars 6 forks source link

Add make test-kitodo #63

Closed markusweigelt closed 1 year ago

markusweigelt commented 1 year ago

Atm we call make testand run all test. Possibly only test-kitodo and test-presentation is still necessary as the test-production is indirectly tested by test-kitodo.

bertsky commented 1 year ago

Possibly only test-kitodo and test-presentation is still necessary as the test-production is indirectly tested by test-kitodo.

Yes, but test-production (alone) tests in sync mode, which is good for coverage.

BTW, on the CI we could do make -j test which will run all 3 tests in parallel. To make that work with the Controller, we can either add CONTROLLER_WORKERS=3 make -j test so that they will actually be computed in parallel, or keep the default 1 (which will have each job wait for the previous job to free the semaphore). We could even test both...

markusweigelt commented 1 year ago

BTW, on the CI we could do make -j test which will run all 3 tests in parallel. To make that work with the Controller, we can either add CONTROLLER_WORKERS=3 make -j test so that they will actually be computed in parallel, or keep the default 1 (which will have each job wait for the previous job to free the semaphore). We could even test both...

Probably we have to adjust the make call in Makefile too https://github.com/slub/ocrd_kitodo/blob/6f665a2c8e550fd5a2498d501aaeb950017ac9c6/Makefile#L153.

Atm rerunning of test-kitodo without make down and make start is a problem cause we need to flush the manager log to provide the check for KitodoActiveMQClient if ocr processing has finished. https://github.com/slub/ocrd_kitodo/blob/c3d29a5713aa3894e75117a27da01e3768b92221/Makefile#L169

truncate -s 0 $(docker inspect --format='{{.LogPath}}' ocrd_kitodo-ocrd-manager-1 ) flush's the log with sudo but probably there are permission issues without sudo at the runner. But i will check this.

bertsky commented 1 year ago

Atm rerunning of test-kitodo without make down and make start is a problem cause we need to flush the manager log to provide the check for KitodoActiveMQClient if ocr processing has finished.

indeed.

But truncating the Docker log file is asking for trouble. I'd rather keep it simple and require down-up.

Or is there another way to query the ActiveMQ was received, perhaps something on the process directory?

markusweigelt commented 1 year ago

Yes. Instead of testing for existing KitodoActiveMQClient request in log, we can check if the alto files are already in the directory. If they are not present within of 5 minutes, an error will be thrown. We would only have to delete the alto directory before rerun next test.

bertsky commented 1 year ago

Yes, sounds good. (Or directly intercept the ActiveMQ result by interacting with our kitodo-mq container?)

markusweigelt commented 1 year ago

Changed the implementation to check for existing of an alto file.

markusweigelt commented 1 year ago

Add the issue #64 and merge branch as discussed