GH Actions fix - Githubissues

esl / MongooseIM

MongooseIM is Erlang Solutions' robust, scalable and efficient XMPP server, aimed at large installations. Specifically designed for enterprise purposes, it is fault-tolerant and can utilise the resources of multiple clustered machines.

Other

1.64k stars 422 forks source link

GH Actions fix #3992

Closed DenysGonchar closed 1 year ago

DenysGonchar commented 1 year ago

This PR introduces the following changes:

Replace of ErlGang/setup-erlang@v1.0.0 with erlef/setup-beam@v1 action + version update for other actions
Update of OTP versions used by GH Actions to 24.3.4 and 25.2
Update of GH Actions runners to ubuntu-22.04
Split of the big-tests steps into a separate action
Adding testing of dynamic domains profiles
Re-enabling of coveralls reporting from GH Actions
Fix of Jingle/SIP testing by GH Actions. codecov VS coveralls coverage report comparison showed that Jingle/SIP functionality previously was not tested by GH Actions at all.
Elimination of coverage reporting for automatically generated modules (XmppAddr and eldap_filter_yecc):
- This is required for aligning coveralls and codecov coverage reporting
- codecov simply excludes missing in GH repo modules from the total coverage calculations, while coveralls includes them.
- However, that doesn't make much sense since analysis of such unavailable files is impossible using coveralls service
- As an example, see how XmppAddr module is reported and compare it to xml module.
- missing functionality has been added to rebar3_codecov plugin, check this commit for more details.

codecov.io report coveralls.io reports GH Actions reports CircleCI reports

codecov/coveralls reports diff for 541b9c19 commit (before Jingle/SIP testing fix and with generated modules) is here

TODO: some of the tests are flaky and this results in frequent GH actions jobs failures (especially MySQL jobs), this should be investigated and fixed in the following PRs.

mongoose-im commented 1 year ago

small_tests_24 / small_tests / 570ede1b1df79ae3b77f87efbbed307a6580417a Reports root / small

small_tests_25_arm64 / small_tests / 570ede1b1df79ae3b77f87efbbed307a6580417a Reports root / small

small_tests_25 / small_tests / 570ede1b1df79ae3b77f87efbbed307a6580417a Reports root / small

ldap_mnesia_24 / ldap_mnesia / 570ede1b1df79ae3b77f87efbbed307a6580417a Reports root/ big OK: 2219 / Failed: 0 / User-skipped: 825 / Auto-skipped: 0

dynamic_domains_pgsql_mnesia_24 / pgsql_mnesia / 570ede1b1df79ae3b77f87efbbed307a6580417a Reports root/ big OK: 4178 / Failed: 0 / User-skipped: 88 / Auto-skipped: 0

ldap_mnesia_25 / ldap_mnesia / 570ede1b1df79ae3b77f87efbbed307a6580417a Reports root/ big OK: 2219 / Failed: 0 / User-skipped: 825 / Auto-skipped: 0

dynamic_domains_mysql_redis_25 / mysql_redis / 570ede1b1df79ae3b77f87efbbed307a6580417a Reports root/ big OK: 4151 / Failed: 1 / User-skipped: 114 / Auto-skipped: 0

service_domain_db_SUITE:db:db_keeps_syncing_after_cluster_join

```erlang {error,{test_case_failed,{[<<"example1.com">>,<<"example2.com">>, <<"example3.com">>], [<<"example1.com">>,<<"example2.com">>, <<"example3.com">>,<<"example4.com">>]}}} ``` [Report log](https://circleci-mim-results.s3.eu-central-1.amazonaws.com/PR/3992/170867/mysql_redis.25.3-amd64/big/ct_run.test%4024248355f061.2023-03-21_14.00.20/big_tests.tests.service_domain_db_SUITE.logs/run.2023-03-21_14.13.11/service_domain_db_suite.db_keeps_syncing_after_cluster_join.html)

dynamic_domains_mssql_mnesia_25 / odbc_mssql_mnesia / 570ede1b1df79ae3b77f87efbbed307a6580417a Reports root/ big OK: 4174 / Failed: 1 / User-skipped: 91 / Auto-skipped: 0

muc_SUITE:register:user_changes_nick

```erlang {error, {{assertion_failed,assert,is_iq_result, [{xmlel,<<"iq">>, [{<<"type">>,<<"set">>}, {<<"id">>,<<"bbe9bc9f9d3ccc7ef108b8aebc89e7cf">>}, {<<"to">>,<<"groupchats.domain.example.com">>}], [{xmlel,<<"query">>, [{<<"xmlns">>,<<"jabber:iq:register">>}], [{xmlel,<<"x">>, [{<<"xmlns">>,<<"jabber:x:data">>}, {<<"type">>,<<"submit">>}], [{xmlel,<<"field">>, [{<<"type">>,<<"hidden">>}, {<<"var">>,<<"FORM_TYPE">>}], [{xmlel,<<"value">>,[], [{xmlcdata,<<"jabber:iq:register">>}]}]}, {xmlel,<<"field">>, [{<<"type">>,<<"text-single">>}, {<<"var">>,<<"nick">>}], [{xmlel,<<"value">>,[], [{xmlcdata, <<"thirdwitch1room-dcbde9d99c">>}]}]}]}]}]}], {xmlel,<<"iq">>, [{<<"from">>,<<"groupchats.domain.example.com">>}, {<<"to">>, <<"alice_user_changes_nick_2447@domain.example.com/res1">>}, {<<"type">>,<<"error">>}, {<<"id">>,<<"bbe9bc9f9d3ccc7ef108b8aebc89e7cf">>}], [{xmlel,<<"query">>, [{<<"xmlns">>,<<"jabber:iq:register">>}], [{xmlel,<<"x">>, [{<<"xmlns">>,<<"jabber:x:data">>}, {<<"type">>,<<"submit">>}], [{xmlel,<<"field">>, [{<<"type">>,<<"hidden">>}, {<<"var">>,<<"FORM_TYPE">>}], [{xm... ``` [Report log](https://circleci-mim-results.s3.eu-central-1.amazonaws.com/PR/3992/170858/odbc_mssql_mnesia.25.3-amd64/big/ct_run.test%407b89cabf0877.2023-03-21_13.59.42/big_tests.tests.muc_SUITE.logs/run.2023-03-21_14.10.17/muc_suite.user_changes_nick.196482.html)

dynamic_domains_pgsql_mnesia_25 / pgsql_mnesia / 570ede1b1df79ae3b77f87efbbed307a6580417a Reports root/ big OK: 4178 / Failed: 0 / User-skipped: 88 / Auto-skipped: 0

pgsql_mnesia_24 / pgsql_mnesia / 570ede1b1df79ae3b77f87efbbed307a6580417a Reports root/ big OK: 4551 / Failed: 0 / User-skipped: 97 / Auto-skipped: 0

internal_mnesia_25 / internal_mnesia / 570ede1b1df79ae3b77f87efbbed307a6580417a Reports root/ big OK: 2361 / Failed: 0 / User-skipped: 683 / Auto-skipped: 0

elasticsearch_and_cassandra_25 / elasticsearch_and_cassandra_mnesia / 570ede1b1df79ae3b77f87efbbed307a6580417a Reports root/ big OK: 2721 / Failed: 0 / User-skipped: 662 / Auto-skipped: 0

pgsql_mnesia_25 / pgsql_mnesia / 570ede1b1df79ae3b77f87efbbed307a6580417a Reports root/ big OK: 4551 / Failed: 0 / User-skipped: 97 / Auto-skipped: 0

mysql_redis_25 / mysql_redis / 570ede1b1df79ae3b77f87efbbed307a6580417a Reports root/ big OK: 4537 / Failed: 0 / User-skipped: 111 / Auto-skipped: 0

mssql_mnesia_25 / odbc_mssql_mnesia / 570ede1b1df79ae3b77f87efbbed307a6580417a Reports root/ big OK: 4547 / Failed: 1 / User-skipped: 100 / Auto-skipped: 0

pep_SUITE:pep_tests:unsubscribe_after_presence_unsubscription

```erlang {error,{{assertion_failed,assert_many,false,[is_presence],[],[]}, [{escalus_new_assert,assert_true,2, [{file,"/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_new_assert.erl"}, {line,84}]}, {escalus_story,'-make_all_clients_friends/1-fun-0-',2, [{file,"/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_story.erl"}, {line,109}]}, {escalus_utils,'-each_with_index/3-fun-0-',3, [{file,"/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_utils.erl"}, {line,87}]}, {lists,foldl_1,3,[{file,"lists.erl"},{line,1355}]}, {escalus_utils,'-each_with_index/3-fun-0-',3, [{file,"/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_utils.erl"}, {line,87}]}, {lists,foldl,3,[{file,"lists.erl"},{line,1350}]}, {escalus_utils,distinct_pairs,2, [{file,"/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_utils.erl"}, {line,60}]}, {escalus_story,make_all_clients_friends,1, [{file,"/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_story.erl"}, {line,106}]}]}} ``` [Report log](https://circleci-mim-results.s3.eu-central-1.amazonaws.com/PR/3992/170864/odbc_mssql_mnesia.25.3-amd64/big/ct_run.test%40155fba96a70b.2023-03-21_13.59.43/big_tests.tests.pep_SUITE.logs/run.2023-03-21_14.13.25/pep_suite.unsubscribe_after_presence_unsubscription.144387.html)

codecov[bot] commented 1 year ago

Codecov Report

Patch coverage has no change and project coverage change: +0.08 :tada:

Comparison is base (afb9ecf) 82.06% compared to head (474757b) 82.15%.

Additional details and impacted files

```diff @@ Coverage Diff @@ ## master #3992 +/- ## ========================================== + Coverage 82.06% 82.15% +0.08% ========================================== Files 536 535 -1 Lines 33902 33879 -23 ========================================== + Hits 27822 27833 +11 + Misses 6080 6046 -34 ``` [see 10 files with indirect coverage changes](https://codecov.io/gh/esl/MongooseIM/pull/3992/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=esl) Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=esl). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=esl)

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

mongoose-im commented 1 year ago

small_tests_24 / small_tests / 9888d9835a9066bd8d051bafd8b7619a594b5ce8 Reports root / small

small_tests_25_arm64 / small_tests / 9888d9835a9066bd8d051bafd8b7619a594b5ce8 Reports root / small

small_tests_25 / small_tests / 9888d9835a9066bd8d051bafd8b7619a594b5ce8 Reports root / small

ldap_mnesia_24 / ldap_mnesia / 9888d9835a9066bd8d051bafd8b7619a594b5ce8 Reports root/ big OK: 2219 / Failed: 0 / User-skipped: 825 / Auto-skipped: 0

dynamic_domains_pgsql_mnesia_24 / pgsql_mnesia / 9888d9835a9066bd8d051bafd8b7619a594b5ce8 Reports root/ big OK: 4178 / Failed: 0 / User-skipped: 88 / Auto-skipped: 0

ldap_mnesia_25 / ldap_mnesia / 9888d9835a9066bd8d051bafd8b7619a594b5ce8 Reports root/ big OK: 2219 / Failed: 0 / User-skipped: 825 / Auto-skipped: 0

dynamic_domains_pgsql_mnesia_25 / pgsql_mnesia / 9888d9835a9066bd8d051bafd8b7619a594b5ce8 Reports root/ big OK: 4178 / Failed: 0 / User-skipped: 88 / Auto-skipped: 0

dynamic_domains_mysql_redis_25 / mysql_redis / 9888d9835a9066bd8d051bafd8b7619a594b5ce8 Reports root/ big OK: 4152 / Failed: 0 / User-skipped: 114 / Auto-skipped: 0

pgsql_mnesia_24 / pgsql_mnesia / 9888d9835a9066bd8d051bafd8b7619a594b5ce8 Reports root/ big OK: 4551 / Failed: 0 / User-skipped: 97 / Auto-skipped: 0

dynamic_domains_mssql_mnesia_25 / odbc_mssql_mnesia / 9888d9835a9066bd8d051bafd8b7619a594b5ce8 Reports root/ big OK: 4175 / Failed: 0 / User-skipped: 91 / Auto-skipped: 0

internal_mnesia_25 / internal_mnesia / 9888d9835a9066bd8d051bafd8b7619a594b5ce8 Reports root/ big OK: 2361 / Failed: 0 / User-skipped: 683 / Auto-skipped: 0

elasticsearch_and_cassandra_25 / elasticsearch_and_cassandra_mnesia / 9888d9835a9066bd8d051bafd8b7619a594b5ce8 Reports root/ big OK: 2721 / Failed: 0 / User-skipped: 662 / Auto-skipped: 0

pgsql_mnesia_25 / pgsql_mnesia / 9888d9835a9066bd8d051bafd8b7619a594b5ce8 Reports root/ big OK: 4551 / Failed: 0 / User-skipped: 97 / Auto-skipped: 0

mysql_redis_25 / mysql_redis / 9888d9835a9066bd8d051bafd8b7619a594b5ce8 Reports root/ big OK: 4537 / Failed: 0 / User-skipped: 111 / Auto-skipped: 0

mssql_mnesia_25 / odbc_mssql_mnesia / 9888d9835a9066bd8d051bafd8b7619a594b5ce8 Reports root/ big OK: 4548 / Failed: 0 / User-skipped: 100 / Auto-skipped: 0

mongoose-im commented 1 year ago

small_tests_24 / small_tests / 5d29461f4b3332ec8cda1392d06174ef720f0d3c Reports root / small

small_tests_25 / small_tests / 5d29461f4b3332ec8cda1392d06174ef720f0d3c Reports root / small

small_tests_25_arm64 / small_tests / 5d29461f4b3332ec8cda1392d06174ef720f0d3c Reports root / small

ldap_mnesia_24 / ldap_mnesia / 5d29461f4b3332ec8cda1392d06174ef720f0d3c Reports root/ big OK: 2219 / Failed: 0 / User-skipped: 825 / Auto-skipped: 0

dynamic_domains_pgsql_mnesia_24 / pgsql_mnesia / 5d29461f4b3332ec8cda1392d06174ef720f0d3c Reports root/ big OK: 4178 / Failed: 0 / User-skipped: 88 / Auto-skipped: 0

ldap_mnesia_25 / ldap_mnesia / 5d29461f4b3332ec8cda1392d06174ef720f0d3c Reports root/ big OK: 2219 / Failed: 0 / User-skipped: 825 / Auto-skipped: 0

dynamic_domains_mysql_redis_25 / mysql_redis / 5d29461f4b3332ec8cda1392d06174ef720f0d3c Reports root/ big OK: 4152 / Failed: 0 / User-skipped: 114 / Auto-skipped: 0

dynamic_domains_pgsql_mnesia_25 / pgsql_mnesia / 5d29461f4b3332ec8cda1392d06174ef720f0d3c Reports root/ big OK: 4178 / Failed: 0 / User-skipped: 88 / Auto-skipped: 0

dynamic_domains_mssql_mnesia_25 / odbc_mssql_mnesia / 5d29461f4b3332ec8cda1392d06174ef720f0d3c Reports root/ big OK: 4175 / Failed: 0 / User-skipped: 91 / Auto-skipped: 0

pgsql_mnesia_24 / pgsql_mnesia / 5d29461f4b3332ec8cda1392d06174ef720f0d3c Reports root/ big OK: 4551 / Failed: 0 / User-skipped: 97 / Auto-skipped: 0

elasticsearch_and_cassandra_25 / elasticsearch_and_cassandra_mnesia / 5d29461f4b3332ec8cda1392d06174ef720f0d3c Reports root/ big OK: 2721 / Failed: 0 / User-skipped: 662 / Auto-skipped: 0

internal_mnesia_25 / internal_mnesia / 5d29461f4b3332ec8cda1392d06174ef720f0d3c Reports root/ big OK: 2361 / Failed: 0 / User-skipped: 683 / Auto-skipped: 0

mssql_mnesia_25 / odbc_mssql_mnesia / 5d29461f4b3332ec8cda1392d06174ef720f0d3c Reports root/ big OK: 4547 / Failed: 1 / User-skipped: 100 / Auto-skipped: 0

rest_client_SUITE:roster:add_and_remove_some_contacts_with_nonexisting

```erlang {error,{{badmatch,{{<<"500">>,<<"Internal Server Error">>},<<>>}}, [{rest_client_SUITE,add_contact_check_roster_push,2, [{file,"/home/circleci/project/big_tests/tests/rest_client_SUITE.erl"}, {line,1390}]}, {lists,foreach_1,2,[{file,"lists.erl"},{line,1442}]}, {rest_client_SUITE,'-add_and_remove_some_contacts_with_nonexisting/1-fun-1-', 4, [{file,"/home/circleci/project/big_tests/tests/rest_client_SUITE.erl"}, {line,1358}]}, {escalus_story,story,4, [{file,"/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_story.erl"}, {line,72}]}, {rest_client_SUITE,add_and_remove_some_contacts_with_nonexisting,1, [{file,"/home/circleci/project/big_tests/tests/rest_client_SUITE.erl"}, {line,1353}]}, {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1782}]}, {test_server,run_test_case_eval1,6, [{file,"test_server.erl"},{line,1291}]}, {test_server,run_test_case_eval,9, [{file,"test_server.erl"},{line,1223}]}]}} ``` [Report log](https://circleci-mim-results.s3.eu-central-1.amazonaws.com/PR/3992/170921/odbc_mssql_mnesia.25.3-amd64/big/ct_run.test%40687165ce2930.2023-03-21_20.46.10/big_tests.tests.rest_client_SUITE.logs/run.2023-03-21_21.01.14/rest_client_suite.add_and_remove_some_contacts_with_nonexisting.166659.html)

mysql_redis_25 / mysql_redis / 5d29461f4b3332ec8cda1392d06174ef720f0d3c Reports root/ big OK: 4537 / Failed: 0 / User-skipped: 111 / Auto-skipped: 0

pgsql_mnesia_25 / pgsql_mnesia / 5d29461f4b3332ec8cda1392d06174ef720f0d3c Reports root/ big OK: 4551 / Failed: 0 / User-skipped: 97 / Auto-skipped: 0

mongoose-im commented 1 year ago

small_tests_24 / small_tests / a2c834c4b1b208669e73a36b4eb041c15391c9fd Reports root / small

small_tests_25_arm64 / small_tests / a2c834c4b1b208669e73a36b4eb041c15391c9fd Reports root / small

small_tests_25 / small_tests / a2c834c4b1b208669e73a36b4eb041c15391c9fd Reports root / small

ldap_mnesia_24 / ldap_mnesia / a2c834c4b1b208669e73a36b4eb041c15391c9fd Reports root/ big OK: 2219 / Failed: 0 / User-skipped: 825 / Auto-skipped: 0

dynamic_domains_pgsql_mnesia_24 / pgsql_mnesia / a2c834c4b1b208669e73a36b4eb041c15391c9fd Reports root/ big OK: 4178 / Failed: 0 / User-skipped: 88 / Auto-skipped: 0

ldap_mnesia_25 / ldap_mnesia / a2c834c4b1b208669e73a36b4eb041c15391c9fd Reports root/ big OK: 2219 / Failed: 0 / User-skipped: 825 / Auto-skipped: 0

dynamic_domains_pgsql_mnesia_25 / pgsql_mnesia / a2c834c4b1b208669e73a36b4eb041c15391c9fd Reports root/ big OK: 4178 / Failed: 0 / User-skipped: 88 / Auto-skipped: 0

dynamic_domains_mysql_redis_25 / mysql_redis / a2c834c4b1b208669e73a36b4eb041c15391c9fd Reports root/ big OK: 4152 / Failed: 0 / User-skipped: 114 / Auto-skipped: 0

pgsql_mnesia_24 / pgsql_mnesia / a2c834c4b1b208669e73a36b4eb041c15391c9fd Reports root/ big OK: 4551 / Failed: 0 / User-skipped: 97 / Auto-skipped: 0

internal_mnesia_25 / internal_mnesia / a2c834c4b1b208669e73a36b4eb041c15391c9fd Reports root/ big OK: 2361 / Failed: 0 / User-skipped: 683 / Auto-skipped: 0

dynamic_domains_mssql_mnesia_25 / odbc_mssql_mnesia / a2c834c4b1b208669e73a36b4eb041c15391c9fd Reports root/ big OK: 4175 / Failed: 0 / User-skipped: 91 / Auto-skipped: 0

elasticsearch_and_cassandra_25 / elasticsearch_and_cassandra_mnesia / a2c834c4b1b208669e73a36b4eb041c15391c9fd Reports root/ big OK: 2721 / Failed: 0 / User-skipped: 662 / Auto-skipped: 0

mysql_redis_25 / mysql_redis / a2c834c4b1b208669e73a36b4eb041c15391c9fd Reports root/ big OK: 4537 / Failed: 0 / User-skipped: 111 / Auto-skipped: 0

pgsql_mnesia_25 / pgsql_mnesia / a2c834c4b1b208669e73a36b4eb041c15391c9fd Reports root/ big OK: 4551 / Failed: 0 / User-skipped: 97 / Auto-skipped: 0

mssql_mnesia_25 / odbc_mssql_mnesia / a2c834c4b1b208669e73a36b4eb041c15391c9fd Reports root/ big OK: 4547 / Failed: 1 / User-skipped: 100 / Auto-skipped: 0

muc_SUITE:hibernation:hibernated_room_can_be_queried_for_archive

```erlang {error,{{assertion_failed,assert,is_groupchat_message, [<<"Restorable message">>], undefined,"undefined"}, [{escalus_new_assert,assert_true,2, [{file,"/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_new_assert.erl"}, {line,84}]}, {muc_SUITE,wait_for_mam_result,3, [{file,"/home/circleci/project/big_tests/tests/muc_SUITE.erl"}, {line,4394}]}, {muc_SUITE,'-hibernated_room_can_be_queried_for_archive/1-fun-0-',3, [{file,"/home/circleci/project/big_tests/tests/muc_SUITE.erl"}, {line,4130}]}, {escalus_story,story,4, [{file,"/home/circleci/project/big_tests/_build/default/lib/escalus/src/escalus_story.erl"}, {line,72}]}, {muc_SUITE,hibernated_room_can_be_queried_for_archive,1, [{file,"/home/circleci/project/big_tests/tests/muc_SUITE.erl"}, {line,4126}]}, {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1782}]}, {test_server,run_test_case_eval1,6, [{file,"test_server.erl"},{line,1291}]}, {test_server,run_test_case_eval,9, [{file,"test_server.erl"},{line,1223}]}]}} ``` [Report log](https://circleci-mim-results.s3.eu-central-1.amazonaws.com/PR/3992/170946/odbc_mssql_mnesia.25.3-amd64/big/ct_run.test%40ad33c250086c.2023-03-21_21.21.14/big_tests.tests.muc_SUITE.logs/run.2023-03-21_21.33.38/muc_suite.hibernated_room_can_be_queried_for_archive.88324.html)