esl / escalus

An XMPP client library in Erlang for conveniently testing XMPP servers
Apache License 2.0
129 stars 76 forks source link

Use unique names for fresh story users #249

Closed arcusfelis closed 2 years ago

arcusfelis commented 2 years ago

Some suites run the same testcase several times This means a duplicate key error could occure when registering a new user This also means some random stanzas would come from another cases It is especially visible when two cases run in parallel (but it is not a requirement)

A collision example: https://circleci-mim-results.s3.eu-central-1.amazonaws.com/PR/3420/91739/pgsql_mnesia.23.1/big/ct_run.test@9111b0329547.2021-12-02_09.32.48/big_tests.tests.amp_big_SUITE.logs/run.2021-12-02_09.33.01/amp_big_suite.notify_deliver_to_online_user_recipient_privacy_test.13699.html

Error in the case:

bOb_notify_deliver_to_online_user_recipient_privacy_test_86.765794@localhost/res1 in
<stream:error>
    <conflict xmlns='urn:ietf:params:xml:ns:xmpp-streams'/>
    <text xml:lang='en' xmlns='urn:ietf:params:xml:ns:xmpp-streams'>Replaced by new connection</text>
</stream:error>

You can think, wtf, we use escalus:fresh_story and there is only one story function in the case. Yeah, but amp suite implements a complex trick to run the same testcase several times with different configs. And apparently, circle CI reproduces a collision in suffixes very well (impossible to reproduce on my machine though...)

See in logs:

55 when=2021-12-02T09:33:06.769075+00:00 level=error what=sql_execute_failed reason={error,duplicate_key} pid=<0.5697.0> at=mongoose_rdbms:execute_successfully/3:205 statement_query="INSERT INTO users(server, username, password, pass_details) VALUES (?, ?, ?, ?)" statement_params="[<<\"localhost\">>,<<\"alice_notify_deliver_to_online_user_recipient_privacy_test_86.765794\">>,<<>>,<<\"==MULTI_SCRAM==,64,==SHA256==CL0aTTPG1CX7PImxi+zscA==|uA6CuzJYWxNELyi4bKdrxPr9w55UBMh7fRqrxcM365c=|2emATuKdGIn50GMAAjLGSfvw/GOZHs34iVGaTdhgNWs=\">>]" statement_name=auth_add_user_scram host=localhost 
56 when=2021-12-02T09:33:06.769985+00:00 level=error what=registration_failed reason="#{host => <<\"localhost\">>,reason => {error,duplicate_key},statement_name => auth_add_user_scram,statement_params => [<<\"localhost\">>,<<\"alice_notify_deliver_to_online_user_recipient_privacy_test_86.765794\">>,<<>>,<<\"==MULTI_SCRAM==,64,==SHA256==CL0aTTPG1CX7PImxi+zscA==|uA6CuzJYWxNELyi4bKdrxPr9w55UBMh7fRqrxcM365c=|2emATuKdGIn50GMAAjLGSfvw/GOZHs34iVGaTdhgNWs=\">>],statement_query => <<\"INSERT INTO users(server, username, password, pass_details) VALUES (?, ?, ?, ?)\">>,what => sql_execute_failed}" pid=<0.5697.0> at=ejabberd_auth_rdbms:try_register/4:185 user=alice_notify_deliver_to_online_user_recipient_privacy_test_86.765794 stacktrace="mongoose_rdbms:execute_successfully/3:206 ejabberd_auth_rdbms:try_register/4:178 ejabberd_auth:'-do_try_register_if_does_not_exist/3-fun-0-'/5:237 ejabberd_auth:fold_auth_modules/3:537 timer:tc/2:181 ejabberd_auth:call_auth_modules_for_domain/3:500 ejabberd_admin:register/3:328 erpc:execute_call/4:416" server=localhost 
5

We also have two cases with the same fresh story suffix.

MongooseIM branch, configured with this PR https://github.com/esl/MongooseIM/pull/3429