search

389ds / 389-ds-base

The enterprise-class Open Source LDAP server for Linux
https://www.port389.org/
Other
211 stars 91 forks source link

lib389 - build "suite" framework #1334

Closed 389-ds-bot closed 4 years ago

389-ds-bot commented 4 years ago

Cloned from Pagure issue: https://pagure.io/389-ds-base/issue/48003

Create a test suite, similar to TET's "basic" acceptance suite, to lib389. This test suite should perform a variety of basic/common operations and tasks.

Also, start the overall framework for "suite" tests(TET porting)

389-ds-bot commented 4 years ago

Comment from nhosoi (@nhosoi) at 2015-02-14 05:00:19

Looks great. A couple of trivial comments/quiestions...

1) It'd be nice to have a clean dse.ldif in the repo: We don't need these attributes: creatorsName, modifiersName, createTimestamp, modifyTimestamp, numSubordinates. We don't need o=NetscapeRoot related acis. The value of nsslapd-pluginVersion could be none.

2) If we could also provide a standard style for checking the result might be nice? E.g., the result of each operation in test_basic_ops could be checked by search, but we could just call a standard api for verifying the expected entry's existence (or not)? Having the count of the entries in a db instance would be useful, too. Maybe, we could add a doc how to add such a shared library functions to lib389?

3) clu/db2ldif_test.py is not finished yet?

    75      # test 1 function... 
    76      # test 2 function... 
    77      # ...

4) This question is more on my current task... I'm so glad you added cleanallruv testsuite. I'm now hoping to add a test case to check shutting down the servers while running the tasks (both cleanallruv and cleanruv_abort) does not crash the servers (ticket48005), do you think I could piggyback on the testsuite to check it? Or we'd better copy part of the testsuite and do the test separately?

389-ds-bot commented 4 years ago

Comment from mreynolds (@mreynolds389) at 2015-02-14 06:05:06

Replying to [comment:3 nhosoi]:

Looks great. A couple of trivial comments/quiestions...

1) It'd be nice to have a clean dse.ldif in the repo: We don't need these attributes: creatorsName, modifiersName, createTimestamp, modifyTimestamp, numSubordinates. We don't need o=NetscapeRoot related acis. The value of nsslapd-pluginVersion could be none.

I can clean it up. It is a broken dse.ldif that is meant to stop the server from starting up. It probably only needs one entry :-)

2) If we could also provide a standard style for checking the result might be nice?

You mean checking for an exception?

E.g., the result of each operation in test_basic_ops could be checked by search, but we could just call a standard api for verifying the expected entry's existence (or not)?
Having the count of the entries in a db instance would be useful, too.

Like a objectclass=* search on a backend?

Sorry I'm not sure I'm fully understanding your questions

Maybe, we could add a doc how to add such a shared library functions to lib389?

Like how to add a function to utils.py, or to the DirSrv object?

3) clu/db2ldif_test.py is not finished yet?

No, not yet, I was going to start writing skeleton scripts, but decided to wait. This patch is already quite large. I almost saw this patch as phase 1 of many phases.

  75      # test 1 function... 
  76      # test 2 function... 
  77      # ...

4) This question is more on my current task... I'm so glad you added cleanallruv testsuite. I'm now hoping to add a test case to check shutting down the servers while running the tasks (both cleanallruv and cleanruv_abort) does not crash the servers (ticket48005), do you think I could piggyback on the testsuite to check it? Or we'd better copy part of the testsuite and do the test separately?

It should definitely be in the suite, because that is also a feature of cleanallruv. You can stop the server in the middle of task, and it should resume at server startup. If you don't mind, I'll write that testcase(unless you already have one).

389-ds-bot commented 4 years ago

Comment from nhosoi (@nhosoi) at 2015-02-14 06:29:16

I can clean it up. It is a broken dse.ldif that is meant to stop the server from starting up. It probably only needs one entry :-)

Ah, I see. I thought it could be used as a template. If it is just a broken dse.ldif, never mind!

You mean checking for an exception?

Well, my thought is more basic. Like an added entry is really found in the db or a moved entry is really moved to the new location, or ... that level.

Test cases are often formed like: set up a test env do some test get ther result verify the result (e.g., comparing the result with some expected result) I thought the verification part could be somehow formalized. But probably, it depends upon each test case.

Rather "Like how to add a function to utils.py, or to the DirSrv? object?" <-- this would work better -- sharing handy utilities!!

If you don't mind, I'll write that testcase(unless you already have one).

Thank you soooooo much, Mark!! I'm still working on the standalone cases. If you could cover the cleanruv tasks, I'd be much more confident on the patch being added!

389-ds-bot commented 4 years ago

Comment from mreynolds (@mreynolds389) at 2015-02-17 22:58:46

$ py.test -v =============================== test session starts === platform linux2 -- Python 2.7.5 -- pytest-2.4.2 -- /usr/bin/python collected 61 items

basic/basic_test.py:72: test_basic_init PASSED basic/basic_test.py:89: test_basic_ops PASSED basic/basic_test.py:204: test_basic_import_export PASSED basic/basic_test.py:270: test_basic_backup PASSED basic/basic_test.py:306: test_basic_acl PASSED basic/basic_test.py:427: test_basic_searches PASSED basic/basic_test.py:465: test_basic_referrals PASSED basic/basic_test.py:526: test_basic_systemctl PASSED basic/basic_test.py:607: test_basic_ldapagent PASSED basic/basic_test.py:642: test_basic_dse PASSED basic/basic_test.py:662: test_basic_final PASSED betxns/betxn_test.py:51: test_betxn_init PASSED betxns/betxn_test.py:60: test_betxt_7bit PASSED betxns/betxn_test.py:115: test_betxn_attr_uniqueness PASSED betxns/betxn_test.py:169: test_betxn_final PASSED clu/clu_test.py:51: test_clu_init PASSED clu/clu_test.py:59: test_clu_pwdhash PASSED clu/clu_test.py:84: test_clu_final PASSED clu/db2ldif_test.py:51: test_db2ldif_init PASSED clu/db2ldif_test.py:59: test_db2ldif_final PASSED config/config_test.py:51: test_config_init PASSED config/config_test.py:58: test_config_listen_backport_size PASSED config/config_test.py:109: test_config_deadlock_policy PASSED config/config_test.py:164: test_config_final PASSED dynamic-plugins/test_dynamic_plugins.py:74: test_dynamic_plugins PASSED dynamic-plugins/test_dynamic_plugins.py:457: test_dynamic_plugins_final PASSED filter/filter_test.py:51: test_filter_init PASSED filter/filter_test.py:58: test_filter_escaped PASSED filter/filter_test.py:102: test_filter_search_original_attrs PASSED filter/filter_test.py:124: test_filter_final PASSED memory_leaks/range_search_test.py:52: test_range_search_init PASSED memory_leaks/range_search_test.py:76: test_range_search PASSED memory_leaks/range_search_test.py:128: test_range_search_final PASSED password/password_test.py:51: test_password_init PASSED password/password_test.py:59: test_password_delete_specific_password PASSED password/password_test.py:118: test_password_final PASSED password/pwdAdmin_test.py:62: test_pwdAdmin_init PASSED password/pwdAdmin_test.py:173: test_pwdAdmin PASSED password/pwdAdmin_test.py:388: test_pwdAdmin_final PASSED password/pwdPolicy_test.py:51: test_pwdPolicy_init PASSED password/pwdPolicy_test.py:58: test_pwdPolicy_final PASSED replication/cleanallruv_test.py:456: test_cleanallruv_init PASSED replication/cleanallruv_test.py:519: test_cleanallruv_clean PASSED replication/cleanallruv_test.py:643: test_cleanallruv_clean_restart PASSED replication/cleanallruv_test.py:791: test_cleanallruv_clean_force PASSED replication/cleanallruv_test.py:930: test_cleanallruv_abort PASSED replication/cleanallruv_test.py:1040: test_cleanallruv_abort_restart PASSED replication/cleanallruv_test.py:1157: test_cleanallruv_abort_certify PASSED replication/cleanallruv_test.py:1294: test_cleanallruv_stress_clean PASSED replication/cleanallruv_test.py:1460: test_cleanallruv_final PASSED rootdn_plugin/rootdn_plugin_test.py:54: test_rootdn_init PASSED rootdn_plugin/rootdn_plugin_test.py:113: test_rootdn_access_specific_time PASSED rootdn_plugin/rootdn_plugin_test.py:195: test_rootdn_access_day_of_week PASSED rootdn_plugin/rootdn_plugin_test.py:280: test_rootdn_access_denied_ip PASSED rootdn_plugin/rootdn_plugin_test.py:351: test_rootdn_access_denied_host PASSED rootdn_plugin/rootdn_plugin_test.py:421: test_rootdn_access_allowed_ip PASSED rootdn_plugin/rootdn_plugin_test.py:495: test_rootdn_access_allowed_host PASSED rootdn_plugin/rootdn_plugin_test.py:568: test_rootdn_config_validate PASSED rootdn_plugin/rootdn_plugin_test.py:746: test_rootdn_final PASSED schema/test_schema.py:152: test_schema_comparewithfiles PASSED schema/test_schema.py:196: test_schema_final PASSED

======================= 61 passed in 1296.17 seconds =====

389-ds-bot commented 4 years ago

Comment from mreynolds (@mreynolds389) at 2015-02-17 23:51:00

revision 0001-Ticket-48003-build-suite-framework.patch

389-ds-bot commented 4 years ago

Comment from nhosoi (@nhosoi) at 2015-02-18 04:37:21

Very nice to see all the tests PASSED!

So, if we add a "new" directory in the suites and "new_test.py" under it. Is it executed automagically? dirsrvtests/suites/new/new_test.py

389-ds-bot commented 4 years ago

Comment from mreynolds (@mreynolds389) at 2015-02-18 06:03:14

Replying to [comment:8 nhosoi]:

Very nice to see all the tests PASSED!

So, if we add a "new" directory in the suites and "new_test.py" under it. Is it executed automagically? dirsrvtests/suites/new/new_test.py

Yes, if you use py.test. I ran "py.test -v" in /ds/dirsrvtests/suites - it recursively looks for all the files with "test" in their names, and executes them. If I would run "py.test -v" in ds/dirsrvtests, it would also execute all the ticket tests as well.

389-ds-bot commented 4 years ago

Comment from nhosoi (@nhosoi) at 2015-02-18 06:13:03

Replying to [comment:9 mreynolds389]:

Replying to [comment:8 nhosoi]:

Very nice to see all the tests PASSED!

So, if we add a "new" directory in the suites and "new_test.py" under it. Is it executed automagically? dirsrvtests/suites/new/new_test.py

Yes, if you use py.test. I ran "py.test -v" in /ds/dirsrvtests/suites - it recursively looks for all the files with "test" in their names, and executes them. If I would run "py.test -v" in ds/dirsrvtests, it would also execute all the ticket tests as well.

Beautiful! Thanks, Mark!!

389-ds-bot commented 4 years ago

Comment from mreynolds (@mreynolds389) at 2015-02-18 06:15:35

1557df5..d33676f master -> master commit d33676f840672f6377ccdcce93c0db38039028ff Author: Mark Reynolds mreynolds389@redhat.com Date: Tue Feb 17 12:18:50 2015 -0500

6a315fe..8e44437 389-ds-base-1.3.3 -> 389-ds-base-1.3.3 commit 8e44437fc8afbf331cc0fafdb43ae78101d648c8

389-ds-bot commented 4 years ago

Comment from mreynolds (@mreynolds389) at 2015-02-19 03:34:13

Reopening, need to add template scripts for all the plugins, and common functional areas. That way its more obvious where community members should add tests.

389-ds-bot commented 4 years ago

Comment from mreynolds (@mreynolds389) at 2015-02-19 04:33:42

add script templates 0001-Ticket-48003-add-template-scripts.patch

389-ds-bot commented 4 years ago

Comment from mreynolds (@mreynolds389) at 2015-02-19 04:41:55

To ssh://git.fedorahosted.org/git/389/ds.git 430410d..6dcc12c master -> master commit 6dcc12ca1444889769be3341e180b236727ac857 Author: Mark Reynolds mreynolds389@redhat.com Date: Wed Feb 18 17:32:05 2015 -0500

8e44437..20888a6 389-ds-base-1.3.3 -> 389-ds-base-1.3.3 commit 20888a606b1eabf79723f247542a17c87dcf354d

389-ds-bot commented 4 years ago

Comment from mreynolds (@mreynolds389) at 2016-02-13 04:52:07

Milestone lib389 1.0 deleted

389-ds-bot commented 4 years ago

Comment from mreynolds (@mreynolds389) at 2017-02-11 23:04:07

Metadata Update from @mreynolds389: