se2p / pynguin

The PYthoN General UnIt Test geNerator is a test-generation tool for Python
https://www.pynguin.eu
MIT License
1.22k stars 74 forks source link

No signature found error when generating tests on a Tensorflow file #59

Closed BergLucas closed 4 months ago

BergLucas commented 4 months ago

Describe the bug When I try to generate tests for the tf_import_time.py file in the Tensorflow project, Pynguin crashes.

To Reproduce Steps to reproduce the behaviour:

  1. Use Pynguin version '0.34.0'
  2. Use the following (minimal) code as a subject for test generation:

The problem seems to come from a C extension module so I think that the easiest way to reproduce the bug is to clone the tensorflow git repository and to install it from there.

  1. Use the following command line arguments to Pynguin:
--module-name tf_import_time
--project-path <path-to-tensorflow-git-directory>/tensorflow/python/tools/
--output-path <output-name>
  1. Give the error (stack trace, etc) you are encountering:
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /home/lucas/.conda/envs/2324-master-thesis/bin/pynguin:8 in <module>                             │
│                                                                                                  │
│   5 from pynguin.cli import main                                                                 │
│   6 if __name__ == '__main__':                                                                   │
│   7 │   sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])                         │
│ ❱ 8 │   sys.exit(main())                                                                         │
│   9                                                                                              │
│                                                                                                  │
│ /home/lucas/Documents/GitHub/pynguin/src/pynguin/cli.py:193 in main                              │
│                                                                                                  │
│   190 │   set_configuration(parsed.config)                                                       │
│   191 │   if console is not None:                                                                │
│   192 │   │   with console.status("Running Pynguin..."):                                         │
│ ❱ 193 │   │   │   return run_pynguin().value                                                     │
│   194 │   else:                                                                                  │
│   195 │   │   return run_pynguin().value                                                         │
│   196                                                                                            │
│                                                                                                  │
│ /home/lucas/Documents/GitHub/pynguin/src/pynguin/generator.py:108 in run_pynguin                 │
│                                                                                                  │
│   105 │   """                                                                                    │
│   106 │   try:                                                                                   │
│   107 │   │   _LOGGER.info("Start Pynguin Test Generation…")                                     │
│ ❱ 108 │   │   return _run()                                                                      │
│   109 │   finally:                                                                               │
│   110 │   │   _LOGGER.info("Stop Pynguin Test Generation…")                                      │
│   111                                                                                            │
│                                                                                                  │
│ /home/lucas/Documents/GitHub/pynguin/src/pynguin/generator.py:507 in _run                        │
│                                                                                                  │
│   504                                                                                            │
│   505                                                                                            │
│   506 def _run() -> ReturnCode:                                                                  │
│ ❱ 507 │   if (setup_result := _setup_and_check()) is None:                                       │
│   508 │   │   return ReturnCode.SETUP_FAILED                                                     │
│   509 │   executor, test_cluster, constant_provider = setup_result                               │
│   510 │   # traces slices for test cases after execution                                         │
│                                                                                                  │
│ /home/lucas/Documents/GitHub/pynguin/src/pynguin/generator.py:258 in _setup_and_check            │
│                                                                                                  │
│   255 │                                                                                          │
│   256 │   # Analyzing the SUT should not cause any coverage.                                     │
│   257 │   tracer.disable()                                                                       │
│ ❱ 258 │   if (test_cluster := _setup_test_cluster()) is None:                                    │
│   259 │   │   return None                                                                        │
│   260 │   tracer.enable()                                                                        │
│   261                                                                                            │
│                                                                                                  │
│ /home/lucas/Documents/GitHub/pynguin/src/pynguin/generator.py:114 in _setup_test_cluster         │
│                                                                                                  │
│   111                                                                                            │
│   112                                                                                            │
│   113 def _setup_test_cluster() -> ModuleTestCluster | None:                                     │
│ ❱ 114 │   test_cluster = generate_test_cluster(                                                  │
│   115 │   │   config.configuration.module_name,                                                  │
│   116 │   │   config.configuration.type_inference.type_inference_strategy,                       │
│   117 │   │   query_type4py=config.configuration.type_inference.type4py,                         │
│                                                                                                  │
│ /home/lucas/Documents/GitHub/pynguin/src/pynguin/analyses/module.py:1447 in                      │
│ generate_test_cluster                                                                            │
│                                                                                                  │
│   1444 │   Returns:                                                                              │
│   1445 │   │   A new test cluster for the given module                                           │
│   1446 │   """                                                                                   │
│ ❱ 1447 │   return analyse_module(                                                                │
│   1448 │   │   parse_module(module_name, query_type4py=query_type4py),                           │
│   1449 │   │   type_inference_strategy,                                                          │
│   1450 │   │   query_type4py=query_type4py,                                                      │
│                                                                                                  │
│ /home/lucas/Documents/GitHub/pynguin/src/pynguin/analyses/module.py:1422 in analyse_module       │
│                                                                                                  │
│   1419 │   │   A test cluster for the module                                                     │
│   1420 │   """                                                                                   │
│   1421 │   test_cluster = ModuleTestCluster(linenos=parsed_module.linenos)                       │
│ ❱ 1422 │   __resolve_dependencies(                                                               │
│   1423 │   │   root_module=parsed_module,                                                        │
│   1424 │   │   type_inference_strategy=type_inference_strategy,                                  │
│   1425 │   │   test_cluster=test_cluster,                                                        │
│                                                                                                  │
│ /home/lucas/Documents/GitHub/pynguin/src/pynguin/analyses/module.py:1294 in                      │
│ __resolve_dependencies                                                                           │
│                                                                                                  │
│   1291 │   │   │   continue                                                                      │
│   1292 │   │                                                                                     │
│   1293 │   │   # Analyze all classes found in the current module                                 │
│ ❱ 1294 │   │   __analyse_included_classes(                                                       │
│   1295 │   │   │   module=current_module,                                                        │
│   1296 │   │   │   root_module_name=root_module.module_name,                                     │
│   1297 │   │   │   type_inference_strategy=type_inference_strategy,                              │
│                                                                                                  │
│ /home/lucas/Documents/GitHub/pynguin/src/pynguin/analyses/module.py:1354 in                      │
│ __analyse_included_classes                                                                       │
│                                                                                                  │
│   1351 │   │                                                                                     │
│   1352 │   │   type_info = test_cluster.type_system.to_type_info(current)                        │
│   1353 │   │                                                                                     │
│ ❱ 1354 │   │   __analyse_class(                                                                  │
│   1355 │   │   │   type_info=type_info,                                                          │
│   1356 │   │   │   type_inference_strategy=type_inference_strategy,                              │
│   1357 │   │   │   module_tree=parse_results[current.__module__].syntax_tree,                    │
│                                                                                                  │
│ /home/lucas/Documents/GitHub/pynguin/src/pynguin/analyses/module.py:1110 in __analyse_class      │
│                                                                                                  │
│   1107 │   else:                                                                                 │
│   1108 │   │   generic = GenericConstructor(                                                     │
│   1109 │   │   │   type_info,                                                                    │
│ ❱ 1110 │   │   │   test_cluster.type_system.infer_type_info(                                     │
│   1111 │   │   │   │   type_info.raw_type.__init__,                                              │
│   1112 │   │   │   │   type4py_data=find_predicted_signature(                                    │
│   1113 │   │   │   │   │   type4py_data, type_info.qualname + ".__init__", type_info.qualname    │
│                                                                                                  │
│ /home/lucas/Documents/GitHub/pynguin/src/pynguin/analyses/typesystem.py:1529 in infer_type_info  │
│                                                                                                  │
│   1526 │   │   """                                                                               │
│   1527 │   │   match type_inference_strategy:                                                    │
│   1528 │   │   │   case config.TypeInferenceStrategy.TYPE_HINTS:                                 │
│ ❱ 1529 │   │   │   │   return self.infer_signature(                                              │
│   1530 │   │   │   │   │   method, type4py_data, self.type_hints_provider                        │
│   1531 │   │   │   │   )                                                                         │
│   1532 │   │   │   case config.TypeInferenceStrategy.NONE:                                       │
│                                                                                                  │
│ /home/lucas/Documents/GitHub/pynguin/src/pynguin/analyses/typesystem.py:1593 in infer_signature  │
│                                                                                                  │
│   1590 │   │   Returns:                                                                          │
│   1591 │   │   │   The inference result                                                          │
│   1592 │   │   """                                                                               │
│ ❱ 1593 │   │   method_signature = inspect.signature(method)                                      │
│   1594 │   │   hints = type_hint_provider(method)                                                │
│   1595 │   │   parameters: dict[str, ProperType] = {}                                            │
│   1596 │   │   type4py_parameters: dict[str, list[ProperType]] = {}                              │
│                                                                                                  │
│ /home/lucas/.conda/envs/2324-master-thesis/lib/python3.10/inspect.py:3254 in signature           │
│                                                                                                  │
│   3251                                                                                           │
│   3252 def signature(obj, *, follow_wrapped=True, globals=None, locals=None, eval_str=False):    │
│   3253 │   """Get a signature object for the passed callable."""                                 │
│ ❱ 3254 │   return Signature.from_callable(obj, follow_wrapped=follow_wrapped,                    │
│   3255 │   │   │   │   │   │   │   │      globals=globals, locals=locals, eval_str=eval_str)     │
│   3256                                                                                           │
│   3257                                                                                           │
│                                                                                                  │
│ /home/lucas/.conda/envs/2324-master-thesis/lib/python3.10/inspect.py:3002 in from_callable       │
│                                                                                                  │
│   2999 │   def from_callable(cls, obj, *,                                                        │
│   3000 │   │   │   │   │     follow_wrapped=True, globals=None, locals=None, eval_str=False):    │
│   3001 │   │   """Constructs Signature for the given callable object."""                         │
│ ❱ 3002 │   │   return _signature_from_callable(obj, sigcls=cls,                                  │
│   3003 │   │   │   │   │   │   │   │   │   │   follow_wrapper_chains=follow_wrapped,             │
│   3004 │   │   │   │   │   │   │   │   │   │   globals=globals, locals=locals, eval_str=eval_st  │
│   3005                                                                                           │
│                                                                                                  │
│ /home/lucas/.conda/envs/2324-master-thesis/lib/python3.10/inspect.py:2468 in                     │
│ _signature_from_callable                                                                         │
│                                                                                                  │
│   2465 │   │   │   │   │   │   │   │   │   │   globals=globals, locals=locals, eval_str=eval_st  │
│   2466 │                                                                                         │
│   2467 │   if _signature_is_builtin(obj):                                                        │
│ ❱ 2468 │   │   return _signature_from_builtin(sigcls, obj,                                       │
│   2469 │   │   │   │   │   │   │   │   │      skip_bound_arg=skip_bound_arg)                     │
│   2470 │                                                                                         │
│   2471 │   if isinstance(obj, functools.partial):                                                │
│                                                                                                  │
│ /home/lucas/.conda/envs/2324-master-thesis/lib/python3.10/inspect.py:2275 in                     │
│ _signature_from_builtin                                                                          │
│                                                                                                  │
│   2272 │                                                                                         │
│   2273 │   s = getattr(func, "__text_signature__", None)                                         │
│   2274 │   if not s:                                                                             │
│ ❱ 2275 │   │   raise ValueError("no signature found for builtin {!r}".format(func))              │
│   2276 │                                                                                         │
│   2277 │   return _signature_fromstr(cls, func, s, skip_bound_arg)                               │
│   2278                                                                                           │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
ValueError: no signature found for builtin <instancemethod __init__ at 0x7f1c03e45960>

Expected behavior Pynguin should have tried to generate a test for file.

Software Version

Additional context I'm trying to reproduce the results of this paper, that is why it is a very specific case.

stephanlukasczyk commented 4 months ago

Closed by merging #60 in 8ff615d