search

dice-group / Ontolearn

Ontolearn is an open-source software library for explainable structured machine learning in Python. It learns OWL class expressions from positive and negative examples.
https://ontolearn-docs-dice-group.netlify.app/index.html
MIT License
41 stars 9 forks source link

Encountering KeyError During Class Expression Learning (CEL) Evaluation #398

Closed farshad68 closed 6 months ago

farshad68 commented 6 months ago

Firstly, I want to express my gratitude for the excellent application and the prompt responses to previous issues. Thank you for your hard work!

I'm currently encountering an error while trying to utilize the Class Expression Learning (CEL) feature. Here's a breakdown of the process:

Training:

I initiate the training process by sending an HTTP request with the following parameters:

{'pos': ['http://www.wikidata.org/entity/Q3895', 'http://www.wikidata.org/entity/Q180855'], 'neg': ['http://www.wikidata.org/entity/Q1359568', 'http://www.wikidata.org/entity/Q169167', 'http://www.wikidata.org/entity/Q192334'], 'model': 'Drill', 'max_runtime': 180, 'iter_bound': 2, 'path_embeddings': '/home/shared/app2/30f40e61-3ee5-483d-afe2-50d5a18e1af2/Keci_embeddings_group1-data.csv'}

The training seems to complete successfully, as indicated by the logs provided.

######### CEL Arguments ###############
Knowledgebase/Triplestore:<ontolearn.triple_store.TripleStore object at 0x7fc04d98dbd0>
Input data: {'pos': ['http://www.wikidata.org/entity/Q3895', 'http://www.wikidata.org/entity/Q180855'], 'neg': ['http://www.wikidata.org/entity/Q1359568', 'http://www.wikidata.org/entity/Q169167', 'http://www.wikidata.org/entity/Q192334'], 'model': 'Drill', 'max_runtime': 180, 'iter_bound': 2, 'pretrained': 'pretrained', 'path_embeddings': '/home/shared/app2/30f40e61-3ee5-483d-afe2-50d5a18e1af2/Keci_embeddings_group1-data.csv'}
######### CEL Arguments ###############

Goal Concept:    Q5172544   E^+:[3]  E^-:[3]
Episode 1:  Taking 3 actions...  Runtime 0.433 secs | Max reward: 0.000 | Prob of Explore 1.000 | Experience replay Experiences (torch.Size([3, 4, 32])) | 0 Replay loss: 0.00036 | 1 Replay loss: 0.55917 | Avg loss: 0.27977
Episode 2:  Taking 3 actions...  Runtime 0.765 secs | Max reward: 0.507 | Prob of Explore 0.990 | Experience replay Experiences (torch.Size([6, 4, 32])) | 0 Replay loss: 0.23879 | 1 Replay loss: 0.06473 | Avg loss: 0.15176
Episode 3:  Taking 3 actions...  Runtime 0.875 secs | Max reward: 0.507 | Prob of Explore 0.980 | Experience replay Experiences (torch.Size([9, 4, 32])) | 0 Replay loss: 0.27447 | 1 Replay loss: 0.29755 | Avg loss: 0.28601
Episode 4:  Taking 3 actions...  Runtime 0.241 secs | Max reward: 0.000 | Prob of Explore 0.970 | Experience replay Experiences (torch.Size([12, 4, 32])) | 0 Replay loss: 0.21812 | 1 Replay loss: 0.10398 | Avg loss: 0.16105
Episode 5:  Taking 3 actions...  Runtime 0.816 secs | Max reward: 0.507 | Prob of Explore 0.960 | Experience replay Experiences (torch.Size([15, 4, 32])) | 0 Replay loss: 0.04782 | 1 Replay loss: 0.06431 | Avg loss: 0.05606
Episode 6:  Taking 3 actions...  Runtime 0.706 secs | Max reward: 0.507 | Prob of Explore 0.950 | Experience replay Experiences (torch.Size([18, 4, 32])) | 0 Replay loss: 0.11775 | 1 Replay loss: 0.13356 | Avg loss: 0.12566
Episode 7:  Taking 3 actions...  Runtime 0.830 secs | Max reward: 0.507 | Prob of Explore 0.940 | Experience replay Experiences (torch.Size([21, 4, 32])) | 0 Replay loss: 0.12483 | 1 Replay loss: 0.08740 | Avg loss: 0.10611
Episode 8:  Taking 3 actions...  Runtime 0.798 secs | Max reward: 0.507 | Prob of Explore 0.930 | Experience replay Experiences (torch.Size([24, 4, 32])) | 0 Replay loss: 0.05277 | 1 Replay loss: 0.02685 | Avg loss: 0.03981
Episode 9:  Taking 3 actions...  Runtime 33.274 secs | Max reward: 0.000 | Prob of Explore 0.920 | Experience replay Experiences (torch.Size([27, 4, 32])) | 0 Replay loss: 0.02341 | 1 Replay loss: 0.03059 | Avg loss: 0.02700
Episode 10:  Taking 3 actions...  Runtime 0.400 secs | Max reward: 0.000 | Prob of Explore 0.910 | Experience replay Experiences (torch.Size([30, 4, 32])) | 0 Replay loss: 0.03576 | 1 Replay loss: 0.03202 | Avg loss: 0.03389
Sum of rewards for each trial [0, 1.04, 0.8799999999999998, 0, 0.9599999999999999, 1.2399999999999998, 0.9999999999999999, 0.9999999999999999, 0, 0]
Training is completed..
Training over learning problems: 100%|██████████| 1/1 [00:39<00:00, 39.24s/it]

Evaluation:

Following the completion of training, I send a request for evaluation using the same parameters as the training request.

{'pos': ['http://www.wikidata.org/entity/Q3895', 'http://www.wikidata.org/entity/Q180855'], 'neg': ['http://www.wikidata.org/entity/Q1359568', 'http://www.wikidata.org/entity/Q169167', 'http://www.wikidata.org/entity/Q192334'], 'model': 'Drill', 'max_runtime': 180, 'iter_bound': 2, 'pretrained': 'pretrained', 'path_embeddings': '/home/shared/app2/30f40e61-3ee5-483d-afe2-50d5a18e1af2/Keci_embeddings_group1-data.csv'}

However, this results in an internal server error with the following traceback:

INFO:     172.19.0.5:40252 - "GET /cel HTTP/1.1" 500 Internal Server Error
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/uvicorn/protocols/http/httptools_impl.py", line 411, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "/usr/local/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 69, in __call__
    return await self.app(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/fastapi/applications.py", line 1054, in __call__
    await super().__call__(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/applications.py", line 123, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/middleware/errors.py", line 186, in __call__
    raise exc
  File "/usr/local/lib/python3.10/site-packages/starlette/middleware/errors.py", line 164, in __call__
    await self.app(scope, receive, _send)
  File "/usr/local/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 65, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "/usr/local/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 756, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 776, in app
    await route.handle(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 297, in handle
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 77, in app
    await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "/usr/local/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 72, in app
    response = await func(request)
  File "/usr/local/lib/python3.10/site-packages/fastapi/routing.py", line 278, in app
    raw_response = await run_endpoint_function(
  File "/usr/local/lib/python3.10/site-packages/fastapi/routing.py", line 191, in run_endpoint_function
    return await dependant.call(**values)
  File "/Ontolearn/ontolearn/scripts/run.py", line 91, in cel
    owl_learner = get_learner(data)
  File "/Ontolearn/ontolearn/scripts/run.py", line 74, in get_learner
    return get_drill(data)
  File "/Ontolearn/ontolearn/scripts/run.py", line 63, in get_drill
    drill.save(directory=data["path_to_pretrained_drill"])
KeyError: 'path_to_pretrained_drill'

Expected Behavior:

I expect the evaluation process to run smoothly without encountering any errors.

Potential Cause:

Upon investigation, it appears that the KeyError is triggered by an attempt to access 'path_to_pretrained_drill' in the code, which is not present in the dictionary data.

Request for Assistance:

Could you please guide me on how to resolve this issue? Any insights or suggestions would be greatly appreciated. Once again, thank you for your support, and I look forward to resolving this issue together.

Best regards, Farshad

Demirrr commented 6 months ago

Please follow the example provided in the readme. You are sending the wrong key argument. 'pretrained': 'pretrained' is the wrong key argument

`import json import requests with open(f"LPs/Mutagenesis/lps.json") as json_file: learning_problems = json.load(json_file)["problems"] for str_target_concept, examples in learning_problems.items(): response = requests.get('http://0.0.0.0:8000/cel', headers={'accept': 'application/json', 'Content-Type': 'application/json'}, json={"pos": examples['positive_examples'], "neg": examples['negative_examples'], "model": "Drill", "path_embeddings": "mutagenesis_embeddings/Keci_entity_embeddings.csv", "path_to_pretrained_drill": "pretrained_drill",

if pretrained_drill exists, upload, otherwise train one and save it there

                              "num_of_training_learning_problems": 2,
                              "num_of_target_concepts": 3,
                              "max_runtime": 60000,  # seconds
                              "iter_bound": 1  # number of iterations/applied refinement opt.
                              })
print(response.json())  # {'Prediction`

path_to_pretrained_drill": "pretrained_drill", if pretrained_drill exists, upload, otherwise train one and save it there.

I am mobile at the moment. If the issue is still unclear, I will provide a more detailed info tomorrow:)

farshad68 commented 6 months ago

Thank you for your prompt response.

farshad68 commented 6 months ago

it works thank you