search

aivillage / llm_verification

LLM prompt attacks for hacker CTFs via CTFd.
https://aivillage.org
8 stars 1 forks source link

Pretty sure there's multiple issues with the logger #21

Closed canyon289 closed 8 months ago

canyon289 commented 11 months ago

A logger is configured in llmv_logger.py but then I'm pretty sure its not correctly referenced in other models.

When we get the init we have A logger that is referenced using the module 'namel here https://github.com/aivillage/llm_verification/blob/ad0425922616ab7779720f38daf69af5cfcbf8c8/__init__.py#L14

but then a separate logger is configured and returned here https://github.com/aivillage/llm_verification/blob/main/__init__.py#L20

but then in every other model a different logger is refernenced because __name__ is not the same. These new loggers don't properly write to the log file or print to console.

Still figuring out how best to go about this. Perhaps instead of using module name we just create a single logger with the name llmv?

canyon289 commented 11 months ago

Method to reproduce.

  1. Start CTFD. Initial info and debug logs will print from __init__ and llmv_logger
  2. Try making a submission from front end (or requests or whatever)
  3. Note that info or debug logs from llmv_routes or remote_llm don't appear, only error and exception level logs do

image

canyon289 commented 10 months ago

The lack of logging in both logfile and console is a still huge pain when trying to debug. Noting as I'm more motivated to figure this out

image

canyon289 commented 8 months ago

Closed by https://github.com/aivillage/llm_verification/pull/28