HXSecurity / DongTai

Dongtai IAST is an open-source Interactive Application Security Testing (IAST) tool that enables real-time detection of common vulnerabilities in Java applications and third-party components through passive instrumentation. It is particularly suitable for use in the testing phase of the development pipeline.
https://dongtai.io
Apache License 2.0
1.24k stars 145 forks source link

[Bug]: Request /api/v1/engine/method_pool/sca 500 error #477

Closed onesecurity closed 2 years ago

onesecurity commented 2 years ago

Preflight Checklist

Version

1.3.1

Installation Type

Official Kubernetes

Service Name

DongTai-WebAPI

Describe the details of the bug and the steps to reproduce it

1.Request /api/v1/engine/method_pool/sca?method_pool_id=30

image

2.K8s Log

Not running behind as many reverse proxies as expected. The right value for XFF_TRUSTED_PROXY_DEPTH for this request is 1 and 20 is configured. Not running behind as many reverse proxies as expected. The right value for XFF_TRUSTED_PROXY_DEPTH for this request is 1 and 20 is configured. [pid: 9|app: 0|req: 1227/1598] 172.xxx.xxx.xxx () {56 vars in 1596 bytes} [Sat Mar 26 01:39:35 2022] GET /api/v1/engine/graph?method_pool_id=30&method_pool_type=normal => generated 28582 bytes in 35 msecs (HTTP/1.0 200) 11 headers in 437 bytes (1 switches on core 0) url: /api/v1/engine/method_pool/sca出错,原因:(1267, "Illegal mix of collations (utf8mb4_general_ci,IMPLICIT) and (utf8mb4_0900_ai_ci,IMPLICIT) for operation '='") [pid: 9|app: 0|req: 1227/1599] 172.xxx.xxx.xxx () {56 vars in 1568 bytes} [Sat Mar 26 01:39:35 2022] GET /api/v1/engine/method_pool/sca?method_pool_id=30 => generated 145 bytes in 43 msecs (HTTP/1.0 500) 9 headers in 379 bytes (1 switches on core 3)

3. desc iast_agent_method_pool

DROP TABLE IF EXISTS iast_agent_method_pool; /!40101 SET @saved_cs_client = @@character_set_client /; /!40101 SET character_set_client = utf8 /; CREATE TABLE iast_agent_method_pool ( id int(11) NOT NULL AUTO_INCREMENT, agent_id int(11) DEFAULT NULL COMMENT 'Agent', url varchar(2000) DEFAULT NULL COMMENT 'URL', uri varchar(2000) DEFAULT NULL COMMENT 'URI', http_method varchar(10) DEFAULT NULL COMMENT 'HTTP请求方法', http_scheme varchar(20) DEFAULT NULL COMMENT '协议', http_protocol varchar(255) DEFAULT NULL COMMENT 'HTTP协议', req_header varchar(2000) DEFAULT NULL COMMENT '请求头', req_params varchar(2000) DEFAULT NULL COMMENT '请求参数', req_data varchar(4000) DEFAULT NULL COMMENT '请求体', res_header varchar(1000) DEFAULT NULL COMMENT '响应头', res_body text COMMENT '响应体', context_path varchar(255) DEFAULT NULL COMMENT '应用上下文', method_pool json DEFAULT NULL COMMENT '方法池', clent_ip varchar(255) DEFAULT NULL COMMENT '客户端IP', create_time int(11) DEFAULT NULL COMMENT '创建时间', update_time int(11) DEFAULT NULL COMMENT '修改时间', pool_sign varchar(40) DEFAULT NULL COMMENT '方法池签名', req_header_for_search text COMMENT '用于搜索功能的请求头,请求方法、uri、参数、协议、请求头的拼接', uri_sha1 varchar(40) NOT NULL DEFAULT '', PRIMARY KEY (id) USING BTREE, UNIQUE KEY pool_sign (pool_sign,agent_id) USING BTREE, KEY agent_id (agent_id) USING BTREE, KEY update_time (update_time) USING BTREE, KEY iast_agent__uri_sha_aac6d7_idx (uri_sha1,http_method,agent_id), KEY iast_agent_method_pool_uri_sha1_a64012e4 (uri_sha1), KEY iast_agent_method_pool_agent_id_30df78eb (agent_id), CONSTRAINT iast_agent_method_pool_ibfk_1 FOREIGN KEY (agent_id) REFERENCES iast_agent (id) ON DELETE NO ACTION ON UPDATE NO ACTION ) ENGINE=InnoDB AUTO_INCREMENT=463 DEFAULT CHARSET=utf8mb4; /!40101 SET character_set_client = @saved_cs_client /;

Additional Information

No response

Logs

No response

Bidaya0 commented 2 years ago

check you default collation. When storing HTTP data collected by the agent, we need to be compatible with as many encodings as possible, so we declare the table utf8mb4_general_ci ''' show variables where Variable_name like 'collation%'; '''