search

CDLUC3 / ezid

CDLUC3 ezid
MIT License
10 stars 4 forks source link

[BUG] Investigate ezid-dev errors 2024-10-24 #773

Open jsjiang opened 4 weeks ago

jsjiang commented 4 weeks ago

On 2024-10-24, EZID-Dev started to report errors from 7:07am in the morning. All are Internal Server Error on the following paths:

/, /login, /demo/simple, /search/results

Log shows the requests came from IP address 172.31.59.115 (this is the load balancer). The X-Forwarded-For "111.90.150.109"

Here are sample request from the Apache access log (ezid_access.log):

172.31.59.115 - - [24/Oct/2024:08:38:29 -0700] "GET /status HTTP/1.1" 200 19 "-" "ELB-HealthChecker/2.0"
172.31.59.115 - - [24/Oct/2024:08:38:59 -0700] "GET /status HTTP/1.1" 200 19 "-" "ELB-HealthChecker/2.0"
172.31.59.115 - - [24/Oct/2024:08:39:30 -0700] "GET /status HTTP/1.1" 200 19 "-" "ELB-HealthChecker/2.0"
172.31.59.115 - - [24/Oct/2024:08:40:00 -0700] "GET /status HTTP/1.1" 200 19 "-" "ELB-HealthChecker/2.0"
172.31.59.115 - - [24/Oct/2024:08:40:28 -0700] "GET / HTTP/1.1" 200 21579 "-" "-"
172.31.59.115 - - [24/Oct/2024:08:40:30 -0700] "GET /status HTTP/1.1" 200 19 "-" "ELB-HealthChecker/2.0"
172.31.59.115 - - [24/Oct/2024:08:40:32 -0700] "GET / HTTP/1.1" 200 21579 "-" "Mozilla/5.0 (compatible; CensysInspect/1.1; +https://about.censys.io/)"
172.31.59.115 - - [24/Oct/2024:08:40:33 -0700] "GET /static/images/favicon.ico?v=2 HTTP/1.1" 200 76046 "-" "Mozilla/5.0 (compatible; CensysInspect/1.1; +https://about.censys.io/)"
172.31.59.115 - - [24/Oct/2024:08:40:34 -0700] "GET /favicon.ico HTTP/1.1" 200 76046 "-" "Mozilla/5.0 (compatible; CensysInspect/1.1; +https://about.censys.io/)"
172.31.59.115 - - [24/Oct/2024:08:40:51 -0700] "GET /search?c_creator=t&c_identifier=t&c_object_type=t&c_publisher=t&c_pubyear=t&c_title=t&creator=1&filtered=t&id_type=doi&identifier=317&keywor
ds=the&modify_search=t&object_type=Audiovisual&publisher=1&pubyear_from=1967&pubyear_to=1967&submit_checks=response.write(9568943*9341939)&title=Mr. HTTP/1.1" 200 23900 "-" "Mozilla/5.0 (Window
s NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36"
172.31.59.115 - - [24/Oct/2024:08:40:52 -0700] "GET /search?c_creator=t&c_identifier=t&c_object_type=t&c_publisher=t&c_pubyear=http://dicrpdbjmemujemfyopp.zzz/yrphmgdpgulaszriylqiipemefmacafkxy
cjaxjs%3F.jpg&c_title=t&creator=1&filtered=t&id_type=doi&identifier=317&keywords=the&modify_search=t&object_type=Audiovisual&publisher=1&pubyear_from=1967&pubyear_to=1967&submit_checks=submit_c
hecks=Apply%20Changes&title=Mr. HTTP/1.1" 200 23976 "https://ezid-dev.cdlib.org/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.3
6"
172.31.59.115 - - [24/Oct/2024:08:40:52 -0700] "GET /search?c_creator=t&c_identifier=t&c_object_type=t&c_publisher=t&c_pubyear=t&c_title=t&creator=1&filtered=t&id_type=doi&identifier=317&keywor
ds=the&modify_search=t&object_type=Audiovisual&publisher=&pubyear_from=1967&pubyear_to=1967&submit_checks=submit_checks=Apply%20Changes&title=Mr. HTTP/1.1" 200 23887 "https://ezid-dev.cdlib.org
/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36"
172.31.59.115 - - [24/Oct/2024:08:40:52 -0700] "GET /search?c_creator=t&c_identifier=t&c_object_type=t&c_publisher=t&c_pubyear=t&c_title=t&creator=1&filtered=t&id_type=doi&identifier=317&keywor
ds=the&modify_search=t&object_type=Audiovisual&publisher=1&pubyear_from=1967&pubyear_to=1967&submit_checks='%2Bresponse.write(9568943*9341939)%2B'&title=Mr. HTTP/1.1" 200 23908 "-" "Mozilla/5.0
 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36"
172.31.59.115 - - [24/Oct/2024:08:40:52 -0700] "GET /search?c_creator=t&c_identifier=t&c_object_type=t&c_publisher=t&c_pubyear=t&c_title=!(()%26%26!%7C*%7C*%7C&creator=1&filtered=t&id_type=doi&
identifier=317&keywords=the&modify_search=t&object_type=Audiovisual&publisher=1&pubyear_from=1967&pubyear_to=1967&submit_checks=submit_checks=Apply%20Changes&title=Mr. HTTP/1.1" 200 23919 "http
s://ezid-dev.cdlib.org/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36"
172.31.59.115 - - [24/Oct/2024:08:40:53 -0700] "GET /search?c_creator=t&c_identifier=t&c_object_type=t&c_publisher=t&c_pubyear=t&c_title=t&creator=1&filtered=t&id_type=doi&identifier=317&keywor
ds=the&modify_search=t&object_type=Audiovisual&publisher=1&pubyear_from=1967&pubyear_to=1967&submit_checks=submit_checks=Apply%20Changes&title=Mr. HTTP/1.1" 200 23898 "-" "Mozilla/5.0 (Windows 
NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36"
172.31.59.115 - - [24/Oct/2024:08:40:53 -0700] "GET /search?c_creator=t&c_identifier=t&c_object_type=t&c_publisher=t&c_pubyear=1yrphmgdpgulaszriylqiipemefmacafkxycjaxjs%00.jpg&c_title=t&creator
=1&filtered=t&id_type=doi&identifier=317&keywords=the&modify_search=t&object_type=Audiovisual&publisher=1&pubyear_from=1967&pubyear_to=1967&submit_checks=submit_checks=Apply%20Changes&title=Mr.
 HTTP/1.1" 200 23945 "https://ezid-dev.cdlib.org/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36"
172.31.59.115 - - [24/Oct/2024:08:40:53 -0700] "GET /search?c_creator=t&c_identifier=t&c_object_type=t&c_publisher=t&c_pubyear=t&c_title=t&creator=1&filtered=t&id_type=doi&identifier=317&keywor
ds=the&modify_search=t&object_type=Audiovisual&publisher=12345'\"\\'\\\");|]*%00{%0d%0a<%00>%bf%27'\xf0\x9f\x92\xa1&pubyear_from=1967&pubyear_to=1967&submit_checks=submit_checks=Apply%20Changes
&title=Mr. HTTP/1.1" 200 24042 "https://ezid-dev.cdlib.org/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36"
172.31.59.115 - - [24/Oct/2024:08:40:53 -0700] "GET /search?c_creator=t&c_identifier=t&c_object_type=t&c_publisher=t&c_pubyear=t&c_title=t&creator=1&filtered=t&id_type=doi&identifier=317&keywor
ds=the&modify_search=t&object_type=Audiovisual&publisher=1&pubyear_from=1967&pubyear_to=1967&submit_checks=\"%2Bresponse.write(9568943*9341939)%2B\"&title=Mr. HTTP/1.1" 200 23912 "-" "Mozilla/5
.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36"
...

172.31.59.115 - - [24/Oct/2024:08:55:42 -0700] "POST /demo/simple HTTP/1.1" 200 21277 "https://ezid-dev.cdlib.org/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36"
172.31.59.115 - - [24/Oct/2024:08:55:43 -0700] "POST /demo/simple HTTP/1.1" 200 23998 "https://ezid-dev.cdlib.org/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36"
172.31.59.115 - - [24/Oct/2024:08:55:44 -0700] "POST /demo/simple HTTP/1.1" 200 21273 "https://ezid-dev.cdlib.org/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36"
172.31.59.115 - - [24/Oct/2024:08:55:44 -0700] "GET /demo/simple?current_profile=datacite&erc.what=2023-03-22T15:34:00Z&erc.when=1&erc.who=1&name=pHqghUme&shoulder=ark:/99999/fk4&target=10\"XOR(1*if(now()=sysdate()%2Csleep(15)%2C0))XOR\"Z HTTP/1.1" 200 23553 "https://ezid-dev.cdlib.org/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36"
172.31.59.115 - - [24/Oct/2024:08:55:44 -0700] "POST /demo/simple HTTP/1.1" 200 21283 "https://ezid-dev.cdlib.org/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36"
172.31.59.115 - - [24/Oct/2024:08:55:45 -0700] "POST /demo/simple HTTP/1.1" 200 21310 "https://ezid-dev.cdlib.org/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36"
172.31.59.115 - - [24/Oct/2024:08:55:45 -0700] "POST /demo/simple HTTP/1.1" 200 21281 "https://ezid-dev.cdlib.org/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36"
172.31.59.115 - - [24/Oct/2024:08:55:45 -0700] "GET /demo/simple?current_profile=datacite&erc.what=2023-03-22T15:34:00Z&erc.when=1&erc.who=1&shoulder=doi:10.5072/FK2'%7C%7CDBMS_PIPE.RECEIVE_MESSAGE(CHR(98)%7C%7CCHR(98)%7C%7CCHR(98)%2C15)%7C%7C'&target=http://1 HTTP/1.1" 200 23556 "https://ezid-dev.cdlib.org/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36"
172.31.59.115 - - [24/Oct/2024:08:55:46 -0700] "GET /demo/simple?current_profile=datacite&erc.what=2023-03-22T15:34:00Z&erc.when=1&erc.who=1&shoulder=doi:10.5072/FK2&target=http://1 HTTP/1.1" 200 23494 "https://ezid-dev.cdlib.org/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36"
172.31.59.115 - - [24/Oct/2024:08:55:46 -0700] "GET /demo/simple?current_profile=datacite&erc.what=2023-03-22T15:34:00Z&erc.when=1&erc.who=1&shoulder=doi:10.5072/FK2'\"&target=http://1 HTTP/1.1" 200 23481 "https://ezid-dev.cdlib.org/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36"
172.31.59.115 - - [24/Oct/2024:08:55:46 -0700] "GET /demo/simple?current_profile=datacite&erc.what=2023-03-22T15:34:00Z&erc.when=1&erc.who=1&shoulder=doi:10.5072/FK2%C0%A7%C0%A2%252527%252522%5C'%5C\"&target=http://1 HTTP/1.1" 200 23513 "https://ezid-dev.cdlib.org/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36"
172.31.59.115 - - [24/Oct/2024:08:55:48 -0700] "GET /demo/simple?current_profile=datacite&erc.what=2023-03-22T15:34:00Z&erc.when=1&erc.who=1&shoulder=%40%40psYvE&target=http://1 HTTP/1.1" 200 23473 "https://ezid-dev.cdlib.org/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36"