apache / seatunnel

SeaTunnel is a next-generation super high-performance, distributed, massive data integration tool.
https://seatunnel.apache.org/
Apache License 2.0
7.84k stars 1.77k forks source link

The URL time parameter of HTTP needs improvement #4666

Closed wenyongning1 closed 1 year ago

wenyongning1 commented 1 year ago

Search before asking

Description

I hope seatunel will automatically convert the time parameter in the URL of hhtp from 2023-04-18 16:25:00 format to startTime=2023-04-18%2016:25:00 format

Usage Scenario

url = "http://mockserver:1080/jsonpath/mock?startTime=2023-04-18 2016:25:00&endTime=2023-04-18 2016:30:00" Seatune generated code automatically converted to url = "http://mockserver:1080/jsonpath/mock?startTime=2023-04-18%202016:25:00&endTime=2023-04-18%202016:30:00"

Related issues

Because it is necessary to dynamically transmit time parameters to obtain HTTP data, but the incoming time parameters do not comply with HTTP specifications, so seatunel needs to automatically convert them.

Are you willing to submit a PR?

Code of Conduct

kim-up commented 1 year ago

Maybe , the url needs to UrlEncode ?

kim-up commented 1 year ago

@wenyongning1 Can you offer the http connector config ?

wenyongning1 commented 1 year ago

Maybe , the url needs to UrlEncode ? Yes, currently the URL of the ST configuration file is written at 2023-04-18 16:25:00 which is an error message, must be written at 2023-04-18%2016:30:00

wenyongning1 commented 1 year ago

@wenyongning1 Can you offer the http connector config ? source { Http { url = "http://mockserver:1080/jsonpath/mock?startTime=2023-04-18%2016:25:00&endTime=2023-04-18%2016:30:00" method = "GET" } } Configure the time and time in this way, ST execution will be fine

kim-up commented 1 year ago

@wenyongning1 You can avoid this problem by putting the time parameter in params instead of splicing it after the Url. Because, in this way, the parameters can be encodedUrlForm automatically when calling Http . As follows:

source {
    Http {
    url = "http://mockserver:1080/jsonpath/mock",
    method = "GET",
    params {
         startTime = "2023-04-18 2016:25:00",
         endTime = "2023-04-18 2016:30:00"
     }
}

Eventually the entire Url will be converted to: http://mockserver:1080/jsonpath/mock?startTime=2023-04-18+2016%3A25%3A00&endTime=2023-04-18+2016%3A30%3A00

wenyongning1 commented 1 year ago

@wenyongning1 You can avoid this problem by putting the time parameter in params instead of splicing it after the Url. Because, in this way, the parameters can be encodedUrlForm automatically when calling Http . As follows:

source {
    Http {
    url = "http://mockserver:1080/jsonpath/mock",
    method = "GET",
    params {
         startTime = "2023-04-18 2016:25:00",
         endTime = "2023-04-18 2016:30:00"
     }
}

Eventually the entire Url will be converted to: http://mockserver:1080/jsonpath/mock?startTime=2023-04-18+2016%3A25%3A00&endTime=2023-04-18+2016%3A30%3A00

wenyongning1 commented 1 year ago

@wenyongning1 You can avoid this problem by putting the time parameter in `params` instead of splicing it after the Url. Because, in this way, the parameters can be encodedUrlForm automatically when calling `Http` . As follows:

source { Http { url = "http://mockserver:1080/jsonpath/mock", method = "GET", params { startTime = "2023-04-18 2016:25:00", endTime = "2023-04-18 2016:30:00" } }



Eventually the entire Url will be converted to: http://mockserver:1080/jsonpath/mock?startTime=2023-04-18+2016%3A25%3A00&endTime=2023-04-18+2016%3A30%3A00
ok,Thank you.