Datasource URL parameters on an asset appear to be url-encoded twice, resulting in invalid queries at the datasource.
Steps:
I have a datasource that accepts a date query parameter in the format 2023-10-13T00:00:00.000Z
Using the mds ui dashboard I added an asset that had a query parameter name: since, value: 2023-10-13T00:00:00.000Z
Set up a contract for the asset using the open policy
Negotiated and attempted a transfer using a different broker
The transfer failed, with the datasource reporting bad-request. The query received by the datasource was https://...?since=2023-10-13T00%3A00%3A00.000Z
which appeared to be the datasource not decoding the url. However, the ingress logs showed the inbound request as
indicating that the '%' char has been url encoded. Looking at the EDC database table edc_assets_dataaddress the query parameter is stored in it's url encoded form as 2023-10-13T00%3A00%3A00.000Z
So it appears that the url is being encoded twice: once when being stored in the edc database, then again when being sent to the datasource.
Expected Result
The datasource ingress receives https://...?since=2023-10-13T00%3A00%3A00.000Z, which can be correctly decoded to https://...?since=2023-10-13T00:00:00.000Z
Issue
Datasource URL parameters on an asset appear to be url-encoded twice, resulting in invalid queries at the datasource.
Steps:
2023-10-13T00:00:00.000Z
since
, value:2023-10-13T00:00:00.000Z
The transfer failed, with the datasource reporting bad-request. The query received by the datasource was
https://...?since=2023-10-13T00%3A00%3A00.000Z
which appeared to be the datasource not decoding the url. However, the ingress logs showed the inbound request as
https://...?since=2023-10-13T00%253A00%253A00.000Z
indicating that the '%' char has been url encoded. Looking at the EDC database table
edc_assets_dataaddress
the query parameter is stored in it's url encoded form as2023-10-13T00%3A00%3A00.000Z
So it appears that the url is being encoded twice: once when being stored in the edc database, then again when being sent to the datasource.
Expected Result
The datasource ingress receives
https://...?since=2023-10-13T00%3A00%3A00.000Z
, which can be correctly decoded tohttps://...?since=2023-10-13T00:00:00.000Z