search

Azure / Batch

Microsoft Azure Batch
https://aka.ms/batch
Creative Commons Attribution 4.0 International
24 stars 20 forks source link

DateTime Encoding Degradation on Tasks API #176

Open vKeanDev opened 1 month ago

vKeanDev commented 1 month ago

Problem Description

When getting tasks from the Tasks API, if the request contains an OData filter with a DateTime that has a timezone offset, the encoding on this DateTime degrades until the request becomes malformed when there are successive requests.

Example filter - stateTransitionTime ge DateTime'2024-07-13T08:57:48.6342861 00:00'

Initial successful request: DateTime'2023-07-13T08%3A57%3A48.6342861%2B00%3A00' 1st response (successful): DateTime%272023-07-13T08%3A57%3A48.6342861%2B00%3A00%27 (note the single quotes have been re-encoded to %27) 2nd response (error): DateTime'2023-07-13T08:57:48.6342861+00:00 (the + is no longer encoded)

The + breaks the request.

The documentation here https://learn.microsoft.com/en-us/rest/api/batchservice/task/list?tabs=HTTP suggests stateTransitionTime is returned as UTC time which does not have a timezone offset but this is only true if the initial request is in UTC time.

Steps to Reproduce

1st Request (successful)

Use the task API to query for all tasks on a job (an example request is included below). Include an OData filter for a DateTime and set the max returned items to a low enough amount so there will be at least 3 pages of results.

Send this request and check the response for an odata.nextlink for a link to the next page of results. This will have a skiptoken with it.

2nd Request (successful)

Use this next link to request the next page of results. This will succeed and will return another odata.nextlink that contains another skiptoken.

3rd Request (errors)

Use the next link provided to try and request the next page of results. This request will fail with a Bad Request as the DateTime filter is no longer properly encoded.

Expected Results

The nextLink should always contain a valid HTTP request and no errors should be returned.

Actual Results

The 3rd nextLink is not a valid HTTP request and returns an error. It looks like the + between the date and the time breaks the HTTP request.

Additional Logs

The initial request should look something like https://[BatchAccount].[Location].batch.azure.com/jobs/[JobId]/tasks?api-version=2024-02-01.19.0&maxresults=10&%24select=id%2Curl%2CdisplayName%2CcreationTime%2Cstate%2CexecutionInfo%2CnodeInfo&paginationeffort=1&_=1723809415756&%24filter=stateTransitionTime%20gt%20DateTime'2023-07-13T08%3A57%3A48.6342861%2000%3A00'

image

The response for the first 2 requests should contain a response with an odata.nextlink image

I found it easiest to navigate to a Batch Job on the Azure Portal and use developer tools to edit & resend the packets as this will ensure the headers are set up correctly.

Additional Comments

Please message me privately if you require examples or screenshots.

zhaoyiluo commented 1 week ago

@vKeanDev , this is Zhaoyi from the Batch team. I am currently attempting to reproduce the issue using the steps you provided, but I have not been successful so far.

I created a test account with testing jobs and tasks to trigger pagination on September 23rd. I tried to reproduce the issue using both Edge and Firefox through Azure Portal - Developer Tools - Edit and Resend, but both attempts failed. Let me detail my steps for you. Please let me know if there is anything I might be missing.

For Edge, I was using the filter, stateTransitionTime gt DateTime'2023-07-13T08:57:48.6342861 00:00' , that I put in the filter query column.

From the screenshot below, you can see that I am using the same encoded filter query as shown at the end of your post, with the same API version. However, I received a 400 Bad Request error, indicating that the filter was mistaken.

Request URL - https://zhaoyiluo.francecentral.batch.azure.com/jobs/zhaoyiluo/tasks?api-version=2024-02-01.19.0&maxresults=1&%24select=id%2Curl%2CdisplayName%2CcreationTime%2Cstate%2CexecutionInfo%2CnodeInfo&%24filter=stateTransitionTime%20gt%20DateTime%272023-07-13T08%3A57%3A48.6342861%2000%3A00%27&paginationeffort=1&_=1727108395530

image

I followed the same steps for Firefox, but I encountered the same result as for Edge.

Here are a few questions I would appreciate your help with:

Q1: How did you initially discover this issue? Was it through the Azure Portal, Postman, Azure Batch SDK, or another method?

Q2: Could you verify if my steps align with your suggestions?

Q3: If the issue persists on your end, is it possible for you to create a test account and provide me with detailed information such as the time, account, and task details?

zhaoyiluo commented 6 days ago

OK, I have reproduced the issue. The problem arises when the plus character is directly encoded in the filter query param and carried along with the URL. I have conducted a few tests directly with Bruno.

For the 4th scenario where I was able to reproduce the issue, the encoded URL resembles what @vKeanDev mentioned in Initial successful request. I will continue working on the fix.

Scenario 1: DateTime is not encoded and uses the space character as the delimiter

Scenario 2: DateTime is encoded and uses the space character as the delimiter

Scenario 3: DateTime is not encoded and uses the plus character as the delimiter

Scenario 4: DateTime is encoded and uses the plus character as the delimiter