AleksandrRogov / DynamicsWebApi

DynamicsWebApi is a Microsoft Dataverse Web API helper library for JavaScript & TypeScript
MIT License
272 stars 60 forks source link

fetchAll still returns no more than 5000 items #80

Closed Suxsem closed 3 years ago

Suxsem commented 3 years ago

Hi, I'm trying to retrieve all records from a fetchXml but I always get 5000 records despite i'm using the executeFetchXmlAll method. This is the code I'm using

    const fetch = `
<fetch no-lock="true">
  <entity name="alt_categoriaprodotto" >
    <attribute name="alt_name" />
    <attribute name="alt_markup" />
    <attribute name="alt_livello" />
    <attribute name="alt_categoriapadreid" />
    <order attribute="alt_name" />
  </entity>
</fetch>
      `;

    return CommonDAO.getWebApi().fetchAll("alt_categoriaprodottos", fetch);

where CommonDAO.getWebApi() returns new DynamicsWebApi({ webApiUrl: baseUrl + "/api/data/v9.0/" });

I found a similar issue here ( #79 ), but I'm not using the useEntityNames parameter.

Thank you

Suxsem commented 3 years ago

I found the issue!

I was searching through the library code when I found this line: https://github.com/AleksandrRogov/DynamicsWebApi/blob/ba89c665a55964e7d02cbc3ce470319da0b6551e/dist/dynamics-web-api.js#L1612

that contains this regex: /^(<fetch)/

The problem was that my fetchXml string actually started with a new line! I removed the starting new line and now it works.

Maybe you can think about handling this edge case.

Thank you for the library!

Suxsem commented 3 years ago

Umh, something else is wrong.

Now i get the @Microsoft.Dynamics.CRM.fetchxmlpagingcookie annotation in the response but the method always returns 5000 record.

What can i try? Thank you

Suxsem commented 3 years ago

I think there is something wrong with concurrent requests when one of them is a fetchAll.

Basically, when a fetchAll (with more then 5000 records returned) is running, if you concurrently fire another fetch the results get corrupted.

To reproduce the issue just make one fetchAll (with a fetchxml you expect to return like 20000 records or more) request followed by a normal fetch, without any "then" in order to execute them concurrently. The results of the first fetchAll will not be complete.

Can you please try?

AleksandrRogov commented 3 years ago

Yes, I found a concurrency issue in the library, even with new DynamicsWebApi objects. I'm working on a solution.

Suxsem commented 3 years ago

Great, glad you can reproduce it. Thank you for your time! Please don't hesitate to ask me if you need other information or if you want me to test something.

AleksandrRogov commented 3 years ago

@Suxsem I fixed the issue. Let me know if the new patch does not fix the issues you reported.

Suxsem commented 3 years ago

Thank you for your effort, I will try it right now

Suxsem commented 3 years ago

Hi @AleksandrRogov it's working great, thank you!