search

glebovpavel / IR_to_MSExcel

Oracle Apex plugin for Interactive Grid or Interactive Report
http://glebovpavel.github.io/Description_IR_TO_XSLX/
Other
39 stars 14 forks source link

Truncation of data with very large data sets #91

Open indy2005 opened 3 years ago

indy2005 commented 3 years ago

Our customers are reporting that the Excel download does not match the interactive grid. The interactive grid is stating 57717 rows, but the Excel file ends at 53873. Have you tested this with wide large data sets? We had found this issue earlier when we had multiple grids on a single page, and had thought we had solved it by separating out the IGs onto separate pages, but the issue is persistent. Attached are several screenshots. I am using APEX 18. If we are losing data, we can't use the plug in. Although we can;t use CSV either as that doesnt work for our Swedish colleagues due to regional settings, which impacts how Excel opens the file. Any help appreciated.

In the dynamic action, we have set 100,000 row download limit across the board, so should not be an issue.

We have had similar issues with other plugins such as SmartPivot, which ended up being related to presence of certain non standard characters in the data, may not be the issue here but thought I would mention it. It is going to be dificult to provide you with a large corporate data set obviously. I may try doing a REGEX replace around some of the fields, to see if replacing some of the characters solves the problem.

IG_Export_records IG_records

glebovpavel commented 3 years ago

Hi @indy2005 ,

have tested with dummy query for 100000 rows, and i got an correct Excel with all the data.

Plugin uses 2 different rendering engines, one for IR (server-based, PL/SQL) and second for IG (mostly client based, Java Script). That means that for IG plugin uses standard interface of Interactive grid to get the data, not build custom SQL like for IR. If interface of Interactive Grid has problems to retrieve the data, plugin should have exactly the same problems - like SmartPivot. Can be that you are right and problem happens because of special symbols. I know nothing about Swedish, have tested moustly for German ÜÄÖüäöß.

indy2005 commented 3 years ago

Test data very rarely replicates the nuances of real data. There is definitely an issue, we see it in every extract over 15k. Just hoping AOP doesn’t do exactly the same. CSV would be fine, if Swedish regional settings didn’t render it all into a single column in Excel.

indy2005 commented 3 years ago

Another thing worth checking is how it performs when there are multiple grids on the same page, and how it works in combination with other reports and templates such as the badge plugin, on the same page. I know it shouldn’t affect it...but we had problems initially with the plugin which were fixed by breaking out the components onto separate pages. If I knew how to let users select their delimiter for CSV in the UI, all my problems would be solved.