sbcgua / ajson

Yet another json parser serializer for ABAP
MIT License
49 stars 15 forks source link

performance question - slice & to_abap in one method call? #174

Closed oblomov-dev closed 5 months ago

oblomov-dev commented 5 months ago

Hello everyone,

similar question as here: https://github.com/sbcgua/ajson/issues/173 But this time the other way around (JSON -> ITAB) for large tables.

When i get a payload i need to slice it first and then do in a second step the mapping into an abap table:

DATA(ajson_val) = ajson->slice( `/` && lr_attri->name_front ).

ajson_val->to_abap(
      IMPORTING
         ev_container = <backend_tab> ).

So i need again two loops. Looking for something like:

ajson_val->to_abap(
      exporting
          iv_path = `/` && lr_attri->name_front
      IMPORTING
         ev_container = <backend_tab> ).

Is there a function to do this in one step?

Thank you & best regards.

mbtools commented 5 months ago

to_abap is like move-corresponding. just put the fields you want into the target structure:

https://github.com/sbcgua/ajson?tab=readme-ov-file#converting-to-abap-structure

you can either pass iv_corresponding = abap_true per call or use to_abap_corresponding_only( ) once after you create the instance.

oblomov-dev commented 5 months ago

yes the problem i have is that the target is not a structure, its an abap class instance with attributes. means i loop over a list with attribute names, then do an assign and then map the json values to certain attributen of a class instance.

simplified it looks like this:

    LOOP AT t_attri_of_abap_instance REFERENCE INTO DATA(lr_attri).

          ASSIGN (lr_attri->name_abap) TO FIELD-SYMBOL(<backend>).

          DATA(ajson_val) = ajson->slice( `/` && lr_attri->name_frontend ).

          ajson_val->to_abap(
            IMPORTING
              ev_container = <backend> ).

          ajson->delete(  `/` && lr_attri->name_frontend  ).

    ENDLOOP.

Additonally i delete the mapped entries that the next iteration in the loop does not need to check the already used entries. so in total ajson does three loops. besides slice, a method called eg. "cut" which does a slide and deletes the entries in the original ajson would be nice. or perfect fit in this situation would be a to_abap_w_cut method, which reduces everything to one loop.

just asking because when i see the API of ajson, it could fit into the concept. but maybe also my use case is very unusual. its also not a big problem, the three loops work fine in all my tests. just when i send large tables to the backend you can recognize a certain moment when running the ajson snippets.

mbtools commented 5 months ago

same logical fields in backend and frontend but different names? that sounds more like a case for the field mapping (see "rename nodes"):

https://github.com/sbcgua/ajson?tab=readme-ov-file#boxed-in-mappers

oblomov-dev commented 5 months ago

thank you for the information, i will take a closer look to the field mapping.

sbcgua commented 5 months ago

I'd agree with Marc that it is closer to mappers. Maybe the mapper functionality could be improved too, e.g., introducing some prefix to source/destination (which will work as slice) or a more complex - tree mapping - but it's an edge case. I'd like to keep the tool simple and convenient and avoid bloating it with custom cases. Anyway, to do this, we need to discuss a real example in more detail, now it is not very clear what is the problem to be solved.

Also:

sbcgua commented 5 months ago

Is it the case?

image
oblomov-dev commented 5 months ago

yes agree, ajson should stay simple. it has nice APIs now and its not good to bloat it up for edge cases. i will create a real world example which underlines my problem and then you can see if its a good idea or not to extend the API (and how).

concerning the second answer: yes this example is close, and now imagine the payload attribute exists for ten (or just say a limited number) times, each payload has its own (mapper+filter) and the content of each payload attribute could be large (lets say an internal table with 20 cols and 5000 rows), i think then you get some unnecessary loops.

yes but give me a little bit more time and i will come back with a good example :)

Thank you to both of you for checking this, in the first step i wanted to make sure that i did not overlook something, next step are potential improvements.

oblomov-dev commented 5 months ago

Ok abap2UI5 runs smoothly now for weeks with ajson. In case this is interesting for you, I collected all ajson calls in this file: https://github.com/abap2UI5/abap2UI5/blob/main/src/01/02/01/z2ui5_cl_core_json_srv.clas.abap

Still need to refactor some obsolete API usage, but besides that everything is fine!

I created this issue for performance improvements, but for now, there are no complains and I don't want to solve/optimize non-existing problems, so I think we can skip this for now. My original test was done with tables for more than thousands of entries but this is only a very edge case, so I think as long as I don't get performance issues we should leave it like this and ajson can stay as small/simple as possible.

But thank you for the support and information so far! 🙏 It's fun to work with projects which are actively maintained, so I hope its fine when I come back when I have new questions or ideas :)

Best regards!