overall optimization

[onur-ozkan]

Once #1247 is done, we should consider doing overall optimization around the project. Aim is simply decreasing the huge compilation time, and increasing runtime performance as much as possible.

Things should be considered:

• [ ] remove un-needed functions
• [ ] don't allocate memory for things that computer could recompute fast enough
• [ ] update expensive algorithms to make it cheaper if possible
• [x] replace global allocator (consider jemalloc for UNIX systems with optimization tweaks)
• [ ] Optimize data structures (prefer smallest possible data type for most cases) if possible
• [ ] Consider caching on heavy operations if possible
• [ ] Precompute/Preexecute things asynchronously if they don't need to be blocked by previous operations.
• [ ] unify duplicated code (specially data-types re-defined for tests)

[tonymorony]

right now we are considering the switching to infrastructure providing for the some of the blockchain nodes

I did the following test:

1) In CLI enabled BNB and one BNB-BEP20 token 2) Left it run for some time

Right now it consume ~10 calls/minute in idle mode. Probably there is a room for optimisation (what might be a cost cut in case of such subscription service usage or infra loading cut in case of own nodes usage)

[artemii235]

@ozkanonur That's a good idea in general, but I would like to highlight some topics from the list, that have higher priority than others:

1. huge compilation time slows down the development, CI builds, etc. so it burns some of our development time. Worth fixing ASAP. I think you can do it along with https://github.com/KomodoPlatform/atomicDEX-API/issues/1247 so I assigned this issue to you too :slightly_smiling_face:
2. Excessive 3rd party APIs calls @tonymorony mentioned also has to be fixed ASAP as it will incur additional costs to us and possibly to our users. Assigned to @caglaryucekaya.
3. As per other topics: I think we have to focus first on code that can potentially become (or already became, like previous item) a bottleneck or can use less network traffic or RPS (requests or messages per second) than it uses now. During research on this issue, please pay more attention to orderbook sync protocol and atomic swap messages exchange. Messages size, how frequently they are sent, how much memory this data consumes and if it's cleaned up properly if not required etc.

[artemii235]

right now we are considering the switching to infrastructure providing for the some of the blockchain nodes ... Right now it consume ~10 calls/minute in idle mode

@tonymorony this should be fixed in https://github.com/KomodoPlatform/atomicDEX-API/issues/1269, could you retest using dev branch, please?

[tonymorony]

performed same (as https://github.com/KomodoPlatform/atomicDEX-API/issues/1269#issuecomment-1114678169) test case today: activated BNB and 1 BEP20 token and left mm2 (without any orders) in idle mode for a day: calls to endpoint were executed only on activation event. So situation above is definitely looks fixed