morganstanley / hobbes

A language and an embedded JIT compiler
http://hobbes.readthedocs.io/
Apache License 2.0
1.17k stars 105 forks source link

farray's cdelim behaves poorly for large inputs #321

Closed adam-antonik closed 4 years ago

adam-antonik commented 4 years ago

show([0..1000000]) will consume a massive amount of memory and time. Moving this down to c++, copying the logic of the existing cdelim makes this work. On a related note, why is the c++ code for cdelim repeated twice, once in include/hobbes/reflect.H, and again in src/lib/hobbes/util/str.C?

kthielen commented 4 years ago

Doh, looks like an entry for Accidentally Quadratic. >_<

I have found this in a few places, where people write array code as if it was list code. It might be better to remove the ++ definition for arrays entirely. :T

Re: reflect.H, that's part of a small set of libraries that work as header only libraries outside of hobbes. For example, storage.H can be used in plain C/C++ programs to log structured data into shared memory without pulling in the hobbes PL/compiler (or having to link anything). There are several of these libraries to interact with hobbes with a minimum of overhead (net.H, fregion.H, cfregion.H, convert.H, slmap.H) and they all rely on reflect.H to describe type structures and define "standard layout" reflective types (e.g. reflect.H defines a tuple type whose memory layout matches the equivalent struct type, unlike std::tuple).

kthielen commented 4 years ago

Fixed here: https://github.com/Morgan-Stanley/hobbes/pull/323

kthielen commented 4 years ago

Fix merged to master.