lloyd / yajl

A fast streaming JSON parsing library in C.
http://lloyd.github.com/yajl
ISC License
2.15k stars 435 forks source link

Unknown old Ptr size when using Realloc function #208

Closed Threas closed 6 years ago

Threas commented 6 years ago

If using custom allocation, realloc might pass in the old ptr in order to copy the old ptr to new one.

drjasonharrison commented 6 years ago

Do you have an example realloc function that takes the old ptr size? Malloc typically returns a pointer to a memory block where the size of the allocated block and other information is stored in lower addresses.

drjasonharrison commented 6 years ago

Here is a discussion of memory allocation. Scroll down about 1/3rd to where multiple heaps are discussed and read the sections on chunk allocation. https://sploitfun.wordpress.com/2015/02/10/understanding-glibc-malloc/

Taymindis commented 6 years ago

I’m sorry for my bad English, I mean I need to know the old ptr size when setup the custom realloc function in yajl, so that I know how much size need to copy to new ptr. I don’t use built realloc as I have my own memory pool

drjasonharrison commented 6 years ago

Ah, you need to store that information in the memory before the address that your allocator returns from malloc. If you don't record the size of the full memory block and the size requested then you can't perform realloc. And free would be difficult too, unless you have an fixed block size per memory address.

Otherwise you can copy the entire range from the old block to the new block and hope that you aren't leaking secrets.

drjasonharrison commented 6 years ago

Here is a similar question and how to implement malloc and realloc. https://stackoverflow.com/questions/10864593/implementation-of-realloc-in-c

Threas commented 6 years ago

you are right, i was thinking to do that too. That's for your suggestion 👍

Taymindis commented 6 years ago

In case anyone who has their own memory pool management, this references might help, just change the malloc function to your memory allocation

#define malloc_with_size_meta(sz)({\
void* __ptr__ = malloc(sizeof(size_t) + sz);\
if(__ptr__){\
*((size_t*)__ptr__) = sz;\
__ptr__ += sizeof(size_t);\
}__ptr__;})

#define free_with_size_meta(ptr)({\
void* __ptr__ = ptr - sizeof(size_t);\
free(__ptr__);})

void *my_yajl_malloc(void *ctx, size_t sz) {
    void* ptr = malloc_with_size_meta(sizeof(size_t) + sz);
    size_t oldptr_sz = *(size_t*)(ptr - sizeof(size_t));
    return ptr;
}

void my_yajl_free(void *ctx, void * ptr) {
    free_with_size_meta(ptr);
}

void *my_yajl_realloc(void *ctx, void *ptr, size_t sz) {
    size_t oldptr_sz;
    void *new_ptr;
    if (ptr) {
        oldptr_sz = *(size_t*)(ptr - sizeof(size_t));
        new_ptr = malloc_with_size_meta (sz);
        memcpy(new_ptr, ptr, oldptr_sz);
        free_with_size_meta(ptr);
    } else {
        new_ptr = malloc_with_size_meta (sz);
    }
    return new_ptr;
}