Load image from disk to pre-allocated buffer

[giordi91]

Hi, I am using SOIL(great lib by the way, big thumb up) to load images into memory, but from that point on I handle all my stuff manually. I am also doing memory handling myself with stack allocators etc. At this point I would love to be able to tell SOIL to read memory in my given pre-allocated cpu buffer. This is due to avoid an extra copy from "SOIL memory" to my preallocated memory. Of course I don't think this is particularly simple task, mainly because I might not know upfront the size of the image so my questions are the following: I don't know much about image storage format, but do images have their size in the header? that can be queried fairly cheaply, to let me allocate the buffer and then have SOIL write in my memory?

Would you be up for a discussion about that? If so I would like to help on the matter. Just bear in mind I am here mainly for a discussion of the topic, is not a feature request, loading texture happens async and doesn't happen on a frame bases so is not really a critical path of the code, and I can totally live with the extra copy, that s why I would like to see if you think is something that might be useful, and if so how much work you think it can be.

Cheers

M.

[njcrawford]

If you look at SOIL_load_image() and SOIL_load_image_from_memory() in SOIL.c, it's using stbi_load() and stbi_load_from_memory() from the stb project to take care of loading the file. Both of those do their own allocation and return a pointer.

The latest version of stb_image.h (here) doesn't show any stbi_load() variants that can take a pre-allocated buffer, so I don't think this is something that would be easy to implemented in SOIL, IMHO. (I should note that I'm not the maintainer of SOIL, and haven't used it in more than a year... I'm just passing on what I know to hopefully save you some time)

I don't mean to discourage you from trying to implement it, of course! (it would likely be a learning experience!) But for a situation like this, I think you might be better served by a different library. Alternately, you might want to consider writing your own image loader, since it would probably be about the same amount of work as implementing this feature in SOIL.

[giordi91]

@njcrawford thank you for getting back to me , yes I was having a look at the code and stumbled on stb_image.h About writing my image loader I would rather not, I did a simple bmp loaders in the past , but I would like not to spend time on that and focus more on the design and dev of my engine. Now, If stb would provide me a simple way to query the metadata of the file like size, then should be trivial to modify the stb_load_image functions , or overload one to actually get in the parameter. If I can get that from the stb guys. Would be something you or the owner of the repository would get into consideration of adding as a feature?

M.

[njcrawford]

As I see it, you'd need 2 additional functions from stb for this to work.

The first would be something like unsigned int stbi_get_image_size(const char * filename) which would return the number of bytes required to store the loaded file.

The second would be something like bool stbi_load_image_prealloc(stbi_uc * pre_alloc_buf, char const *filename, int *x, int *y, int *comp, int req_comp) which would write the image to pre_alloc_buf instead of allocating it's own buffer, and return true/false to indicate success/failure.

If you can get those 2 functions in stb, then I think it would be fairly easy to add support in SOIL. One other thing I'm curious about, do you call any SOIL functions other than SOIL_load_image() and SOIL_load_image_from_memory()?

[giordi91]

I will talk to the stb guys and see what they tell me. Yeah I only use SOIL_load_image() and that s it, I copy the data where i need in my data-structure and free the soil memory. Soil is wrapped into a single call in my engine. I will let you know what the stb guys say.

M.

[njcrawford]

Ok, sounds good. I'm interested to see what they have to say about it.

I'm not trying to discourage you from using SOIL, but since you're only using SOIL_load_image(), you could replace the calls to SOIL_load_image()/SOIL_free_image_data() with calls to stbi_load()/stbi_image_free() and get the same functionality without a library in between.

See the definition of SOIL_load_image() - it's basically a passtrough to stbi_load().

[giordi91]

You are making a really valid point there! Thank you for your help. You can close the issue if you want, i will chase the matter with the stb guys. Thanks a lot M.

[njcrawford]

No problem! If you do open an issue with stb or hear something back from them, please let me know. I might be interested in using pre-allocated buffers as well.