Memory buffer instead of a file

[Teles1]

Hello,

I looked around and couldn't find anyone running into the same "issue" that I did.

I currently have a Resource manager that handles multiple files formats, zips, packed, tars, compressed, raw formats etc and I would love to use my Resource manager to grab the sqlite file, read it to a buffer and pass that content along (read only). Is there an easy way to support/do such without modifying the internal functions?

Thanks a lot!

[fnc12]

Hello. Looks like it is more task for your file system not SQLite. But probably I don't get it quite right. Anyway how would you implement this if let's say you have only C API with libsqlite3 without sqlite_orm?

[Teles1]

sorry, let me try to exemplify it. Eg: Assuming the following folder structure Folder -

• binary_run
• packed.pkt

My resource manager loads and all files stored inside of packet.pkt are indexed into a hash map that has the file name and it's offset + size. My goal here is to pass on a memory buffer to sqlite_orm as a "char* buffer, uint64_t size" and flag the database as read only obviously since we can't alter the database at all in this state. I guess what I really need is a ":memory:" but with a way to pass on a buffer as well. Also, thank you very much for such quick response time! damn! 👍

[fnc12]

ok I see. It more less looks like unarchiving .apk and obtaining assets from it. There are two ideas I have in my mind: 1) SQLite has virtual file systems feature. It is one more abstraction layer between SQLite and database representation. Probably it may help. Try to answer my question 'how would you implement this if let's say you have only C API with libsqlite3 without sqlite_orm'. 2) use :memory: database but load and save it into file using backup feature. This is feature is also supported in sqlite_orm. TL;DR it allows you moving all DB contents from one connection to another. So you may have one connection in memory, second in file and you can move the contents between them. It is not exactly what you need but it is a good start.

[Teles1]

2. I guess that sort of works. I need to add an extra step to make this work properly which would involve a tmp file to serve as a backup_from into a :memory: database. Good enough compromise vs amount of work.

[fnc12]

@Teles1 BTW we have a discord server. You can join and we can talk there https://discord.gg/gq8sdAZt

[fnc12]

@Teles1 should we close this issue or you have more questions?

[trueqbit]

@Teles1 You may want to have a look at the "pointer-passing interfaces" and whether they solve your requirements. This allows you to create application-defined SQLite functions that generate objects of any kind that you can pass on to other functions. IOW you can create an ad-hoc extension within your application.

sqlite_orm supports this through the ""_pointer_type, pointer_arg_t<>, pointer_binding_t<> and bind_pointer() API. An example is available.

[Teles1]

@Teles1 BTW we have a discord server. You can join and we can talk there https://discord.gg/gq8sdAZt

send me the link again please! I opened my e-mail on my phone and then completely forgot to check back on it...

@Teles1 You may want to have a look at the "pointer-passing interfaces" and whether they solve your requirements. This allows you to create application-defined SQLite functions that generate objects of any kind that you can pass on to other functions. IOW you can create an ad-hoc extension within your application.

sqlite_orm supports this through the ""_pointer_type, pointer_arg_t<>, pointer_binding_t<> and bind_pointer() API. An example is available.

Yep! That does it for sure. I've been messing with it for the past couple of days.. thank you!