search

NeilMacMullen / kusto-loco

C# KQL query engine with flexible I/O layers and visualization
MIT License
37 stars 4 forks source link

Support for removing tables #7

Closed Bazefield closed 6 months ago

Bazefield commented 7 months ago

I am testing out this library/project and I need to generate dynamic tables and adding them to to the query context. But I also need to remove them again to prevent buildup of memory over time.

A simple RemoveTable function in the KustoQueryContext that removes the table using the table name would be great.

lothar7 commented 7 months ago

The global table list should probably be a concurrent dictionary instead of a list so we don't get locking/concurrency issues when adding/removing tables.

lothar7 commented 7 months ago

I just realized that one can just create a new BabyKustoEngine or KustoQueryContext so perhaps this is not needed. The only use-case would be when some tables should stay in memory and other tables unloaded. Having the RemoveTable function would save me having to keep a shadow copy of the current table list already in memory.

NeilMacMullen commented 7 months ago

Apologies - have been a bit quiet on this project in the last few weeks due to other commitments. There's another case which is becoming increasing common in our own code which is where we want to modify a context with an updated version of table (e.g. if file changes on disk)

NeilMacMullen commented 7 months ago

I've added support for various table-set manipulations in KustoQueryContext.

I've added some notes here. I don't believe there are any concurrency issues involved in table manipulation as long as it's understood that KustoQueryContext is not intended to be thread-safe. (Clearly it would be possible to make it so but I can't see what it would benefit?)