Make Library a class

[jkomoros]

Currently it's a particularly-shaped data structure that is passed to a number of different functions.

That means the structure of the data has to be known by a number of different scripts, making it hard to change.

It also means that if we want to do something clever with the format, like allow chunks to reference a shared info_id, we need to teach everything that deals with chunks how to do that.

[dglazkov]

One thing to consider: since the data structure is roughly set, perhaps the class needs to be controller-style -- it takes library as argument and knows how to operate on it.

[jkomoros]

• [x] Support Library(filename=foo)
• [x] Move Library to library.py
• [x] server/client.py updated to use Library
• [x] set_chunk and set_chunk_field should respect Library.omit configuration
• [ ] set_chunk_fields if it includes text (and not embedding or token_count) then it should recalculate embedding or token_count
• [x] factor out load_multiple_libraries
• [ ] Decide if library.chunk() should return a copy or not (and then update the defensive use of set_chunk)
• [x] move library_for_query to be on Library
• [x] remove load_library
• [ ] Add a completion_model to the library format, because technically the token_count of chunks is specific to the completion_model being used
• [ ] Add a clone() method that returns a full copy
• [ ] Add a set_all_fields(self, field_name, property_dict), which calls self.set_chunk_field(chunk_id, value) for each field in property_dict. That's useful for burning in similarities in library.query()
• [ ] Add a truncate(count=, count_type) that truncates a library down to a given context length. Basically get_context.
• [ ] get rid of get_context_for_library and get_chunk_infos_for_library
• [x] Move all use cases of data() to using defined getters/setters.
• [ ] Consider having Library.chunk() return a Chunk object.
• [ ] Remove all methods in ask_embeddings that are not tied to Library