aarashy
commented
1 year ago It occurs to me that opening the 4GB model binary file from within the browser generally isn't practical. I had the following conversation with GPT-4, and I agree with its conclusion.
Prompt: Is it practical for TypeScript code to open a 4GB file while it executes in the browser? I suppose this file could be stored with other UI assets such as html and JavaScript files.
Response: _Opening a 4GB file in a browser using TypeScript (or JavaScript) is not practical for several reasons:
Memory constraints: Browsers are designed to handle web applications with much smaller memory footprints. Loading a 4GB file into memory may cause the browser to crash, run out of memory, or become unresponsive. Performance: Reading and processing a large file will require significant processing power and may cause the browser to slow down or freeze, leading to a poor user experience. Network constraints: Even if the browser could handle a 4GB file, downloading such a large file would take a significant amount of time, especially for users with slow or unreliable internet connections. This would also increase the load on your server. Instead, consider the following alternatives:
Use server-side processing: Instead of loading the entire file in the browser, process the file on the server and send only the necessary data to the client. Stream the data: If you need to work with the data in real-time, consider streaming the data in smaller chunks from the server to the client. Pagination and progressive loading: If you have to display large datasets, break them into smaller parts and load only the required data when the user requests it. Use efficient data formats: Optimize the file format to reduce the size, such as by using compression or binary formats, and consider using specialized libraries to handle large datasets. Overall, it is essential to design your application to work with large files efficiently by minimizing the amount of data sent to the browser and avoiding loading the entire file into memory._
Therefore, I'm going to close this issue. It seems like TypeScript running outside of the browser is the only use case for this.
I saw this issue and was thinking of taking a crack at it. https://github.com/hwchase17/langchain/issues/2401
Similarly to the Python GPT4All, the user will naturally need to provide a path to the
.binfile of the pretrained model.The only part I was uncertain about was how to fulfill the job of
pyllamacpp.modelfrom TypeScript. I will do some research on this.