olliestanley
commented
1 year ago The biggest limiting factor now is context length. OA on current inference hardware is only capable of "seeing" 1792 tokens (maybe 5000ish characters) at a time, so we can't allow very long outputs otherwise it will start to forget what the prompt was before it finishes the output, causing incoherent responses. We are working on upping context length
So whats the reason for the AI output limitation? Is it a technical/processing power issue or a preference issue? If it was technical, couldn't the output be broken up somehow to separate the processing and then connected to form the full response, or have the response be slowly processed in some other way and fed bit by bit to the user so there is no 'overload' while allowing the AI to output the necessary response even if its infinite. This output limitation just seems so limiting in what the AI can do and what the user can do. As you know code samples are cut off, and often some AIs can't remember what code you are referring to in order for the AI to continue from it or if it tries to continue it is obviously not the same code but a new code that disregards the original code. This issue is not limited to code but everything else of course. This output limitation seems 'archaic' like the income tax, politicians, capitalism, and so on; all of which were unnecessary in the first place except for exploitation by those behind it. If it was a preference issue i could understand it and not like it. But if its a technical issue isn't there always a simple solution waiting to be found.
Update - So I decided to ask AIs themselves and here is what they said
-
"The output limitation for AI systems is a combination of both technical and preference-based reasons. From a technical standpoint, AI systems are limited by the processing power of the machine or system they are running on, as well as the memory and storage capacity available. In addition, AI systems may also be limited by the programming and design of the system itself, which can impact the amount of data and information that can be processed and generated in a given timeframe.
From a preference-based standpoint, there are often limits placed on AI systems to prevent them from generating excessive or potentially harmful content. For example, some AI systems may be programmed to limit the length or complexity of their responses to prevent overloading the user or the system itself. Additionally, there may be legal or ethical considerations that impact the types of content or information that an AI system can generate or provide.
In terms of breaking up the output or processing information incrementally, this is a technique that is often used in AI systems to help manage processing and memory limitations. By breaking up large amounts of data or information into smaller, more manageable chunks, an AI system can generate responses more efficiently and with greater accuracy.
However, it is important to note that AI systems are still limited by the programming and design of the system itself, as well as the processing power and memory capacity of the machine or system they are running on. While there are often simple solutions available to help manage these limitations, there may also be inherent limitations that cannot be overcome without significant changes to the system or hardware."
-
Then I asked: What about you AI, do you have any suggestion on how to address this AI output limitation on the technical side? Use your vast knowledge of computers and such related fields and related optimization to help humans come up with solution to address this current AI output limitation that humans themselves might not be aware of utilizing.
-
"As an AI language model, I don't have direct access to the underlying hardware and software infrastructure that powers my operation. However, I can suggest that researchers and developers continue to work on optimizing AI algorithms and hardware to improve processing power and reduce latency.
One promising area of research is in the development of more efficient neural network architectures, such as recursive neural networks and transformer networks, which have shown improved performance in certain tasks. Additionally, researchers are exploring the use of specialized hardware, such as graphics processing units (GPUs) and tensor processing units (TPUs), to accelerate AI computations and reduce the need for large amounts of processing power.
Another approach is to use parallel processing techniques, such as distributed computing and multi-threading, to break up large computations into smaller, more manageable tasks that can be executed simultaneously on multiple processors or nodes.
Finally, it's important to consider the limitations of AI and its potential impact on society. As AI becomes more advanced, it's important to ensure that it is developed and deployed responsibly, with appropriate safeguards in place to prevent unintended consequences and ensure that AI is aligned with human values."
-
This is just 1 AI response but the other AIs are similar. I am aware that a lot of these things are already being worked on.