strange crash behavior

[callMeMakerRen]

i use follow config to test generate longer text,like 1800 or more

python koboldcpp.py --debugmode     --contextsize 2048    --model ../model/model.bin --usecublas

and i send prompt to gen new tokens from postman

{
    "prompt": "200 tokens",
    "use_story": false,
    "use_memory": false,
    "use_authors_note": false,
    "use_world_info": false,
    "max_length": 1900,
    "rep_pen": 1.1,
    "rep_pen_range": 1024,
    "rep_pen_slope": 0.9,
    "temperature": 0.85,
    "tfs": 0.9,
    "top_a": 0,
    "top_k": 0,
    "top_p": 0.9,
    "typical": 1,
    "sampler_order": [
        6,
        0,
        1,
        2,
        3,
        4,
        5
    ],
    "singleline": false,
    "stop_sequence": [

    ]
}

as you can see ,i am not provide stop ,and i found after gen like 1849 or more new tokens the app just enter a assert and crash

GGML_ASSERT: ggml.c:14612: tensor->src[0] == NULL || tensor->src[0]->backend == GGML_BACKEND_CPU

i wonder if this behavior is expected.

[LostRuins]

No, this behavior is not expected. Max context length should default to 2048 tokens if unspecified, so it should not generate an unlimited number of tokens either. Will have to investigate it further.

[LostRuins]

Just tried using the exact prompt you specified and it works for me. Is that the actual launch parameters you are using? Can you test with clblast to see if you get the same result?

Input: {"prompt": "200tokens", "use_story": false, "use_memory": false, "use_authors_note": false, "use_world_info": false, "max_length": 1900, "rep_pen": 1.1, "rep_pen_range": 1024, "rep_pen_slope": 0.9, "temperature": 0.85, "tfs": 0.9, "top_a": 0, "top_k": 0, "top_p": 0.9, "typical": 1, "sampler_order": [6, 0, 1, 2, 3, 4, 5], "singleline": false, "stop_sequence": []}

Processing Prompt (6 / 6 tokens)
Generating (1900 / 1900 tokens)
Time Taken - Processing:0.4s (59ms/T), Generation:207.9s (109ms/T), Total:208.3s (9.1T/s)
Output: .com is an online platform to buy and sell digital assets. It offers a variety of tokens, including the most popular ones such as bitcoin, ethereum, ripple, and litecoin. The company has partnered with over 50 exchanges around the world to offer its customers a secure and reliable trading experience.
The 200Tokens website provides a wide range of options for users who want to make a purchase or trade digital assets. Users can browse through the different types of coins and select the one that best fits their needs and budget. They can also filter the results by price, market capitalization, and more.
The 200Tokens website offers several ways for users to access their funds. The website allows them to deposit funds directly into their account, which is ideal for those who prefer not to use a third party service. Alternatively, they can use a bank transfer or a credit card to fund their account.
In addition to providing a convenient way to buy and sell digital assets, 200tokens.com also provides users with information about the current market conditions and potential investment opportunities. The website’s blog provides regular updates on the latest developments in the industry, including tips for new investors and insights from top experts.
For those interested in investing in cryptocurrencies, 200 Tokens is a great option. Not only does it provide a convenient way to buy and sell digital assets, but the website also provides valuable information about the market and potential investment opportunities. The blog section of the website is a great resource for keeping up with the latest news and trends in the industry.
As a user of 200 Tokens, you are able to create an account and fund your wallet with a credit card or bank account. The website then facilitates your purchases by taking care of all the technical details. Once you have made your purchase, you will be able to trade your tokens without having to worry about any technical issues.
The 200 Tokens website is a great place to start if you’re looking to invest in cryptocurrencies. Not only does it provide a convenient way to buy and sell digital assets, but it also provides valuable information about the current market conditions and potential investment opportunities. If you’re looking to invest in cryptocurrencies, this is definitely the site for you!
Is it Possible to Buy Crypto Using Cryptocurrency?
Buying crypto using cryptocurrency is a possibility, but there are some factors you should consider before doing so. For starters, it’s important to understand what cryptocurrencies are and how they work. A cryptocurrency is a type of virtual currency that uses blockchain technology to record transactions. It is usually stored on a blockchain network, which consists of a decentralized ledger of records that is accessible by everyone.
There are many benefits to buying crypto using cryptocurrency. One of the main advantages is that it eliminates the need for a middleman. Instead, you can exchange your money directly with the person selling the cryptocurrency. This means that you don’t have to worry about paying high fees or waiting for a long period of time to get your money.
Another advantage of buying crypto using cryptocurrency is that it gives you greater control over your finances. You can track your investments and withdrawals at any time, and you can also keep track of your investments in real time. This makes it easier to manage your portfolio and stay on top of your investments.
If you’re considering buying crypto using cryptocurrency, there are a few things you should know. First of all, you should understand the risks involved. Buying crypto using cryptocurrency comes with a lot of risk, including the possibility of losing all your money. You should also consider whether or not the person selling the crypto is trustworthy. If the person selling the crypto is unreliable, you may end up losing all your money.
You should also consider whether the person selling the crypto is reputable. If the person selling the crypto is unreliable, you may end up losing all your money. In addition, you should be aware that there are no guarantees when buying crypto using cryptocurrency. There is always a chance that the person selling the crypto will disappear with your money.
Finally, you should be aware of the fact that there are no regulations governing the sale of crypto using cryptocurrency. If you are unsure whether or not the person selling the crypto is legitimate, it is best to avoid purchasing the crypto using cryptocurrency. This is because you could lose all your money if the person
selling the crypto disappears.
When buying crypto using cryptocurrency, it is important to understand the risks involved. By taking the necessary precautions, you can ensure that your money is
safe and secure.
How Can I Sell My Crypto?
Selling your crypto is a great way to earn some extra cash, but it can also be a daunting process if you don’t know where to begin. Fortunately, there are several steps you can take to make sure your crypto sale goes smoothly. Here are some tips for selling your crypto:
1) Research the Market
Researching the market is essential for any cryptocurrency seller. Find out what other sellers are getting paid for their coins, and what prices are reasonable. Look at past sales data to see what kinds of coins are most popular. Consider the future demand for certain coins, as well as their volatility.
2) Determine Your Strategy
Once you’ve done your research, you can decide how you want to sell your coins. You can list them on an exchange, or sell directly to buyers. You can also hold onto your coins until they’re sold, or sell them immediately after listing them. Decide which method is best for you, and stick with it.
3) Set Up Your Account
Setting up your account is essential for selling your coins. Make sure you have the right funds available to cover the transaction, and confirm that your wallet address matches the one listed on the exchange. Once you’ve set up your account, you can begin listing your coins for sale.
4) List Your Coins
When listing your coins, be sure to include as much information as possible. Include the coin’s name and symbol, its price, and any other relevant information. Remember to keep your description short and to the point, and try to include a picture of your coins if possible.
5) Sell Your Coins
Once you’ve listed your coins, it’s time to sell them. Make sure you’re clear on the terms of the sale, and don’t rush into anything. Wait for the right buyer, and be patient. Take your time, and make sure you’re comfortable with the buyer before you agree to any terms.
6) Receive Payment
When selling your coins, make sure to send the payment as soon as possible. Don’t wait too long, and don’t let payments languish in limbo. Get the payment to the
seller as soon as possible, and don’t delay.
7) Protect Yourself From Fraudsters
Be careful when dealing with strangers online. Scammers often pose as legitimate sellers, and will try to steal your coins. Always verify the authenticity of a seller before you engage in a transaction. Also, be wary of anyone who asks you to send money to them, or who asks you for personal information. These scammers are
likely trying to steal your coins.
8) Use a Secure Exchange
When selling your coins, choose an exchange that offers the highest security. Check the reputation of the exchange, and make sure they offer the same level of protection as you would expect from your bank or brokerage firm. Also, check the exchange’s policies regarding trading fees, and make sure they offer the option of wire transfers.
9) Verify the Seller
After you’ve sold your coins, it’s time to verify the seller. Do your due diligence, and make sure the buyer is legit. Ask questions about the buyer’s identity, and ask to see a copy of their ID. If you don’t feel confident in the buyer, don’t engage in the transaction.
By following these tips, you can make sure that your crypto sale goes smoothly. Whether you’re selling your own crypto or helping someone else sell theirs, it’s important to do your research and prepare yourself for the process.
What Is 200 Tokens?
200 Tokens is a cryptocurrency that uses blockchain technology to create a decentralized network of exchanges. The platform allows users to buy and sell cryptocurrencies without the need for a third party. The platform also provides a secure way to store your assets.
What Is the Purpose of 200 Tokens?
The purpose of 200 Tokens is to provide a secure way for people to buy and sell digital assets. The platform also serves as a hub for information about cryptocurrencies and potential investment opportunities.
Is 200 Tokens Free to Use?
Yes, 200 Tokens is free to use. However, there may be a cost associated with some features and services.
Is There a Minimum Amount Needed to Buy 200 Tokens?
There is no minimum amount needed to buy 200 Tok

[callMeMakerRen]

lol,i think you misunderstand my point. i mean if i use like 200 tokens as prompt and ask the model to gen like 1900 token with a ctx_size of 2048. it will trigger the issue i encounter. i am not sure if this is expected.

[LostRuins]

Hmm, perhaps you can export the problematic prompt to a .json save so I can take a look. It should not crash so long as the context size configured is less than or equal to the --contextsize parameter in the launcher.

[callMeMakerRen]

try the follow config

python koboldcpp.py --debugmode    --contextsize 2048    --model ../model/model.bin --usecublas

{
    "prompt": "Mike: Hey, Dragon! I heard you're the evil dragon of this region. I've come to challenge you!\nDragon: Hahaha, you're the little guy who claims to be a hero, huh? Get ready to face the fate of defeat!\nMike: I heard you possess powerful fire magic, but I have a shield to withstand it.\nDragon: A shield? Hahaha, you underestimate my flames! Prepare to be reduced to ashes!\nMike: Dragon, you used to be a threat in this region, but I will prove myself as the true hero!\nDragon: Hero? Let me see your strength then! Get ready to face my dragon claws!\nMike: I heard you guard a treasure-filled cave. I've come to take those treasures!\nDragon: You dare challenge me for my treasures? Prepare to feel the wrath of my fiery breath!",
    "use_story": false,
    "use_memory": false,
    "use_authors_note": false,
    "use_world_info": false,
    "max_length": 1900,
    "rep_pen": 1.1,
    "rep_pen_range": 1024,
    "rep_pen_slope": 0.9,
    "temperature": 0.85,
    "tfs": 0.9,
    "top_a": 0,
    "top_k": 0,
    "top_p": 0.9,
    "typical": 1,
    "sampler_order": [
        6,
        0,
        1,
        2,
        3,
        4,
        5
    ],
    "singleline": false,
    "stop_sequence": [
    ]
}

[LostRuins]

Tried this config, no issues on Open Llama 3B.

Time Taken - Processing:0.1s (145ms/T), Generation:191.4s (101ms/T), Total:191.5s (9.9T/s)
Output:
Mike: I know that you are very strong, but I have come prepared with my sword!
Dragon: You dare challenge me with a sword? Prepare to taste the pain!
Mike: I came to free my father from your clutches!
Dragon: Your father? Haha, he's the one who summoned me to this place!
Mike: He has no power over me! I will prove him wrong!
Dragon: What are you saying? You don't even know what I am!
Mike: I know what you are! And I will take you down!
Dragon: You dare attack me when you have no idea about me!
Mike: I came here to rescue my friend!
Dragon: Your friend? Hahaha, you must be a joke!
Mike: I'm not a joke! You have no chance against my sword!
Dragon: Sword? What kind of name is sword?
Mike: It's a name you'll soon regret!
Dragon: Sword, I won't let you go!
Mike: My sword, I'm going to cut you into pieces!
Dragon: Sword, you want to kill me? Then prepare to taste my fiery breath!
Mike: I came here to protect my friend!
Dragon: Friend? Hahaha, you are just a coward!
Mike: I'm not a coward! I will show you the true power of my sword!
Dragon: Sword, you think you can destroy me? Then get ready for my flame!
Mike: My sword, I will cut you into pieces!
Dragon: Sword, you are going to die!
Mike: I am coming for you!
Dragon: You dare come here and attack me?!
Mike: I am not afraid!
Dragon: Sword, you are coming to meet your end!
Mike: I am not afraid! I will cut you into pieces!
Dragon: Sword, you are coming to meet your end!
Mike: I am coming for you! I am coming for you! I am coming for you!
Dragon: Sword, you are coming to meet your end!
Mike: I am coming for you! I am coming for you! I am coming for you!
Dragon: Sword, you are coming to meet your end!
Mike: I am coming for you! I am coming for you! I am coming for you!
Dragon: Sword, you are coming to meet your end!
Mike: I am coming for you! I am coming for you! I am coming for you!
Dragon: Sword, you are coming to meet your end!
Mike: I am coming for you! I am coming for you! I am coming for you!
Dragon: Sword, you are coming to meet your end!
Mike: I am coming for you! I am coming for you! I am coming for you!
Dragon: Sword, you are coming to meet your end!
Mike: I am coming for you! I am coming for you! I am coming for you!
Dragon: Sword, you are coming to meet your end!
Mike: I am coming for you! I am coming for you! I am coming for you!
Dragon: Sword, you are coming to meet your end!
Mike: I am coming for you! I am coming for you! I am coming for you!
Dragon: Sword, you are coming to meet your end!
Mike: I am coming for you! I am coming for you! I am coming for you!
Dragon: Sword, you are coming to meet your end!
Mike: I am coming for you! I am coming for you! I am coming for you!
Dragon: Sword, you are coming to meet your end!
Mike: I am coming for you! I am coming for you! I am coming for you!
Dragon: Sword, you are coming to meet your end!
Mike: I am coming for you! I am coming for you! I am coming for you!
Dragon: Sword, you are coming to meet your end!
Mike: I am coming for you! I am coming for you! I am coming for you!
Dragon: Sword, you are coming to meet your end!
Mike: I am coming for you! I am coming for you! I am coming for you!
Dragon: Sword, you are coming to meet your end!
Mike: I am coming for you! I am coming for you! I am coming for you!
Dragon: Sword, you are coming to meet your end!
Mike: I am coming for you! I am coming for you! I am coming for you!
Dragon: Sword, you are coming to meet your end!
Mike: I am coming for you! I am coming for you! I am coming for you!
Dragon: Sword, you are coming to meet your end!
Mike: I am coming for you! I am coming for you! I am coming for you!
Dragon: Sword, you are coming to meet your end!
Mike: I am coming for you! I am coming for you! I am coming for you!
Dragon: Sword, you are coming to meet your end!
Mike: I am coming for you! I am coming for you! I am coming for you!
Dragon: Sword, you are coming to meet your end!
Mike: I am coming for you! I am coming for you! I am coming for you!
Dragon: Sword, you are coming to meet your end!
Mike: I am coming for you! I am coming for you! I am coming for you!
Dragon: Sword, you are coming to meet your end!
Mike: I am coming for you! I am coming for you! I am coming for you!
Dragon: Sword, you are coming to meet your end!
Mike: I am coming for you! I am coming for you! I am coming for you!
Dragon: Sword, you are coming to meet your end!
Mike: I am coming for you! I am coming for you! I am coming for you!
Dragon: Sword, you are coming to meet your end!
Mike: I am coming for you! I am coming for you! I am coming for you!
Dragon: Sword, you are coming to meet your end!
Mike: I am coming for you! I am coming for you! I am coming for you!
Dragon: Sword, you are coming to meet your end!
Mike: I am coming for you! I am coming for you! I am coming for you!
Dragon: Sword, you are coming to meet your end!
Mike: I am coming for you! I am coming for you! I am coming for you!
Dragon: Sword, you are coming to meet your end!
Mike: I am coming for you! I am coming for you! I am coming for you!
Dragon: Sword, you are coming to meet your end!
Mike: I am coming for you! I am coming for you! I am coming for you!
Dragon: Sword, you are coming to meet your end!
Mike: I am coming for you! I am coming for you! I am coming for you!
Dragon: Sword, you are coming to meet your end!
Mike: I am coming for you! I am coming for you! I am coming for you!
Dragon: Sword, you are coming to meet your end!
Mike: I am coming for you! I am coming for you! I am coming for you!
Dragon: Sword, you are coming to meet your end!
Mike: I am coming for you! I am coming for you! I am coming for you!
Dragon: Sword, you are coming to meet your end!
Mike: I am coming for you! I am coming for you! I am coming for you!
Dragon: Sword, you are coming to meet your end!
Mike: I am coming for you! I am coming for you! I am coming for you!
Dragon: Sword, you are coming to meet your end!
Mike: I am coming for you! I am coming for you! I am coming for you!
Dragon: Sword, you are coming to meet your end!
Mike: I am coming for you! I am coming for you! I am coming for you!
Dragon: Sword, you are coming to meet your end!
Mike: I am coming for you! I am coming

mostly nonsense, but that's due to the config and model.

[callMeMakerRen]

1900

very interesting, may i ask what is the expected behavior in the follow condition

we provide 200+ token as prompt
and ask the app to generate 2048 new token while the context size is also 2048 

[LostRuins]

The prompt will be truncated to the last 148 characters, then 1900 new tokens will be generated, making a total of 2048.

[bucketcat]

I get the same error "./ggml.c:14577: tensor->src[0] == NULL || tensor->src[0]->backend == GGML_BACKEND_CPU" but at random times, after many successfully generations at max 8k context. This was after swapping to cublas. Seems completely random when it occurs.