[Bug]: 2023-10-20 12:31:18 autolabel.models.refuel WARNING: Retrying autolabel.models.refuel.RefuelLLM._label_with_retry in 2.0 seconds as it raised HTTPError: 500 Server Error: Internal Server Error for url: https://llm.refuel.ai/models/refuel-llm/generate.

[myrainbowandsky]

I've got key from refuel, I tried all of them: refuel-llm, llama-13b-chat, flan-t5-xxl. None is working. I ran:

ds = agent.run(ds,max_items=10) I got: 2023-10-20 12:31:18 autolabel.models.refuel WARNING: Retrying autolabel.models.refuel.RefuelLLM._label_with_retry in 2.0 seconds as it raised HTTPError: 500 Server Error: Internal Server Error for url: https://llm.refuel.ai/models/refuel-llm/generate.

[rajasbansal]

Hey, looking into this. Would it be possible for you to share the input which leads to an error? Want to confirm that they are below the token limit for our model (currently 4096 tokens). Looks like these errors are being sent to Autolabel, but they are not being shown to the user.

[myrainbowandsky]

I've checked the doc of refuel. The headcase problem is that I cannot find details of how to write a precise params() in a config file for different providers. In addition to several examples, I suggest a detailed instruction of writing precise params() is included in the doc. E.g. in this case, I should set the max tokens to 4096, then how to write in the config for classification. I also got 13:34:33 accelerate.utils.modeling INFO: We will use 90% of the memory on device 0 for storing the model, and 10% for the buffer to avoid OOM. You can setmax_memoryin to a higher value to use more memory (at your own risk). ,then how to set max_memory in autolable?

The text is sth like Continuing from the article:Here is a good summary of these [propaganda] efforts courtesy of Nighvision:- For instance when the Russian POW torture video came out, it was Ukraine that cried “fake” until days later all western ‘authorities’ were forced to admit it was real.- When the ‘maternity hospital’ falseflag in Mariupol occurred, Kiev supporters blamed Russia and once again Russia was the one proven right and vindicated when the pregnant girl at the center of it herself released an interview completely refuting Ukrainnian lies, and stating that not only did Ukrainian soldiers turn the hospital into a barracks but stole the precious food from pregnant women and then shelled the hospital themselves.- When Ukrainie claimed Russia bombed an ‘innocent’ civilian mall, yet the very next moment the Russian MOD released detailed videos showing precisely how Ukrop forces positioned mobile artillery in the parking garage of the said ‘mall’.- When Ukrops claimed Russia tried to blow up the Zaporizhzhia plant, yet security footage showed that as a Russian security force arrived, it was fired upon by an RPG from Ukrop positions in one of the administrative buildings of the plant.- When the famed American journalist was killed in Irpin right outside Kiev a month ago, and all Ukrainian supporters shouted Russia, but the entire incident was quickly swept under the rug when the journalist’s own friend/companion in an interview stated they were fired upon by Ukrop troops at a Ukrop checkpoint miles away from the nearest Russian forces..This brings us to the latest “atrocity” Ukraine insists was carried out by Russia–Bucha, a suburb of Kiev. Here is The Guardian’s account:>As Ukrainian armoured columns rolled into Bucha, a town north-west of the capital, they found streets blocked by burned-out Russian tanks and military vehicles, and strewn with the bodies of civilians whom locals said had been killed by the invading forces without provocation..Here’s the problem. Bucha was declared ‘completely liberated’ of Russian troops by its mayor on Thursday, March 31. The dead lying in the streets are all wearing a white arm band, which is worn by Ukrainians to signal to Russian troops they are not backing the Ukrainian Army. .If the city was liberated on 31 March why are the bodies still lying in place three days later? Why were they not gathered up by grieving relatives? This report is having the predicable effect in the West–pundits and politicians are demanding a NATO military response. Russia is not fazed by this latest propaganda barb; it is calling for an emergency meeting of the UN Security Council to present evidence refuting the charge.

[rajasbansal]

• Autolabel does not support setting the max_memory taken on the GPU at the moment. That is handled by accelerate through huggingface. We load the model here - https://github.com/refuel-ai/autolabel/blob/941b6e7a992d6de9bbe5f41757875b85437526cb/src/autolabel/models/hf_pipeline.py#L84. However, once a model is loaded I don't think this should be an issue.
• On the input that you have shared, looks like the input token length is ~600. That should be under the RefuelLLM token limit. Wanted to check if you were facing the error with this input as well. I am currently working on a fix that shows token limit exceeded errors more transparently in autolabel. If some huggingface model was being loaded locally, the maximum tokens that can be handled depend on the gpu resources

[myrainbowandsky]

Any recommended tools to count #tokens of texts?