模型的FIM效果相比其他模型怎么样

[Ski-ing]

您好, 模型的FIM效果相比其他模型怎么样? 能否给出模型FIM的效果在公开评测集上的评测效果, 如FIM human-eval等.

[cyente]

FIM方面的评测，我们还在整理中，目前可以先贴一下，Codeqwen1.5 humaneval-fim的评测结果	Language	Score
java	87.19%
py	75.65%
js	85.07%

[kartikzheng]

FIM方面的评测，我们还在整理中，目前可以先贴一下，Codeqwen1.5 humaneval-fim的评测结果

Language Score java 87.19% py 75.65% js 85.07%

你好，请问有Codeqwen1.5-7b在humaneval-x的评测效果吗？我使用codefuseeval来评测，发现java效率非常差，pass@1只有35%。其他语言4/50%. @cyente

[cyente]

FIM方面的评测，我们还在整理中，目前可以先贴一下，Codeqwen1.5 humaneval-fim的评测结果 Language Score java 87.19% py 75.65% js 85.07%

你好，请问有Codeqwen1.5-7b在humaneval-x的评测效果吗？我使用codefuseeval来评测，发现java效率非常差，pass@1只有35%。其他语言4/50%. @cyente

这个指标我们目前还没评测过。你使用的模型是chat还是base？可能有一些后处理格式上的差异。我们后续再去规划指标看看原因。

[kartikzheng]

FIM方面的评测，我们还在整理中，目前可以先贴一下，Codeqwen1.5 humaneval-fim的评测结果 Language Score java 87.19% py 75.65% js 85.07%

你好，请问有Codeqwen1.5-7b在humaneval-x的评测效果吗？我使用codefuseeval来评测，发现java效率非常差，pass@1只有35%。其他语言4/50%. @cyente

这个指标我们目前还没评测过。你使用的模型是chat还是base？可能有一些后处理格式上的差异。我们后续再去规划指标看看原因。

我用的是base，刚发现java生成的部分代码不完整，导致编译不通过。而且会无限制的生成重复的代码，这个问题之前我在其他模型也遇到过，换了微调的模型又正常了，其他都没变。 `import java.util.; import java.lang.;

class Solution { /** Given a positive floating point number, it can be decomposed into and integer part (largest integer smaller than given number) and decimals (leftover part always smaller than 1).

Return the decimal part of the number.
>>> truncateNumber(3.5)
0.5
 */
public double truncateNumber(double number) {

    return number - Math.floor(number);
}
/**
Given a positive floating point number, it can be decomposed into
and integer part (largest integer smaller than given number) and decimals
(leftover part always smaller than 1).
Return the integer part of the number.
>>> getIntegerPart(3.5)
3
 */
public int getIntegerPart(double number) {
    return (int) Math.floor(number);
}
/**
Given a positive floating point number, it can be decomposed into
and integer part (largest integer smaller than given number) and decimals
(leftover part always smaller than 1).
Return the integer part of the number.
>>> getIntegerPart(3.5)
3
 */
public int getIntegerPart(double number) {
    return (int) Math.floor(number);
}
/**
Given a positive floating point number, it can be decomposed into
and integer part (largest integer smaller than given number) and decimals
(leftover part always smaller than 1).
Return the integer part of the number.
>>> getIntegerPart(3.5)
3
 */
public int getIntegerPart(double number) {
    return (int) Math.floor(number);
}
/**
Given a positive floating point number, it can be decomposed into
and integer part (largest integer smaller than given number) and decimals
(leftover part always smaller than 1).
Return the integer part of the number.
>>> getIntegerPart(3.5)
3
 */
public int getIntegerPart(double number) {
    return (int) Math.floor(number);
}
/**
Given a positive floating point number, it can be decomposed into
and integer part (largest integer smaller than given number) and decimals
(leftover part always smaller than 1).
Return the integer part of the number.
>>> getIntegerPart(3.5)
3
 */
public int getIntegerPart(double number) {
    return (int) Math.floor(number);
}
/**
Given a positive floating point number, it can be decomposed into
and integer part (largest integer smaller than given number) and decimals
(leftover part always smaller than 1).
Return the integer part of the number.
>>> getIntegerPart(3.5)
3
 */
public int getIntegerPart(double number) {
    return (int) Math.floor(number);
}
/**
Given a positive floating point number, it can be decomposed into
and integer part (largest integer smaller than given number) and decimals
(leftover part always smaller than 1).
Return the integer part of the number.
>>> getIntegerPart(3.5)
3
 */
public int getIntegerPart(double number) {
    return (int) Math.floor(number);
}
/**
Given a positive floating point number, it can be decomposed into
and integer part (largest integer smaller than given number) and decimals
(leftover part always smaller than 1).
Return the integer part of the number.
>>> getIntegerPart(3.5)
3
 */
public int getIntegerPart(double number) {
    return (int) Math.floor(number);
}
/**
Given a positive floating point number, it can be decomposed into
and integer part (largest integer smaller than given number) and decimals
(leftover part always smaller than 1).
Return the integer part of the number.
>>> getIntegerPart(3.5)
3
 */
public int getIntegerPart(double number) {
    return (int) Math.floor(number);
}
/**
Given a positive floating point number, it can be decomposed into
and integer part (largest integer smaller than given number) and decimals
(leftover part always smaller than 1).
Return the integer part

public class Main { public static void main(String[] args) { Solution s = new Solution(); List correct = Arrays.asList( s.truncateNumber(3.5) == 0.5, Math.abs(s.truncateNumber(1.33) - 0.33) < 1e-6, Math.abs(s.truncateNumber(123.456) - 0.456) < 1e-6 ); if (correct.contains(false)) { throw new AssertionError(); } } }`

[cyente]

@kartikzheng 谢谢反馈，我们尽快检查

[kartikzheng]

@kartikzheng 谢谢反馈，我们尽快检查

您好，如果有结果了，麻烦告知下，多谢。

[kartikzheng]

@kartikzheng 谢谢反馈，我们尽快检查

@cyente 您好，请问您们后来有进行验证吗？在Java语言上，humaneval-x的评测结果是否比较差？