no result

[da-123-snake]

Hi, I am trying to run mBERT on my machine but here comes an error, cmd@zkti:/mnt/disk1/cmd/mBERT$ ./mBERT.sh -in=/mnt/disk1/cmd/defects4j_fixed/Chart/Chart_1_fixed/source/org/jfree/chart/annotations/AbstractAnnotation.java -out=/mnt/disk1/cmd/mBERTm/Chart/Chart1 -N=5 -l=176 [INFO] Scanning for projects... [WARNING] [WARNING] Some problems were encountered while building the effective model for SnT:mBERT:jar:1.0-SNAPSHOT [WARNING] 'build.plugins.plugin.version' for org.apache.maven.plugins:maven-compiler-plugin is missing. @ line 12, column 21 [WARNING] [WARNING] It is highly recommended to fix these problems because they threaten the stability of your build. [WARNING] [WARNING] For this reason, future Maven versions might no longer support building such malformed projects. [WARNING] [INFO] [INFO] -----------------------------< SnT:mBERT >------------------------------ [INFO] Building mBERT 1.0-SNAPSHOT [INFO] from pom.xml [INFO] --------------------------------[ jar ]--------------------------------- [INFO] [INFO] --- exec-maven-plugin:3.2.0:java (default-cli) @ mBERT --- Input: /mnt/disk1/cmd/defects4j_fixed/Chart/Chart_1_fixed/source/org/jfree/chart/annotations/AbstractAnnotation.java Output: /mnt/disk1/cmd/mBERTm/Chart/Chart1 Settings{CODEBERT_TIMEOUT=60, MAX_NUM_OF_MUTANTS=5} SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". SLF4J: Defaulting to no-operation (NOP) logger implementation SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details. Here is the standard error of the command (if any):

Some weights of the model checkpoint at /mnt/disk1/cmd/mBERT/pre-trained/codebert-base-mlm were not used when initializing RobertaForMaskedLM: ['roberta.pooler.dense.bias', 'roberta.pooler.dense.weight']

• This IS expected if you are initializing RobertaForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
• This IS NOT expected if you are initializing RobertaForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model). Here is the standard error of the command (if any):

Some weights of the model checkpoint at /mnt/disk1/cmd/mBERT/pre-trained/codebert-base-mlm were not used when initializing RobertaForMaskedLM: ['roberta.pooler.dense.bias', 'roberta.pooler.dense.weight']

• This IS expected if you are initializing RobertaForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
• This IS NOT expected if you are initializing RobertaForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model). Saving mutants... Settings{CODEBERT_TIMEOUT=60, MAX_NUM_OF_MUTANTS=5} [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 01:00 min [INFO] Finished at: 2024-02-23T15:53:54+08:00 [INFO] ------------------------------------------------------------------------

BUILD SUCCESS! But there has no result!

[da-123-snake]

Python 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] on linux Type "help", "copyright", "credits" or "license" for more information.

import torch

print(torch.version) 2.2.1+cu121

import transformers _)>>> print(transformers.version) 4.38.1 print(torch.cuda.is_available()) True

cmd@zkti:/mnt/disk1/cmd$ mvn -v Apache Maven 3.8.8 (4c87b05d9aedce574290d1acc98575ed5eb6cd39) Maven home: /home/cmd/apache-maven-3.8.8 Java version: 1.8.0_391, vendor: Oracle Corporation, runtime: /home/cmd/jdk1.8.0_391/jre Default locale: en_US, platform encoding: UTF-8 OS name: "linux", version: "5.15.0-52-generic", arch: "amd64", family: "unix"

[michaelkonstantinou]

Hey @da-123-snake ,

I also faced some similar issues and this is what I did to make it work for me:

• I used Java 11
• Python 3.9
• When installing transformers, I found the solution here very helpful as my conda environment couldn't detect the correct transformers version: https://github.com/huggingface/transformers/issues/10277#issuecomment-968069516

I am not sure whether the problem is in Python or in Java, but you can try this out by running the following command: python3 run-codebert.py "int <mask> = b;"

If you don't get the following results then most likely this is a problem with python. Otherwise, it might be from Java

    {'score': 0.23396340012550354, 'token': 740, 'token_str': 'c', 'sequence': 'int c= b;'}

    {'score': 0.05450829118490219, 'token': 939, 'token_str': 'i', 'sequence': 'int i= b;'}

    {'score': 0.05004948750138283, 'token': 741, 'token_str': 'b', 'sequence': 'int b= b;'}

    {'score': 0.04164685681462288, 'token': 10, 'token_str': 'a', 'sequence': 'int a= b;'}

    {'score': 0.023635799065232277, 'token': 181, 'token_str': 'p', 'sequence': 'int p= b;'}

Hope it helps. If not, perhaps you can provide more information in case I missed anything

[rdegiovanni]

Hello @da-123-snake, Sorry for my late reply. Thanks @michaelkonstantinou for your comment.

@da-123-snake I was wondering if you managed to solve the issue. In my case, I was using Java 11, torch:1.10.1 and transformers:4.15.0. But if you need to attach to Java 1.8, you can check this issue: https://github.com/rdegiovanni/mBERT/issues/4#issuecomment-1291692165

Please let us know if you have any further questions.

Best, Renzo