425776024 / nlpcda

一键中文数据增强包 ; NLP数据增强、bert数据增强、EDA:pip install nlpcda
Apache License 2.0
1.78k stars 169 forks source link

simbert相似句的生成速度太慢 #8

Open pc123s opened 3 years ago

pc123s commented 3 years ago

simbert进行相似句的生成速度太慢了,其中设备选用的cuda

425776024 commented 3 years ago

simbert进行相似句的生成速度太慢了,其中设备选用的cuda

使用的时候nvidia-smi的GPU计算率有超过0%吗,你可以加我qq,等我有空了详细看看

425776024 commented 3 years ago

simbert进行相似句的生成速度太慢了,其中设备选用的cuda

另外,之前有个小哥是这样用的。。。肯定慢。。每次从新加载模型:


for i in range(len(x)):
        sent = x[i]
        simbert = Simbert(config=config)
        synonyms = simbert.replace(sent=sent, create_num=5)
        print(synonyms)
svjack commented 3 years ago

Does it only support sentence length below 32, the source code of it truncated the input at this length.

425776024 commented 3 years ago

Does it only support sentence length below 32, the source code of it truncated the input at this length.

you can edit config 'max_len': 32

svjack commented 3 years ago

Does it only support sentence length below 32, the source code of it truncated the input at this length.

you can edit config 'max_len': 32

i say is the source code of simbert, if i require longer length i don't have data to train simbert by myself. does simbert support longer text?

Robert130129 commented 3 years ago

请问现在是不支持simbert了吗?

425776024 commented 3 years ago

请问现在是不支持simbert了吗?

不是不支持,是默认不安装simbert所需要的keras、tensorflow、bert4keras这些依赖包了,不然pip install nlpcda的时候太庞大了,得下几百兆。希望使用的可以自己去安装依赖

Robert130129 commented 3 years ago

请问现在是不支持simbert了吗?

不是不支持,是默认不安装simbert所需要的keras、tensorflow、bert4keras这些依赖包了,不然pip install nlpcda的时候太庞大了,得下几百兆。希望使用的可以自己去安装依赖

多谢,我之前tf和keras版本没对上

physics39 commented 3 years ago

请问安装tensorflow-gpu的版本也是2.2.0吗?我之前安装了tensorflow2.2.0是可以跑成功simbert的,但是发现并没有在gpu上运行,后来换成tensorflow-gpu==2.2.0,发现simbert不能正常使用了。

425776024 commented 3 years ago

请问安装tensorflow-gpu的版本也是2.2.0吗?我之前安装了tensorflow2.2.0是可以跑成功simbert的,但是发现并没有在gpu上运行,后来换成tensorflow-gpu==2.2.0,发现simbert不能正常使用了。

忘记提示simbert的版本了,测试报错、有误、GPU没跑起来的,可以试试这组版本:

keras==2.3.1
bert4keras==0.7.7
# tensorflow==1.13.1
tensorflow-gpu==1.13.1
ioslide commented 3 years ago

https://github.com/ZhuiyiTechnology/roformer-sim/blob/main/test/generate.py

xiaokai01 commented 1 year ago

一秒大概能够生产多少数据量