search

yangheng95 / PyABSA

Sentiment Analysis, Text Classification, Text Augmentation, Text Adversarial defense, etc.;
https://pyabsa.readthedocs.io
MIT License
923 stars 159 forks source link

Question about the version of the package used by the framework #186

Closed yaoysyao closed 2 years ago

yaoysyao commented 2 years ago

Hello, excuse me

  1. It is not convenient for the party to write a document listing the versions of each package used by the framework.
  2. One more question, will the packages used by the framework be updated in a timely manner? For example, if the torch is upgraded to 1.11.0, will the framework be updated in a timely manner?
yangheng95 commented 2 years ago

setup.py specifies the version of some packages, such as torch<1.11.0 (known issues found). If no version is specified of some packages in setup.py, use the latest version of the packages. If anyone report that the latest version of any package does not work, please feedback and I will solve the problem

yaoysyao commented 2 years ago

setup.py specifies the version of some packages, such as torch<1.11.0 (known issues found). If no version is specified of some packages in setup.py, use the latest version of the packages. If anyone report that the latest version of any package does not work, please feedback and I will solve the problem 好的，谢谢，我下载了源码，阅读源码发现注释比较少，另外，api文档似乎也不是特别多，不过基本使用没问题，按照示例已经跑起来了，不知道后期有没有计划编写详细的api文档？

yaoysyao commented 2 years ago

setup.py specifies the version of some packages, such as torch<1.11.0 (known issues found). If no version is specified of some packages in setup.py, use the latest version of the packages. If anyone report that the latest version of any package does not work, please feedback and I will solve the problem

请问框架默认的参数需要什么GPU条件才可以，我的机器跑起来以后出现了：RuntimeError: CUDA out of memory. Tried to allocate 376.00 MiB (GPU 0; 4.00 GiB total capacity; 3.15 GiB already allocated; 0 bytes free; 3.32 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF 我尝试修改模型参数如下： _atepc_config_english = {'model': LCF_ATEPC, 'optimizer': "adamw", 'learning_rate': 0.00002, 'pretrained_bert': "microsoft/deberta-v3-base", 'cache_dataset': True, 'warmup_step': -1, 'use_bert_spc': False, 'show_metric': False, 'max_seq_len': 10, 'SRD': 3, 'use_syntax_based_SRD': False, 'lcf': "cdw", 'window': "lr", 'dropout': 0.5, 'l2reg': 0.00001, 'num_epoch': 10, 'batch_size': 4, 'initializer': 'xavier_uniform_', 'seed': 52, 'polarities_dim': 2, 'log_step': 50, 'patience': 99999, 'gradient_accumulation_steps': 1, 'dynamic_truncate': True,'srd_alignment': True, # for srd_alignment 'evaluate_begin': 0} 降低了训练的batch_size等参数，但是还是出现该错误

yangheng95 commented 2 years ago

这是因为显存不足，如果想试试跑起来的话可以试试设置batch_size=1, max_seq_len=40。 目前这个库是我一个人维护，也不是主要主要项目，所以写API文档是一个耗时的过程，后期看有没有人能合作完成这件事

yaoysyao commented 2 years ago

这是因为显存不足，如果想试试跑起来的话可以试试设置batch_size=1, max_seq_len=40。 目前这个库是我一个人维护，也不是主要主要项目，所以写API文档是一个耗时的过程，后期看有没有人能合作完成这件事

好的好的，辛苦了，如果文档写起来有些耗时，可以考虑在代码中增加注释，然后让用户自己根据源码去进行修改之类的，我刚才看代码，发现注释很少，就比如我想使用已经训练好的模型，修改代码不知道怎么修改，如果电脑配置比较好的，可能直接跑就行，但是如果需要自定义配置的话，确实用起来有点费劲，大佬辛苦了

yaoysyao commented 2 years ago

这是因为显存不足，如果想试试跑起来的话可以试试设置batch_size=1, max_seq_len=40。 目前这个库是我一个人维护，也不是主要主要项目，所以写API文档是一个耗时的过程，后期看有没有人能合作完成这件事

确实写文档很耗时，也可以抽时间写写基本就行，比如怎么设置模型参数，怎么更换模型，怎么使用自己的数据集，怎么加载已经训练好的数据集，能保证基本使用就行

yaoysyao commented 2 years ago

如果我想加载已经预训练过得模型，这样的话代码有例子吗？

yaoysyao commented 2 years ago

查看了源代码，加载预训练的模型代码如下： aspect_extractor = ATEPCCheckpointManager.get_aspect_extractor(checkpoint='E:\Download\BaiDuDriveDownload\PYABSA\V0.8.8.0\English\ATEPC\fast_lcf_atepc_cdw_apcacc_85.62_apcf1_76.87_atef1_81.1.zip')

但是在执行过程中，出现了： Traceback (most recent call last): File "F:\Myjob\Anaconda\lib\site-packages\pyabsa\core\atepc\prediction\aspect_extractor.py", line 60, in init self.opt = pickle.load(f) ModuleNotFoundError: No module named 'pyabsa.tasks' 除了下载网盘中的已经预训练过得模型，配置模型的路径，还需要其他的配置吗？

yangheng95 commented 2 years ago

请使用Google Drive上的模型，百度网盘很久没有维护过了

yangheng95 commented 2 years ago

我后面回重新上传至百度网盘，但这速度感人

yaoysyao commented 2 years ago

我后面回重新上传至百度网盘，但这速度感人

Google Drive的地址是checkpoint-v1.2.json这个json文件中的地址吗，我去下载一下

yaoysyao commented 2 years ago

请使用Google Drive上的模型，百度网盘很久没有维护过了

地址好像不太对，麻烦发一下Google Drive地址，谢谢

yangheng95 commented 2 years ago

你可以使用available_checkpoints函数查询可用模型，但需要能连接drive

yangheng95 commented 2 years ago

https://drive.google.com/drive/folders/1yiMTucHKy2hAx945lgzhvb9QeHvJrStC

yaoysyao commented 2 years ago

https://drive.google.com/drive/folders/1yiMTucHKy2hAx945lgzhvb9QeHvJrStC

收到，谢谢