search

mnt-ltd / moredoc

moredoc，魔豆文库，基于golang开发的类似百度文库的开源文库系统，dochub文库的重构版本。
https://moredoc.mnt.ltd
Apache License 2.0
443 stars 106 forks source link

中文文档上传后发生乱码问题 #3

Closed bluezealot closed 1 year ago

bluezealot commented 1 year ago

中文文档上传后发生乱码问题，请看图。 我的操作系统是ubuntu, MoreDoc版本是：v1.0.0。请问是不是缺少什么包？ image 错误日志：

023/02/23 11:17:55 /home/truthhun/git.mnt.ltd/moredoc/model/document.go:573 Error 1055: Expression #2 of SELECT list is not in GROUP BY clause and contains nonaggregated column 'moredoc_test.a.type_id' which is not functionally dependent on columns in GROUP BY clause; this is incompatible with sql_mode=only_full_group_by
[1.325ms] [rows:0] select a.hash,a.type_id from mnt_attachment a left join mnt_document d on a.type_id = d.id where a.hash in ('ea673f9254b52333f4f096dfa74893fc') and d.status = 2 group by a.hash
2023-02-23T11:17:55.092Z    error   model   model/document.go:575   GetDocumentStatusConvertedByHash    {"error": "Error 1055: Expression #2 of SELECT list is not in GROUP BY clause and contains nonaggregated column 'moredoc_test.a.type_id' which is not functionally dependent on columns in GROUP BY clause; this is incompatible with sql_mode=only_full_group_by"}
moredoc/model.(*DBModel).GetDocumentStatusConvertedByHash
    /home/truthhun/git.mnt.ltd/moredoc/model/document.go:575
moredoc/model.(*DBModel).ConvertDocument
    /home/truthhun/git.mnt.ltd/moredoc/model/document.go:627
moredoc/model.(*DBModel).loopCovertDocument
    /home/truthhun/git.mnt.ltd/moredoc/model/util.go:264
2023-02-23T11:17:55.092Z    info    model.converter converter/converter.go:224  convert to pdf by soffice   {"cmd": "soffice", "args": ["--headless", "--convert-to", "pdf", "--outdir", "cache/convert/2023/02/23/ea673f9254b52333f4f096dfa74893fc", "documents/e/a/6/7/3/ea673f9254b52333f4f096dfa74893fc.pptx"]}
2023-02-23T11:18:00.143Z    info    model.converter converter/converter.go:260  count pdf pages {"cmd": "mutool", "args": ["show", "cache/convert/2023/02/23/ea673f9254b52333f4f096dfa74893fc/ea673f9254b52333f4f096dfa74893fc.pdf", "pages"]}
2023-02-23T11:18:00.147Z    debug   model.converter converter/converter.go:272  count pdf pages {"line": ""}
2023-02-23T11:18:00.147Z    debug   model.converter converter/converter.go:272  count pdf pages {"line": "page 9 = 57 0 r"}
2023-02-23T11:18:00.147Z    info    model.converter converter/converter.go:192  convert pdf to page {"cmd": "mutool", "args": ["convert", "-o", "cache/convert/2023/02/23/ea673f9254b52333f4f096dfa74893fc/%d.png", "cache/convert/2023/02/23/ea673f9254b52333f4f096dfa74893fc/ea673f9254b52333f4f096dfa74893fc.pdf", "1-1"]}
2023-02-23T11:18:00.196Z    info    model.converter converter/converter.go:192  convert pdf to page {"cmd": "mutool", "args": ["convert", "-o", "cache/convert/2023/02/23/ea673f9254b52333f4f096dfa74893fc/%d.svg", "cache/convert/2023/02/23/ea673f9254b52333f4f096dfa74893fc/ea673f9254b52333f4f096dfa74893fc.pdf", "1-9"]}
2023-02-23T11:18:00.266Z    debug   model   model/document.go:703   ConvertDocument CopyFile    {"src": "cache/convert/2023/02/23/ea673f9254b52333f4f096dfa74893fc/1.gzip.svg", "dst": "documents/e/a/6/7/3/ea673f9254b52333f4f096dfa74893fc/1.gzip.svg"}
2023-02-23T11:18:00.266Z    debug   model   model/document.go:703   ConvertDocument CopyFile    {"src": "cache/convert/2023/02/23/ea673f9254b52333f4f096dfa74893fc/2.gzip.svg", "dst": "documents/e/a/6/7/3/ea673f9254b52333f4f096dfa74893fc/2.gzip.svg"}
2023-02-23T11:18:00.266Z    debug   model   model/document.go:703   ConvertDocument CopyFile    {"src": "cache/convert/2023/02/23/ea673f9254b52333f4f096dfa74893fc/3.gzip.svg", "dst": "documents/e/a/6/7/3/ea673f9254b52333f4f096dfa74893fc/3.gzip.svg"}
2023-02-23T11:18:00.267Z    debug   model   model/document.go:703   ConvertDocument CopyFile    {"src": "cache/convert/2023/02/23/ea673f9254b52333f4f096dfa74893fc/4.gzip.svg", "dst": "documents/e/a/6/7/3/ea673f9254b52333f4f096dfa74893fc/4.gzip.svg"}
2023-02-23T11:18:00.267Z    debug   model   model/document.go:703   ConvertDocument CopyFile    {"src": "cache/convert/2023/02/23/ea673f9254b52333f4f096dfa74893fc/5.gzip.svg", "dst": "documents/e/a/6/7/3/ea673f9254b52333f4f096dfa74893fc/5.gzip.svg"}
2023-02-23T11:18:00.267Z    debug   model   model/document.go:703   ConvertDocument CopyFile    {"src": "cache/convert/2023/02/23/ea673f9254b52333f4f096dfa74893fc/6.gzip.svg", "dst": "documents/e/a/6/7/3/ea673f9254b52333f4f096dfa74893fc/6.gzip.svg"}
2023-02-23T11:18:00.267Z    debug   model   model/document.go:703   ConvertDocument CopyFile    {"src": "cache/convert/2023/02/23/ea673f9254b52333f4f096dfa74893fc/7.gzip.svg", "dst": "documents/e/a/6/7/3/ea673f9254b52333f4f096dfa74893fc/7.gzip.svg"}
2023-02-23T11:18:00.268Z    debug   model   model/document.go:703   ConvertDocument CopyFile    {"src": "cache/convert/2023/02/23/ea673f9254b52333f4f096dfa74893fc/8.gzip.svg", "dst": "documents/e/a/6/7/3/ea673f9254b52333f4f096dfa74893fc/8.gzip.svg"}
2023-02-23T11:18:00.268Z    debug   model   model/document.go:703   ConvertDocument CopyFile    {"src": "cache/convert/2023/02/23/ea673f9254b52333f4f096dfa74893fc/9.gzip.svg", "dst": "documents/e/a/6/7/3/ea673f9254b52333f4f096dfa74893fc/9.gzip.svg"}
2023-02-23T11:18:00.268Z    info    model.converter converter/converter.go:123  convert pdf to txt  {"cmd": "mutool", "args": ["convert", "-o", "cache/convert/2023/02/23/ea673f9254b52333f4f096dfa74893fc/dst.txt", "cache/convert/2023/02/23/ea673f9254b52333f4f096dfa74893fc/ea673f9254b52333f4f096dfa74893fc.pdf"]}

2023/02/23 11:18:00 /home/truthhun/git.mnt.ltd/moredoc/model/document.go:723
[99.492ms] [rows:1] UPDATE `mnt_document` SET `description`='初初 AI  • 初初 • 初初 初初初 一  • 初初初初初初  初初 • 初初初初 -- 初初初  1943 初初初  初初初 Input 初  初初初 Output 初  初初初初初初初  初初初初初初初初初初初初初初初初初初初初 初初初初初初初初初初初 初初初初初初初初初初初初初初初初初初初初初初初初初初初初初初初初初 一。。  初初初初初初初初初初初初初初初初初初初初初初初初初初初初初初初初初 初初初初初初初初初初 一。 1943 初初初初初初初初初初初初初初初初初初初初初初 初初初初初 。 A',`width`=1280,`height`=720,`preview`=0,`pages`=9,`status`=2,`updated_at`='2023-02-23 11:18:00.278',`enable_gzip`=true WHERE id = 5 AND `mnt_document`.`deleted_at` IS NULL AND `id` = 5
2023-02-23T11:18:00.375Z    info    model.converter converter/converter.go:362  clean workspace {"workspace": "cache/convert/2023/02/23/ea673f9254b52333f4f096dfa74893fc"}
2023-02-23T11:18:00.376Z    info    model.converter converter/converter.go:367  clean workspace success {"workspace": "cache/convert/2023/02/23/ea673f9254b52333f4f096dfa74893fc"}
2023-02-23T11:18:00.376Z    info    model   model/util.go:263   loopCovertDocument，start...
bluezealot commented 1 year ago

执行下面的安装后解决了乱码问题

apt-get update \
  && apt-get install -y language-pack-zh-hans \
  && apt-get install -y language-pack-ja \
  && apt-get install -y chinese* \
  && apt-get install -y libreoffice-l10n-zh-cn libreoffice-help-zh-cn \
  && apt-get install -y libreoffice-help-ja \
  && rm -rf /var/lib/apt/lists/*