gangly / datafaker

Datafaker is a large-scale test data and flow test data generation tool. Datafaker fakes data and inserts to varied data sources. 测试数据生成工具
625 stars 168 forks source link

写入文件使用--format json参数报错 #81

Closed Keene-Chen closed 1 year ago

Keene-Chen commented 3 years ago

根据示例 datafaker file e:\zm\home out1.txt 10 --meta meta.txt --format json报错

Traceback (most recent call last): File "E:\Scoop\apps\python\current\lib\site-packages\datafaker\cli.py", line 87, in main args = parse_args() File "E:\Scoop\apps\python\current\lib\site-packages\datafaker\cli.py", line 65, in parse_args raise ParamError('rdb not support for json format') datafaker.exceptions.ParamError: rdb not support for json format rdb not support for json format

然后我根据拉取请求 #72 修改not in逻辑后,可以正常生成json数据,但是生成的数据又是ASCII不是UTF-8。 希望给个提示,谢谢!

生成数据如下:

{"id": 1, "name": "\u5f20\u5229", "school": "\u5317\u4eac\u5927\u5b66", "nickname": "\u9b3c\u6ce3", "age": 49, "class_num": 82, "score": 86.17, "phone": "15030834802", "email": "liaoxia@ly.cn", "ip": "198.51.47.62", "address": "\u5e7f\u897f\u58ee\u65cf\u81ea\u6cbb\u533a\u6625\u6885\u53bf\u6881\u5e73\u9ad8\u8857N\u5ea7 476777"}

{"id": 2, "name": "\u5510\u6d77\u71d5", "school": "\u5317\u4eac\u5927\u5b66", "nickname": "\u9ad8\u5c0f\u738b\u5b50", "age": 71, "class_num": 12, "score": 42.81, "phone": "14722661894", "email": "zlin@guiyingxiong.cn", "ip": "193.82.145.217", "address": "\u5c71\u4e1c\u7701\u67f3\u53bf\u9521\u5c71\u516d\u76d8\u6c34\u8857x\u5ea7 322680"}

Keene-Chen commented 3 years ago

我看到解决办法了😅,后面有同样报错的同学可以尝试修改 #72 和 https://github.com/gangly/datafaker/issues/78#issuecomment-866712048 两处的方法,终于解决问题了😴

使用datafaker file e:\zm\home out1.txt 10 --meta meta.txt --format json命令 meta.txt文件内容如下:

id||int||自增id[:inc(id,1)] name||varchar(20)||学生名字[:name] school||varchar(20)||学校名字[:enum(清华大学,北京大学)] nickname||varchar(20)||学生小名[:enum(鬼泣, 高小王子, 歌神, 逗比)] age||int||学生年龄[:age] class_num||int||班级人数[:int(10, 100)] score||decimal(4,2)||成绩[:decimal(4,2,1)] phone||bigint||电话号码[:phone_number] email||varchar(64)||家庭网络邮箱[:email] ip||varchar(32)||IP地址[:ipv4] address||text||家庭地址[:address]

最后是生成的数据如下:

{"id": 1, "name": "杜梅", "school": "清华大学", "nickname": "高小王子", "age": 25, "class_num": 39, "score": 3.0, "phone": "18556570354", "email": "smo@gmail.com", "ip": "117.54.105.186", "address": "四川省成都市浔阳张街z座 710240"}

{"id": 2, "name": "陈淑英", "school": "清华大学", "nickname": "逗比", "age": 23, "class_num": 89, "score": 51.28, "phone": "13200140438", "email": "yonglai@jingli.cn", "ip": "198.51.59.108", "address": "上海市俊市新城兴城路C座 509477"}

mqqgithub commented 3 years ago

78修改后报错了

D:>datafaker file D:\ out.txt 10 --meta meta.txt --format json Exception in thread Thread-2: Traceback (most recent call last): File "D:\dev\python\lib\threading.py", line 917, in _bootstrap_inner self.run() File "D:\dev\python\lib\threading.py", line 865, in run self._target(*self._args, **self._kwargs) File "D:\dev\python\lib\site-packages\datafaker\dbs\basedb.py", line 122, in save self.save_data(lines) File "D:\dev\python\lib\site-packages\datafaker\dbs\filedb.py", line 24, in save_data save2file(items, filepath) File "D:\dev\python\lib\site-packages\datafaker\utils.py", line 26, in save2file fp.writelines(items) UnicodeEncodeError: 'gbk' codec can't encode character '\u3dab' in position 41: illegal multibyte sequence