search

Gmousse / dataframe-js

No Maintenance Intended
https://gmousse.gitbooks.io/dataframe-js/
MIT License
460 stars 38 forks source link

[BUG]utf8-bom编码引起的第一列无法选择 #96

Closed southwhale closed 4 years ago

southwhale commented 5 years ago

node环境中，传入DataFrame.fromCSV的文件是utf8-bom编码的，文件内容如下： | symbol | direction | offset | price | volume | time |

| IF1304 | 　卖 | 平仓 | 2478.6 | 1 | 11:27:52 | | IF1304 | 　卖 | 平仓 | 2478.6 | 1 | 11:27:52 | | IF1304 | 买　 | 开仓 | 2477.2 | 1 | 11:27:39 | | IF1304 | 买　 | 开仓 | 2477.2 | 1 | 11:27:39 | | IF1304 | 　卖 | 平仓 | 2477 | 1 | 11:27:37 | | IF1304 | 　卖 | 平仓 | 2477 | 1 | 11:27:37 | | IF1304 | 　卖 | 平仓 | 2477 | 1 | 11:27:36 | | IF1304 | 买　 | 开仓 | 2477.2 | 1 | 11:27:18 | | IF1304 | 买　 | 开仓 | 2477.2 | 1 | 11:27:18 | | IF1304 | 买　 | 开仓 | 2476.8 | 1 | 11:27:17 |

当调用df.select('symbol', 'direction')时报错：(node:60500) UnhandledPromiseRejectionWarning: NoSuchColumnError: symbol not found in [symbol, direction, offset, price, volume, time].

当把文件编码改成utf8，则正常。 那么，是否可以做到兼容utf8-bom编码的文件？

谢谢！

southwhale commented 5 years ago

如果我改成df.select('\uFEFFsymbol', 'direction')，则不会报错，由此可见的确是因为bom引起的问题。

southwhale commented 5 years ago

我修改了 io.js中的代码： `function fromDSV(pathOrFile) { var sep = arguments.length > 1 && arguments[1] !== undefined ? arguments[1] : ";"; var header = arguments.length > 2 && arguments[2] !== undefined ? arguments[2] : true; var parser = (0, _d3Dsv.dsvFormat)(sep); return new Promise(function (resolve) { var parseText = function parseText(fileContent) { if (fileContent.indexOf('\uFEFF') === 0) { fileContent = fileContent.replace('\uFEFF', ''); } if (fileContent.includes("Error: ENOENT")) return resolve(null); var data = header ? parser.parse(fileContent) : parser.parseRows(fileContent); return resolve(data); };

return typeof pathOrFile === "string" ? (0, _d3Request.text)(addFileProtocol(pathOrFile), parseText) : loadTextFile(pathOrFile, parseText);

})`

加入了加粗的部分，暂时这样解决。希望后续版本能修复该问题。

Gmousse commented 5 years ago

hi @iamweilee !

Thank you for your issue.

Next time, could you use the template and write it in english please ?

Il will have a look at your PR https://github.com/Gmousse/dataframe-js/pull/97.