Open peeyee opened 1 year ago
Item No.6 kdb+'s tasks: Is it referring to direct file access (similar to loadFile
), bypassing q process (similar to loadTable
)?
Yes, the way of loadTable/File is totally loading the data into DolphinDB's in-memory table. Thus comes a problem if we do not have enough memory,especially for community users. So we want to have a method loadTextEx that directly loading data into DolphinDB's DFS table, meanwhile reduce the memory usage, for example, each batch costs 512MB.
To do list for new Contributors
Contributions are welcome, here is a list for new contributors:
No | Plugin | Content | Priority | Difficulty | Contributer | Status | Issue/PR -- | -- | -- | -- | -- | -- | -- | -- 1 | clickhouse | | low | medium | | | 2 | hdf5 | add docs about the perfect way of loading data and use cases. | medium | low | | | 3 | httpClient | support for windows | high | medium | | | 4 | kafka | support for kerberos | low | medium | | | 5 | kafka | support for windows | high | medium | | | 6 | kdb | support for using loadTextEx | medium | medium | | | 7 | kdb | support for reading compressed data of type q IPC (1), snappy (3), L4Z (4) | high | high | | | 8 | mqtt | support for arm | low | medium | | | 9 | mqtt | publish stream engine calculation results directly to the mqtt server | medium | medium | | | 10 | mysql | add docs about return type | medium | low | | | 11 | mysql | support for type blob | high | high | | | 12 | mysql | support for type decimal | high | high | | | 13 | oracle | | high | high | | | 14 | parquet | support for reading column of type repeated | high | high | | | 15 | parquet | support for windows | high | medium | | | 16 | signal | support for rfft, irfft | high | high | | | 17 | xgboost | upgrade the library of xgboost to version 1.7 | medium | medium | | | 18 | zip | support for arm | low | medium | | | 19 | zip | support for compressing folders | high | medium | | |