Open CS123n opened 1 year ago
I have the same confusion. Could you provide some codes about data processing?
After downloading, I have a 100GB JSON file. How can I use it for training?
[root@VM-117-206 LAION-Glyph-1M]# ll -h total 100G -rw-r--r-- 1 root root 100G Aug 23 18:38 LAION-Glyph-1M.json
Hi, Could you please tell me where did you download the datas?
After downloading, I have a 100GB JSON file. How can I use it for training?
[root@VM-117-206 LAION-Glyph-1M]# ll -h total 100G -rw-r--r-- 1 root root 100G Aug 23 18:38 LAION-Glyph-1M.json