scrapy / scrapyd

A service daemon to run Scrapy spiders
https://scrapyd.readthedocs.io/en/stable/
BSD 3-Clause "New" or "Revised" License
2.98k stars 569 forks source link

how to convert jl to csv? #136

Closed xyniceboy closed 8 years ago

xyniceboy commented 8 years ago

How to convert jl which the item is splited by '\n' to csv? thanks.

{"url": "http://wap.kpi.com/?fr=ad&bid=xrlst-waps-e243aa93e6b6e031", "label1": "\u70ed\u95e8", "label2": "\u641c\u72d7", "label3": "null", "source": "dh.6821.xrlst.com", "host": "wap.xrlst.com"} {"url": "http://m.kll.com/?_trans_=000012_xrlst_mz", "label1": "\u70ed\u95e8", "label2": "\u641c\u72d0", "label3": "null", "source": "dh.6821.xrlst.com", "host": "m.sohu.com"} {"url": "http://sdfa.cn", "label1": "\u70ed\u95e8", "label2": "\u65b0\u6d6a", "label3": "null", "source": "dh.6821.xrlst.com", "host": "sina.cn"} {"url": "http://3g.xert.com", "label1": "\u70ed\u95e8", "label2": "\u51e4\u51f0", "label3": "null", "source": "dh.6821.xrlst.com", "host": "3g.ifeng.com"} {"url": "http://m.oasdfb.com", "label1": "\u70ed\u95e8", "label2": "\u6dd8\u5b9d", "label3": "null", "source": "dh.6821.xrlst.com", "host": "m.oasdfb.com"} {"url": "http://m.kan.dfa.com", "label1": "\u70ed\u95e8", "label2": "\u5f71\u89c6", "label3": "null", "source": "dh.6821.xrlst.com", "host": "m.kan.xrlst.com"} {"url": "http://s.click.oasdfb.com/t?e=m%3D2%26s%3DdZM0BLTzuFYcQipKwQzePCperVdZeJviK7Vc7tFgwiFRAdhuF14FMVcLWlqfoDVv1aH1Hk3GeOhbpnDuwQww1MeQPEbgvEegqK%2BYx3SKaIIQGcQGTfFobKUuZxIcp9pfUIgVEmFmgnbDX0%2BHH2IEVa7A5ve%2FEYDnFveQ9Ld2jopwTqWNBsAwm%2BIKl4JSR4lzomfkDJRs%2BhU%3D", "label1": "\u70ed\u95e8", "label2": "\u5929\u732b", "label3": "null", "source": "dh.6821.xrlst.com", "host": "s.click.oasdfb.com"}

Digenis commented 8 years ago

This question is not related to scrapyd or scrapy at all. Scrapy allows you to directly export items in CSV http://doc.scrapy.org/en/latest/topics/feed-exports.html#topics-feed-format-csv If this doesn't cover you, try searching the web for json/json-lines to csv conversion.