Too many new lines

[andresriancho]

The migration tool did a great job, but it kept some new lines we don't need. Also, when we '\n'.join(description) we add more new lines resulting on markdown that's 100% valid but hard to read for humans:

It seems that we need to convert the json files again, finding all simple new lines (\n) and removing them. The "double new lines" (\n\n) should remain untouched.

[m0sth8]

According to database design the right way to join multilines is ' '.join(description)

On Monday, March 30, 2015, Andres Riancho notifications@github.com wrote:

The migration tool did a great job, but it kept some new lines we don't need. Also, when we '\n'.join(description) we add more new lines resulting on markdown that's 100% valid but hard to read for humans:

[image: too-many-new-lines] https://cloud.githubusercontent.com/assets/865200/6899746/b9ff308a-d6d8-11e4-903a-497f176d40d8.png

It seems that we need to convert the json files again, finding all simple new lines (\n) and removing them. The "double new lines" (\n\n) should remain untouched.

— Reply to this email directly or view it on GitHub https://github.com/vulndb/data/issues/14.

[andresriancho]

Not sure what you intended to write there, or the email did something strange?

[m0sth8]

Sorry, I mean than you have to join strings with empty space like described in https://github.com/vulndb/data/issues/5

"So, the parser should check if the description and solution fields are strings or arrays. If they are arrays, the contents of the array should be joined using an empty space"

[andresriancho]

See Join multiline strings with '' , ' ' or '\n'? #16

After reading that, read https://github.com/vulndb/data/blob/master/db/45-sql-injection.json , pay special attention to https://github.com/vulndb/data/blob/master/db/45-sql-injection.json#L7 . In the python-sdk I'm using \n to join the strings (because I was unsure about #16), so the result for the first lines is:

  "description": [
    "Due to the requirement for dynamic content of today's web", 
    "applications, many\nrely on a database backend to store data that will", ...

Result text: Due to the requirement for dynamic content of today's web\napplications, many\nrely on a database backend to store data that will

Printed text:

Due to the requirement for dynamic content of today's web
applications, many
rely on a database backend to store data that will

Rendered markdown as html:

<p>Due to the requirement for dynamic content of today's web applications, many rely on a database backend to store data that will</p>

Which is similar to what's shown in the screenshot. So the \n after "many" was inherited from Arachni and is not really needed (IMHO), since it only adds new lines to the text that markdown will ignore anyways

It doesn't matter if we decide to join with \n or ' ', but those new lines (IMHO) don't make any sense

Of course the double new lines we imported from arachni do make sense, since they are new paragraphs for markdown

[m0sth8]

Oh, I see =) that makes sense.

[andresriancho]

Your opinion on #16 ?