nicholasdavidson / pybit

Python Build Integration Toolkit - a distributed cross platform AMQP based build system
17 stars 5 forks source link

Version strings must be allowed to contain special characters #97

Closed ghost closed 11 years ago

ghost commented 11 years ago

Version strings are allowed to contain various special characters that are currently being replaced with spaces by pybit-web. This results in the version string being sent to the client containing space which then breaks all file-name constructions on that client:

(cd /home/codehelp/pybit/development/svn/lwconfd && dpkg-buildpackage -nc -S -d -uc -us) >> /home/codehelp/pybit/logs/lwconfd_0.0.2.3 nmu1-i386-1356008937 2>&1 returned error: 2

The command fails because nm1-i386 now looks like a command and even if this was handled, the resulting changes file for upload will be lwconfd_0.0.2.3+nmu1_i386.changes but the client won't find that file.

jamesbennet commented 11 years ago

So the problem is that it replaces the underscore with a space? Do you have any idea where in pybit-web they get replaced?

It doesn't seem to be remove_nasties() in db.py (A helper we use to escape <,>,&, which could otherwise mess us up)

Perhaps it is being stripped when we insert into the database?

ghost commented 11 years ago

bottle.py


def _parse_qsl(qs):
    r = []
    for pair in qs.replace(';','&').split('&'):
        if not pair: continue
        nv = pair.split('=', 1)
        if len(nv) != 2: nv.append('')
        key = urlunquote(nv[0].replace('+', ' '))
        value = urlunquote(nv[1].replace('+', ' '))
        r.append((key, value))
    return r
ghost commented 11 years ago

The problem is not replacing underscore with space (underscore is not allowed in version strings or package names or architecture names) but + and ~ are allowed and must be retained / replaced.

jamesbennet commented 11 years ago

We don't use that _parse_qsl() funtion in bottle.py though, we use cgi.escape() [To stop XSS injection of malicious i.e. javascript code, into the database] in combination with the paramaterisation feaures of psycopg2 [To stop SQL injection].

I just made a package called "test" with a version "+" and another with "+stuff~". It worked? - I can get that JSOn just fine.

Ah, so its with the url encoding/decoding.

Use --data-urlencode in the curl in hook, its a problem that end. ?

ghost commented 11 years ago

The curl command needs to encode the strings prior to being sent.

DATASTR="${DATASTR} --data-urlencode distribution=${DISTRO_NAME}"