svnlog2sqlite aborts with "you must not use 8-bit bytestrings" due to special chars in the SVN commit paths?

[GoogleCodeExporter]

What steps will reproduce the problem?

1. Trying to call svnlog2sqlite on repositories containing national
characters in the commit log (file names)?

What is the expected output? What do you see instead?

Error You must not use 8-bit bytestrings unless you use a text_factory that
can interpret 8-bit bytestrings (like text_factory = str). It is highly
recommended that you instead just switch y
our application to Unicode strings.
Traceback (most recent call last):
  File "c:\Programme\Python\Lib\site-packages\svnplot\svnlog2sqlite.py",
line 42, in convert
    self.ConvertRevs(startrevno, endrevno, bUpdLineCount, maxtrycount)
  File "c:\Programme\Python\Lib\site-packages\svnplot\svnlog2sqlite.py",
line 94, in ConvertRevs
    values(?, ?, ?, ?,?,?)", (revlog.revno, filename, changetype,
linesadded, linesdeleted, lc_updated))
ProgrammingError: You must not use 8-bit bytestrings unless you use a
text_factory that can interpret 8-bit bytestrings (like text_factory =
str). It is highly recommended that you instead j
ust switch your application to Unicode strings.
Trying again (1)

What version of the product are you using? On what operating system?
- SVNPlot-0.5.10.win32.exe
- python-2.6.2.msi
- py26-pysvn-svn156-1.7.1-1233.exe
- numpy-1.3.0rc2-win32-superpack-python2.6.exe
- matplotlib-0.99.1.win32-py2.6.exe

Original issue reported on code.google.com by B.Sch...@exxcellent.de on 28 Sep 2009 at 2:59

[GoogleCodeExporter]

please try replacing 'filename' to unicode(filename). I will check and put more 
cleaner fix soon.

Original comment by nitinbh...@gmail.com on 28 Sep 2009 at 5:08

[GoogleCodeExporter]

Hi Nitin! Thanks for you quick response. I tried to replace filename in the 
insert
statement. Unfortunately this leads to another problem.

Error 'ascii' codec can't decode byte 0xc3 in position 68: ordinal not in 
range(128)
Traceback (most recent call last):

The problem seems to start in Commits where we used Umlaut in Filenames but not 
only
on those.

Original comment by B.Sch...@exxcellent.de on 29 Sep 2009 at 9:56

[GoogleCodeExporter]

Hi,
Is there any opensource repository, where I can reproduce this problem ? or can 
you create a dummy project on google code with filenames similar to what you 
encounters with may be 3-4 versions ?

Original comment by nitinbh...@gmail.com on 29 Sep 2009 at 1:23

[GoogleCodeExporter]

Hi Nitin,

thank you for your support.

Try this here: svnlog2sqlite.py  -l -g http://svnplot-dummy.googlecode.com/svn/
c:\tmp\dummy-project

Cheers
- Ben

Original comment by B.Sch...@exxcellent.de on 29 Sep 2009 at 1:36

[GoogleCodeExporter]

Ben,

I could not reproduce this bug on PySVN 1.7 (SVN 1.6 version). 

This seems to be similar to bug reported in pysvn built again SVN 1.5 (check 
http://pysvn.tigris.org/issues/show_bug.cgi?id=104). 
Some time back I added the work around for 'message' strings. However, I think 
similar bug exists for filename as well. Since the filenames are used any many 
place (including the directory paths), adding the fix and testing the fix will 
take some time.

Is it possible for you to upgrade to latest version of PySVN and check ? 

Original comment by nitinbh...@gmail.com on 29 Sep 2009 at 7:08

[GoogleCodeExporter]

Hello Nitin,
I tried to update PySVN: I used v1.7.1 under windows as noted in my first 
message. I
was unable to find any newer version than that. Unfortunately my problem 
persists.

I just installed Ubuntu 9.04 in a VM, installed all the dependencies via 
apt-get and
tried it again from there. Ubuntu 9.04 comes with Python 2.6 & PsySVN 1.6.3. 
This
combination seems to make less problem but the problem appears here to. I was 
unable
to quickly update PsySVN on ubuntu, so I stopped my try there, too.

This is what I tried under Ubuntu: 
rm gdummyproject.db ; python
/usr/local/lib/python2.6/dist-packages/svnplot/svnlog2sqlite.py -l -v
http://svnplot-dummy.googlecode.com/svn/trunk gdummyproject.db

I understand that the main problem seems to be in the used dependency PsSVN?

Sorry for bothering you. Do you think I can assist someone to pin-down the 
problem or
should should I just wait and hope PsySVN will identify & fix this issue.

Original comment by B.Sch...@exxcellent.de on 30 Sep 2009 at 1:52

[GoogleCodeExporter]

Hi Ben,

can you enable the logging and send the log file ? It will help me find out 
where the
problem originates ?  log file will be in the same directory as 'sqlite 
database path'. 

I did my initial testing on Python 2.5. I will also check today on Python 2.6

>>>Do you think I can assist someone to pin-down the problem or
>>>should should I just wait and hope PsySVN will identify & fix this issue.
I will give it a try again. Let see whom contact after that. :-)

Original comment by nitinbh...@gmail.com on 30 Sep 2009 at 2:30

[GoogleCodeExporter]

Fixed in trunk (revision 235). Will get included in next release.

Original comment by nitinbh...@gmail.com on 12 Oct 2009 at 6:07

• Changed state: Fixed