Open lilireli opened 10 years ago
andygrunwald
commented
10 years ago Hey @lilireli, can you offer your configuration (config file)? And is this subversion repository public available that we can reproduce it? And can you offer your command you executed?
lilireli
commented
10 years ago Hey,
I tried different ways. First I used the config file provided as exemple, changing the information I needed (as shown under). I was running the command: _cvsanaly2 http://my_url_
####### config file ###### debug = False quiet = False profile = False
db_driver = 'mysql' db_user = 'root' db_password = 'kartoffel' db_database = 'development' extensions = ['CommitsLOC'] ###########################
Then I tried to remove this file and rerun cvsanaly with the url still: _cvsanaly2 -u user -p my_password -d mydatabase --extensions=CommitsLOC http://url
Finally I tried to check out the project and run the following command in the right folder _cvsanaly2 -u user -p my_password -d mydatabase --extensions=CommitsLOC
Now, everything works fine without the extension, but when it comes to the CommitLOC extension either it creates an empty table, either it runs continuously, either it does both. I also tried with metrics, and I have the same.
Unfortunately, the repo I'm working on is not public and I cannot provide any access to it
jgbarah
commented
10 years ago Have you tried to run the same setup (same version of CVSAnalY, same environment, etc.) with some other (if possible, simple) Subversion repository? Just to discard that there is some trouble with the setup.
lilireli
commented
10 years ago Hi,
I tried with a public folder, the problem is the same. However I get the following error:
Parsing log for /tmp/extjs (svn)
Executing extensions
Executing extension CommitsLOC
Traceback (most recent call last):
File "/usr/local/bin/cvsanaly2", line 5, in
By printing the error I get that cvsanaly cannot run the command: svn diff -r 131:132 http://extjs-public.googlecode.com/svn
When I try to run it myself, it works. However I'm using a proxy to get access to internet and maybe the problem comes from this. Do you know if the proxy settings are still handled correctly when using cvsanaly ?
Thanks for your help
sduenas
commented
10 years ago I think this is not a bug in CVSAnalY. From my point of view it's a problem related to SVN servers. Usuarlly, SVN servers are slow and commands like svn diff are even more slower.
svn diff is a time consuming operation (I don't really know why) in which the connection between client and server must be kept alive for a long amount of time. For instance, to get the result from the command svn diff -r 1:2 http://extjs-public.googlecode.com/svn, I needed like 30 minutes for downloading around 20MB. And this is just the first diff.
Right now, I have CVSAnalY running using your example. It's up since 314 minutes ago (~5 hours) and now is executing the fifth diff of the list.
sduenas@Guybrush:~$ ps aux |grep svn
sduenas 6203 0.0 0.2 22628 9032 pts/2 S+ 00:18 0:04 svn diff -r 4:5 http://extjs-public.googlecode.com/svnSo, my impression is when you think CVSAnalY is doing nothing, it is really doing things but very slowly because of the problems with the SVN server. If an exception is raised it might be due to a connection error or similar problems.
I fixed the code to know exactly what it is happening when an exception is raised by CommitsLOC extension. I will keep you updated because I want to prove my supposition.
lilireli
commented
10 years ago I agree with you, the diff commands are really slow when computed by cvsanaly, I did the same and it was around one hour for one diff. However (and there I'm sorry not knowing more about how svn server works) when I run the same command by myself I can retrieve the result quite quickly (less than 1 minute). Is it normal ? Can't I do anything to speed a little bit up cvsanaly ?
sduenas
commented
10 years ago It's odd. I've been checking it again and it took the same time running the command in a terminal than by CVSAnalY. Maybe, when you run the command, the data was cached in memory but I'm not sure what SVN servers do on these cases. Anyway, I'm going to have a look to the code and try to find a way to speed up the process.
rodrigokuroda
commented
10 years ago I'm having the same problem. I also noticed that the "svn diff" is a slow process. Anyway, in this case, the parameter "--metrics-noerr" shouldn't work?
The output log of the respective command is as follow:
$ cvsanaly2 --debug --writable-path=./Camel --db-user=USER --db-password=PASS --db-database=DB --metrics-all --metrics-noerr --extensions=CommitsLOC,CommitsLOCDet https://svn.apache.org/repos/asf/camel
DBG: INSERT INTO repositories (id, uri, name, type) values (%s, %s, %s, %s)
Parsing log for https://svn.apache.org/repos/asf/camel (svn)
['svn', '-v', 'log', 'https://svn.apache.org/repos/asf/camel']
[... omitted a lot of logs ...]
[... omitted aprox. ~30 "svn diff" logs (as shown bellow) ...]
['svn', 'diff', '-r', '686458:686459', 'https://svn.apache.org/repos/asf']
Traceback (most recent call last):
File "/usr/local/bin/cvsanaly2", line 5, in <module>
pkg_resources.run_script('cvsanaly2==2.1.0', 'cvsanaly2')
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 528, in run_script
self.require(requires)[0].run_script(script_name, ns)
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 1394, in run_script
execfile(script_filename, namespace, namespace)
File "/usr/local/lib/python2.7/dist-packages/cvsanaly2-2.1.0-py2.7.egg/EGG-INFO/scripts/cvsanaly2", line 37, in <module>
retval = pycvsanaly2.main.main (sys.argv[1:])
File "/usr/local/lib/python2.7/dist-packages/cvsanaly2-2.1.0-py2.7.egg/pycvsanaly2/main.py", line 383, in main
emg.run_extensions(repo, path or uri, db)
File "/usr/local/lib/python2.7/dist-packages/cvsanaly2-2.1.0-py2.7.egg/pycvsanaly2/ExtensionsManager.py", line 105, in run_extensions
self.run_extension(name, extension, repo, uri, db)
File "/usr/local/lib/python2.7/dist-packages/cvsanaly2-2.1.0-py2.7.egg/pycvsanaly2/ExtensionsManager.py", line 66, in run_extension
extension.run(repo, uri, db)
File "/usr/local/lib/python2.7/dist-packages/cvsanaly2-2.1.0-py2.7.egg/pycvsanaly2/extensions/CommitsLOC.py", line 305, in run
(added, removed) = counter.get_lines_for_revision(revision)
File "/usr/local/lib/python2.7/dist-packages/cvsanaly2-2.1.0-py2.7.egg/pycvsanaly2/extensions/CommitsLOC.py", line 115, in get_lines_for_revision
printerr("Error running svn diff command: %s", (str(e)))
File "/usr/local/lib/python2.7/dist-packages/cvsanaly2-2.1.0-py2.7.egg/pycvsanaly2/utils.py", line 111, in printerr
str = str % tuple(to_utf8(arg) for arg in args)
TypeError: not all arguments converted during string formatting
Hello,
I'm using cvsanaly to parse the logs of one subversion repository, and I tried to run differents extensions (CommitLOC and Metric), they both run at infinite, with no error message. I tried with subset of my repo and they run continuously whatever the size of the repo.