wimrijnders commented 11 years ago

I have a situation on my development machine, where there are many git repositories arranged in various locations in the file system. I would like to be able to specify the repositories I would like to see and would really prefer to use a single instance of gitlist to do this.

I hereby propose to allow multiple values for parameter repositories in config.ini. The syntax can follow the current syntax of php ini files:

repositories[]=/path/to/repo1
repositories[]=/path/to/another/repo
repositories[]=/path/to/yet/another/interesting/repo

The current single value assignment can still be supported, for simple cases and backwards compatibility.

The amount of extra code to make this work is zero. The syntax is already supported by PHP.

What does need to change, is the way that the paths are managed internally. In the new situation, the values will be stored as an array in $app['git.repos']. The most important calls in the application using this value are:

$repository = $app['git']->getRepository($app['git.repos'] . $repo);

$this->app['git']->getRepositories($this->app['git.repos'])

The second call is easy to adjust for arrays. The first call needs to change; however, the change can be minimal:

$repository = $app['git']->getRepository($app['git.repos'],  $repo);

(dot replaced by comma)

Special handling for array can then be hidden in method getRepository. A scalar string value will also be accepted, this is a matter of testing is the parameter is an array and running the current code if it is not.

Routing.php does some internal handling as well of $app['git.repos']. This should be not too hard to fix.

I appreciate feedback on this idea. I am entirely willing to make the changes myself. Adding this feature to gitlist would make it extremely more useful for me, and I presume also for others.

Regards,

Wim.

wimrijnders commented 11 years ago

Klaus,

I'm wondering when you're going to merge the multidir branch to main. I see several issues coming along, most related to the specification of multiple directories: #279, #277, #270. All these issues would be solved by merging the multidir solution.

Is there a reason why this issue has not been closed yet? Perhaps there is something that still needs to be done? Please let me know if I can help.

klaussilveira commented 11 years ago

Just an absurd lack of time to properly test the features. I've been trying to reserve an evening to deal with the various features mixed for the 0.4 release, no luck so far.

Is the multidir branch stable and safe to merge? I'll just throw it in master and trust fellow Gitlist developers to help me test it out.

wimrijnders commented 11 years ago

Yes, it's stable. However, a lot has changed in the meantime in the master branch and the merge may not be trivial.

As promised, I will look at the merge in due time. I'll merge master to the multidir branch first, to keep the master safe.

Regards,

Wim.

On Tue, Mar 12, 2013 at 1:21 PM, Klaus Silveira notifications@github.comwrote:

Just an absurd lack of time to properly test the features. I've been trying to reserve an evening to deal with the various features mixed for the 0.4 release, no luck so far.

Is the multidir branch stable and safe to merge? I'll just throw it in master and trust fellow Gitlist developers to help me test it out.

— Reply to this email directly or view it on GitHubhttps://github.com/klaussilveira/gitlist/issues/215#issuecomment-14772025 .

Met vriendelijke groet,

Wim Rijnders http://axizo.nl

tobya commented 11 years ago

I finally got around to testing on Windows. I pulled latest build of gitlist from main then pulled gitter and gitlist from Wimrijnders.

Works good up until I tried to view a file in a sub dir.

I've logged an issue, I'll keep trying to sort it but any help would be appreciated.

https://github.com/klaussilveira/gitlist/issues/290

tobya commented 11 years ago

I need a bit of help from someone on this

When trying to view src/about.dfm

Whoops, looks like something went wrong.

1/1RuntimeException: fatal: Path 'about.dfm' does not exist in 'master'
in D:\Development\Websites\tests\multi\gitlist_multi\vendor\klaussilveira\gitter\lib\Gitter\Client.php line 278
at Client->run(object(Repository), 'show master:"about.dfm"') in D:\Development\Websites\tests\multi\gitlist_multi\vendor\klaussilveira\gitter\lib\Gitter\Model\Blob.php line 31
at Blob->output() in D:\Development\Websites\tests\multi\gitlist_multi\src\GitList\Controller\BlobController.php line 42
at GitList\Controller\{closure}('tfmRegSetup2', 'master/src', 'about.dfm')
at call_user_func_array(object(Closure), array('tfmRegSetup2', 'master/src', 'about.dfm')) in D:\Development\Websites\tests\multi\gitlist_multi\vendor\symfony\http-kernel\Symfony\Component\HttpKernel\HttpKernel.php line 129
at HttpKernel->handleRaw(object(Request), '1') in D:\Development\Websites\tests\multi\gitlist_multi\vendor\symfony\http-kernel\Symfony\Component\HttpKernel\HttpKernel.php line 73
at HttpKernel->handle(object(Request), '1', true) in D:\Development\Websites\tests\multi\gitlist_multi\vendor\silex\silex\src\Silex\Application.php line 504
at Application->handle(object(Request)) in D:\Development\Websites\tests\multi\gitlist_multi\vendor\silex\silex\src\Silex\Application.php line 481
at Application->run() in D:\Development\Websites\tests\multi\gitlist_multi\index.php line 24

The problem seems to be that the Branch variable sent to the `extractRef`` function in Repository.php includes all the directories which are then stripped.

public function extractRef($repository, $branch='', $tree='')

I can fix it in `extractRef`` but this code hasn't changed so I don't think this is where it has broken, but I can't quite see where.

The directories should be part of the $tree variable rather than the $branch variable.

Can anyone give me some direction as to where to look? I assume symphony hasn't changed?

wimrijnders commented 11 years ago

@tobya,

Apologies for not noticing your post sooner.

What you are describing appears to be issue #250, which has been fixed in the code. I just now finished merging the master branch into the multidir branch, so this fix is now also in the multidir branch.

Would you mind checking if it works now?

wimrijnders commented 11 years ago

Hi @klaussilveira,

I have put in the time to merge branch multidir with master. That is, the master changes have been merged into the multidir branch.

The repos 'gitlist' and 'gitter' have both been updated. All unit tests pass, but Travis is still complaining. I'm not sure what to do about this.

Now would be a good time to merge the multidir into the master :-). The multidir branch code is stable.

What does the user notice

The very first time gitlist is loaded, the repository directories are scanned and the resulting info is put in a file in the cache. Depending on the number of directories selected in the config file, and the number of subdirectories in these directories, this can take a very long time.

However, after this initial scan, gitlist should load very quickly, irrespective of the number of repositories found (this is, of course, the whole point of caching).

The scan can be redone by clicking on the 'Refresh' option top-right. This is an addition to the UI (the only one) for multidir. All it does is delete the cached multidir file and reload the page. Just as the initial scan, this can take a long time depending on the number and depth of configured directories. NOTE: I was just thinking, perhaps the Refresh option should not be in the interface. If you're hosting multiple repo's, the admin may want to have control over the refresh. What do you think?

Other solved issues

There are other Issues which will be fixed with this issue. After the merge to master, please test if the following issues have been resolved: #297, #290, #270, #249, #213.

Further comments on issues:

I'm not sure sure about #286, it is related to this issue, but it appears to be a statement about the results of Travis.

274 should work as well, submodules should be displayed as separate repo's in the gitlist main page. Please test to confirm.

260 might be okay too, but the initial scan is going to take VERY long. In addition, all mentioned 20k of repositories will be put in the main page of gitlist, Not good I think.

250 appears to work fine; perhaps you can close it now. Due to the merge it is now also present in multidir.

klaussilveira commented 11 years ago

I don't see a problem with the refresh being public. I really didn't wanted to create any kind of ACL, user management or even a control panel for Gitlist. KISS is taken to the letter, in order to make it quick and easy to setup/maintain.

I will merge into master and ask everyone actively involved with Gitlist to help us stress-test this feature. I'm calling out nice guys like @NateEag, @GromNaN, @attiks, @skwashd, @fabpot... :+1:

jmauntel commented 11 years ago

Has this merge been performed? I'm chomping at the bit for this feature...

wimrijnders commented 11 years ago

@jmauntel,

Yes, this has been merged to the master branch, but I don't think it has been officially released yet as a stable or development install version. You could get the code straight from the git repository to try it out.

jmauntel commented 11 years ago

@wimrijnders,

Looks like a bust. The git repo version is missing the vendor directory.

[Wed May 01 21:26:43 2013] [error] [client 192.168.241.16] PHP Warning: require(vendor/autoload.php): failed to open stream: No such file or directory in /var/www/html/gitlist-dev/index.php on line 17 [Wed May 01 21:26:43 2013] [error] [client 192.168.241.16] PHP Fatal error: require(): Failed opening required 'vendor/autoload.php' (include_path='.:/usr/share/pear:/usr/share/php') in /var/www/html/gitlist-d ev/index.php on line 17

wimrijnders commented 11 years ago

@jmauntel,

You need to run Composer to install the dependencies. Check out section 'Building' on: https://github.com/klaussilveira/gitlist.

I sort of realize now that saying 'use the master branch from the repository' is a lot to ask for from a non-developer. Sorry about that. But maybe you can persevere.

jmauntel commented 11 years ago

Any idea when this will get merged?

On May 1, 2013, at 21:35, "wimrijnders" notifications@github.com<mailto:notifications@github.com> wrote:

@jmauntelhttps://github.com/jmauntel,

You need to run Composer to install the dependencies. Check out section 'Building' on: https://github.com/klaussilveira/gitlist.

I sort of realize now that saying 'use the master branch from the repository' is a lot to ask for from a non-developer. Sorry about that. But maybe you can persevere.

— Reply to this email directly or view it on GitHubhttps://github.com/klaussilveira/gitlist/issues/215#issuecomment-17320075.

wimrijnders commented 11 years ago

It should have already happened,

@klaussilveira, any idea when you can get around to releasing the master branch as the Development release?

klaussilveira commented 11 years ago

The CI server is, finally, finished. So, the master branch is constantly being built and published to the website. I'm sorry it took so long guys. :-1:

tobya commented 11 years ago

That's brilliant klaussilveira :+1:

jdunmire commented 11 years ago

I am trying to get the multiple repository feature to work using the Development (master) build from gitlist.org and I am getting this error:

PHP Fatal error: Uncaught exception 'RuntimeException' with message 'There are no GIT repositories in /data/wgit/repositories/' in /var/www/gitlist2/vendor/klaussilveira/gitter/lib/Gitter/Client.php:244

I get this for even a very simple, single repository configuration.

My config.ini file has these lines. [git] client = '/usr/bin/git' ; git executable path repositories = '/data/wgit/repositories/' default_branch = 'master' ; Default branch when HEAD is detached

/data/wgit/repositories/ can be listed by the www-data user and there are several bare repositories in the directory.

I used the same repositories setting for gitlist 0.3

Any suggestions for where I am going wrong?

jdunmire commented 11 years ago

Update: The problem seems to be that the fix for #304 is not in the development tarball I downloaded from gitlist.org.

The vendor/klaussilveira/gitter/lib/Gitter/Client.php file still has the line if (self::endsWith($path, ".git") || self::endsWith($path, "HEAD")) {

If I replace that with $base = basename($path); if ($base == ".git" || $base == "HEAD") {

Then repositories show up.

Still haven't figured out how to get multiple repositories to work.

Are the multiple repository changes supposed to be in the development tarball from gitlist.org?

klaussilveira commented 11 years ago

I might have screwed things up for everyone in 5122e91b49071915ff78823d55ff9f84f36aba88. But i promise, it's for the best. I was unhappy with the current implementation and started a major refactoring, mostly by isolating Gitter code and applying directory scanning logic to GitList only.

It would be amazing if you guys could test all the features you are actively using and create issues based on that. For multiple directory discussion, go here: https://github.com/klaussilveira/gitlist/issues/326

klaussilveira / gitlist

Proposal: Specifying multiple repository directories #215

What does the user notice

Other solved issues

274 should work as well, submodules should be displayed as separate repo's in the gitlist main page. Please test to confirm.

260 might be okay too, but the initial scan is going to take VERY long. In addition, all mentioned 20k of repositories will be put in the main page of gitlist, Not good I think.

250 appears to work fine; perhaps you can close it now. Due to the merge it is now also present in multidir.