search

go-gitea / gitea

Git with a cup of tea! Painless self-hosted all-in-one software development service, including Git hosting, code review, team collaboration, package registry and CI/CD
https://gitea.com
MIT License
44.57k stars 5.45k forks source link

Error 500 when viewing some of repo's subdirectories #6837

Closed achikhv closed 5 years ago

achikhv commented 5 years ago

Description

I get error 500 when I try to view contents of some directories inside my repo. One subdirs are opened successfully but another gives an error. I can not find any quick reason to it. All subdirs have simple names with only ASCII encoding, no special symbols or spaces.

In my gitea.log I see:

2019/05/03 09:01:08 [...routers/repo/view.go:55 renderDirectory()] [E] GetCommitsInfo: exit status 3221225725

For example I get 500 when I follow: http://sc.lers.ru/LersUchet/LersUchet/src/branch/master/Client But following http://sc.lers.ru/LersUchet/LersUchet/src/branch/master/Server works as expected.

lafriks commented 5 years ago

Can you reproduce this on try.gitea.io?

achikhv commented 5 years ago

Unfortunately no. I can't upload my repo as it is commercial software. This repo is quite big - over 500MB in size and has 35K commits over last 7 years , so it is hard to reproduce it.

Also I get 500 with same error code when I try to view some old commits by SHA-1. 

2019/05/06 14:17:22 [...uters/repo/commit.go:196 Diff()] [E] Repo.GitRepo.GetCommit: exit status 3221225725

Not a big deal for me anyway, but it's quite strange.

zeripath commented 5 years ago

Hmm... Two possible encoding problems come to mind:

Git internally just thinks of filenames as a byte array of non NULs and likely we're just enforcing UTF-8 on that because that's the most likely case. Similarly for the commit messages but AFAIU there's supposed to be an encoding marker for these - which I also suspect we're ignoring.

achikhv commented 5 years ago
  1. No, all source file names are in ASCII encoding.
  2. Right now all commit messages are encoded UTF-8 by default. Initially it was SVN repo, so I suppose that during migration "git svn" should have used UTF-8 by default too.

Anyway I can't be sure about it. In this comment I pointed that 500 occurs when viewing older commits, so maybe it is commit encoding that causes 500.

zeripath commented 5 years ago

Ok that's good to know. I think there are some Chinese users who have used big5 as their commit messages too and are also having problems.

https://www.git-tower.com/help/mac/faq-and-tips/faq/encoding

Might be of some help.

zeripath commented 5 years ago

In particular:

git log -1 --pretty='format:%h: "%B" (Encoding: "%e")' SHA

Which is just for testing one commit's stored encoding but should be extendable to look at all or a particular file.

Now iirc svn is actually quite good at dealing with encodings - so I expect that the encoding will be stored rather than just the data being dumped in the message, however it may not be. If it's the first case that should be relatively easy to fix - assuming go git handles this correctly. If it's the second case well that's more difficult but again should be solvable.

If you could find a commit which doesn't have the utf8 encoding that would be good to know.

zeripath commented 5 years ago

Oh would it be possible to check this is still a problem on master? We made substantial changes to our git infrastructure between 1.7 and 1.9 so it would be good to know if it's still a problem.

You certainly should move off 1.7.6 though.

achikhv commented 5 years ago

Hello!

I have already upgraded to the latest 1.8.1 and 500 is still there.

All commits in my repo are encoded UTF-8 , ie "Encoding: %e" shows empty string Encoding: As I understand it defaults to UTF-8.

zeripath commented 5 years ago

I was meaning master not 1.8.

The lack of encoding headers doesn't rule out misencoded commits, it just means that we're unlikely to be able to easily fix them without catching an error first. But it does make it less likely.

Are you able to find the commit that causes the 500? It sounds like you know a SHA that causes the problem. Is there anything at all you can tell me about that SHA that might help?

achikhv commented 5 years ago

Sorry, right now I can't try master, because it's preferable to stay on stable release.

Actually all commits that was made earlier than 04.07.2016 causes 500. I know last commit causing 500. There is nothing special about it, only couple of line changes in one file. No moves or renames.

What particular information about this commit would you like to know?

zeripath commented 5 years ago

Hmm. I'm trying to work out how to get the information without you revealing your internal data. I'm guessing that the commit message is not in English but likely in Russian - my suspicion is still that the message is not stored as utf8 even though it claims it is.

Basically I would like to check that the commit and the tree make sense.

git show --format=raw $SHA should show us the commit raw but what we want is to definitely prove that its message is in utf8 rather than Windows-1251 or iso-8859-5.

In some ways without being able to check whether this bug is still present on master means that we don't know if it's fixed already.

BNolet commented 5 years ago

Having a similar problem myself, using MySQL as the DB though. Running on v799f5e0.

Can't view here: https://gitea.bnolet.me/brandon/personal-blog/src/branch/master/themes/hello-friend-ng But can view here: https://gitea.bnolet.me/brandon/personal-blog/src/branch/master/themes

There's a README.md in the folder, yes, but I am able to view the file directly here: https://gitea.bnolet.me/brandon/personal-blog/src/branch/master/themes/hello-friend-ng/README.md

Haven't done much troubleshooting yet, but if I find anything interesting I'll post here. Feel free to download and mirror the source.

lunny commented 5 years ago

@BNolet I mirrored your repo to local and https://gitea.com/lunny/test-personal-blog/src/branch/master/themes/hello-friend-ng , but it's OK. Is your database setting as utf8mb4?

BNolet commented 5 years ago

@lunny Strange! I didn't actively change the encoding for the DB and looking at some threads, it doesn't seem that's the default either, so I'm inclined to say no

BNolet commented 5 years ago

The following is from the logs at the moment where I get a 500 error code:

2019/07/15 15:13:32 [...les/context/panic.go:36 1()] [E] PANIC:: runtime error: slice bounds out of range
/usr/local/go/src/runtime/panic.go:54 (0x4340da)
/go/src/code.gitea.io/gitea/vendor/code.gitea.io/git/commit_info.go:176 (0x90b6e0)
/go/src/code.gitea.io/gitea/vendor/code.gitea.io/git/commit_info.go:280 (0x90c101)
/go/src/code.gitea.io/gitea/vendor/code.gitea.io/git/commit_info.go:232 (0x90bc3c)
/go/src/code.gitea.io/gitea/vendor/code.gitea.io/git/commit_info.go:133 (0x90ae03)
/go/src/code.gitea.io/gitea/routers/repo/view.go:52 (0x1226a16)
/go/src/code.gitea.io/gitea/routers/repo/view.go:437 (0x122befd)
/go/src/code.gitea.io/gitea/routers/repo/view.go:377 (0x122b69e)
/usr/local/go/src/reflect/value.go:447 (0x4cb940)
/usr/local/go/src/reflect/value.go:308 (0x4cb3c3)
/go/src/code.gitea.io/gitea/vendor/github.com/go-macaron/inject/inject.go:177 (0x9a1266)
/go/src/code.gitea.io/gitea/vendor/github.com/go-macaron/inject/inject.go:137 (0x9a0b5b)
/go/src/code.gitea.io/gitea/vendor/gopkg.in/macaron.v1/context.go:121 (0x9cfd19)
/go/src/code.gitea.io/gitea/vendor/gopkg.in/macaron.v1/context.go:112 (0x11661a8)
/go/src/code.gitea.io/gitea/modules/context/panic.go:40 (0x116619b)
/usr/local/go/src/reflect/value.go:447 (0x4cb940)
/usr/local/go/src/reflect/value.go:308 (0x4cb3c3)
/go/src/code.gitea.io/gitea/vendor/github.com/go-macaron/inject/inject.go:177 (0x9a1266)
/go/src/code.gitea.io/gitea/vendor/github.com/go-macaron/inject/inject.go:137 (0x9a0b5b)
/go/src/code.gitea.io/gitea/vendor/gopkg.in/macaron.v1/context.go:121 (0x9cfd19)
/go/src/code.gitea.io/gitea/vendor/gopkg.in/macaron.v1/context.go:112 (0x9efc76)
/go/src/code.gitea.io/gitea/vendor/github.com/go-macaron/session/session.go:192 (0x9efc61)
/go/src/code.gitea.io/gitea/vendor/gopkg.in/macaron.v1/context.go:79 (0x9cfbc0)
/go/src/code.gitea.io/gitea/vendor/github.com/go-macaron/inject/inject.go:157 (0x9a0f20)
/go/src/code.gitea.io/gitea/vendor/github.com/go-macaron/inject/inject.go:135 (0x9a0c4a)
/go/src/code.gitea.io/gitea/vendor/gopkg.in/macaron.v1/context.go:121 (0x9cfd19)
/go/src/code.gitea.io/gitea/vendor/gopkg.in/macaron.v1/context.go:112 (0x9e177f)
/go/src/code.gitea.io/gitea/vendor/gopkg.in/macaron.v1/recovery.go:161 (0x9e176d)
/go/src/code.gitea.io/gitea/vendor/gopkg.in/macaron.v1/logger.go:40 (0x9d39b3)
/go/src/code.gitea.io/gitea/vendor/github.com/go-macaron/inject/inject.go:157 (0x9a0f20)
/go/src/code.gitea.io/gitea/vendor/github.com/go-macaron/inject/inject.go:135 (0x9a0c4a)
/go/src/code.gitea.io/gitea/vendor/gopkg.in/macaron.v1/context.go:121 (0x9cfd19)
/go/src/code.gitea.io/gitea/vendor/gopkg.in/macaron.v1/context.go:112 (0x9e0aa0)
/go/src/code.gitea.io/gitea/vendor/gopkg.in/macaron.v1/logger.go:52 (0x9e0a8b)
/go/src/code.gitea.io/gitea/vendor/gopkg.in/macaron.v1/logger.go:40 (0x9d39b3)
/go/src/code.gitea.io/gitea/vendor/github.com/go-macaron/inject/inject.go:157 (0x9a0f20)
/go/src/code.gitea.io/gitea/vendor/github.com/go-macaron/inject/inject.go:135 (0x9a0c4a)
/go/src/code.gitea.io/gitea/vendor/gopkg.in/macaron.v1/context.go:121 (0x9cfd19)
/go/src/code.gitea.io/gitea/vendor/gopkg.in/macaron.v1/router.go:187 (0x9e29c6)
/go/src/code.gitea.io/gitea/vendor/gopkg.in/macaron.v1/router.go:303 (0x9dc435)
/go/src/code.gitea.io/gitea/vendor/gopkg.in/macaron.v1/macaron.go:220 (0x9d4d8c)
/go/src/code.gitea.io/gitea/vendor/github.com/gorilla/context/context.go:141 (0xce32ca)
/usr/local/go/src/net/http/server.go:1995 (0x6f6213)
/usr/local/go/src/net/http/server.go:2774 (0x6f94e7)
/usr/local/go/src/net/http/server.go:1878 (0x6f51d0)
/usr/local/go/src/runtime/asm_amd64.s:1337 (0x464c60)
lunny commented 5 years ago

@BNolet It seems you have a version of v1.8.x or before.

BNolet commented 5 years ago

welp, upgrade fixed the issue. I was running 1.8.0 upgraded to 1.10.

Thank you @lunny. I'll try to remember to upgrade before posting here next time

achikhv commented 5 years ago

Yes, upgrade to 1.9.0 RC2 fixed issue. Thanks!