christianspecht / scm-backup

Makes offline backups of your cloud hosted source code repositories
https://scm-backup.org/
GNU General Public License v3.0
61 stars 20 forks source link

Unhandled exception. System.FormatException: Input string was not in a correct format #45

Closed IT-Yogi closed 4 years ago

IT-Yogi commented 4 years ago

Hi, I'm getting this unhandled exception on my windows system. Authentication and Repository seems fine.

D:\BackupAtlassian\Bitbucket\SCM-Backup>ScmBackup.exe SCM Backup 1.3.77925e4 https://scm-backup.org/ Backing up config files Getting repositories for source 'XXXXXXXXXX-Bitbucket' from hoster 'bitbucket' Found 1 Mercurial repos. Bitbucket will delete all Mercurial repos on June 1, 2020!! Trying to find SCMs on this machine... Found Mercurial 5.3+3-847e582f3cc9 Found Git 2.26.0.windows.1 Backing up source: XXXXXXXXXX-Bitbucket Git: https://bitbucket.org/XXXXXXXXXXX/XXXXXXXXXXX_internetseite.git Unhandled exception. System.FormatException: Input string was not in a correct format. at System.Text.StringBuilder.FormatError() at System.Text.StringBuilder.AppendFormatHelper(IFormatProvider provider, String format, ParamsArray args) at System.IO.StreamWriter.WriteFormatHelper(String format, ParamsArray args, Boolean appendNewLine) at System.IO.StreamWriter.WriteLine(String format, Object[] arg) at System.IO.TextWriter.SyncTextWriter.WriteLine(String format, Object[] arg) at System.Console.WriteLine(String format, Object[] arg) at ScmBackup.Loggers.ConsoleLogger.Log(ErrorLevel level, Exception ex, String message, Object[] arg) in C:\projects\scm-backup\src\ScmBackup\Loggers\ConsoleLogger.cs:line 35 at ScmBackup.Loggers.CompositeLogger.Log(ErrorLevel level, Exception ex, String message, Object[] arg) in C:\projects\scm-backup\src\ScmBackup\Loggers\CompositeLogger.cs:line 25 at ScmBackup.Loggers.CompositeLogger.Log(ErrorLevel level, String message, Object[] arg) in C:\projects\scm-backup\src\ScmBackup\Loggers\CompositeLogger.cs:line 21 at ScmBackup.ErrorHandlingScmBackup.Run() in C:\projects\scm-backup\src\ScmBackup\ErrorHandlingScmBackup.cs:line 43 at ScmBackup.LogMailingScmBackup.Run() in C:\projects\scm-backup\src\ScmBackup\LogMailingScmBackup.cs:line 28 at ScmBackup.Program.Main(String[] args) in C:\projects\scm-backup\src\ScmBackup\Program.cs:line 11

If I remove one of our productive git repositorys, scm is able to backup the git-testing-repo and a mercurial repo.

2020-03-26 12:57:00.7536|INFO|ScmBackup|Backing up source: XXXXXXXXXX-Bitbucket 2020-03-26 12:57:00.7536|INFO|ScmBackup| Git: https://bitbucket.org/XXXXXXXXX/git-testing-repo.git 2020-03-26 12:57:01.6761|INFO|ScmBackup| Mercurial: https://bitbucket.org/XXXXXXXXXXXXXXXXXXX/XXXXXXXXXXXXXXX_internetseite 2020-03-26 12:57:06.7650|INFO|ScmBackup|Backup finished! 2020-03-26 12:57:06.7650|INFO|ScmBackup|The backups are in this directory: D:\BackupAtlassian\Bitbucket\Downstream

christianspecht commented 4 years ago

So the culprit seems to be the repo https://bitbucket.org/XXXXXXXXXXX/XXXXXXXXXXX_internetseite.git ?

Do the actual Bitbucket username or repo name by any chance have any "special" characters?

IT-Yogi commented 4 years ago

Hey christian, Thanks for the fast reply.

No special characters at all, there are just two underlines "_" at the repo name like: https://bitbucket.org/teamname/customername_web_internetseite.git

Also tried one repo without any special characters, same error:

Backing up source: XXXXXX Git: https://bitbucket.org/teamname/genericsdo.git Unhandled exception. System.FormatException: Input string was not in a correct format.

Also the bitbucket username contains a "-".

christianspecht commented 4 years ago

I'm still not sure where this exception is thrown and why.

I just created a repository in the test team that I'm using for SCM Backup's integration tests: https://bitbucket.org/scm-backup-testteam/customername_web_internetseite

On my machine, the backup of this runs without issues.
Can you try backing this up on your machine, by replacing the source in the config file by this one?


sources:

   - title: issue45test
     hoster: bitbucket
     type: org
     name: scm-backup-testteam

If this fails with the same error, maybe it's something on your machine
--> in this case, could you try it on another machine?


If this works on your machine (but the real one still fails), it's probably about something that specific repository.
In this case, more information would help:

  1. Does the backup of this repository exist in the backup directory?
    (-> Restoring your backups -> Folder structure)
    The repository URL in the console output is shown before the repository is actually backed up. So the FormatException maybe happens when building the Git commands via string.Format, and then the actual backup never happened.

  2. You wrote:

    If I remove one of our productive git repositorys, scm is able to backup the git-testing-repo and a mercurial repo.

    What's the name of the repository you removed here?
    Reading your issue again, this looks like the culprit (and not customername_web_internetseite, because that one was backed up when you removed the other one)

  3. I understand that you don't want to disclose your actual team name and repository names to the whole internet, but knowing the real names would help me find the error.
    If you're okay with giving me the names, you can send me via email.
    Or even better, send me the whole log file of an application run where the error occurred.

IT-Yogi commented 4 years ago

Hey Christian, Many thanks for your support. I was able to find the issue. I tried to set up the backup user as described at your manual. But I used the users email address instead of the username. Mea Culpa! So the authentication with the app password failed and I tried it with the regular user password. With this I was able to connect and your tool was downloading the mercurial repos. So it looks to me that the authentication is fine.

Today I studied your manual once again and noticed my mistake. I inserted username and app password and your tool is now working like a charm. Sorry for the trouble.

christianspecht commented 4 years ago

Glad to hear it worked for you!

I'd still like to reproduce your error though, so I can catch it in the code and show a meaningful error message instead of that exception.
Can you give me an example how your config file looked like when you got the error?

So far, I wasn't able to reproduce your error.

I tried this configuration (a user/team I'm using for integration tests):

sources:

  - title: bitbucket-testorg
    hoster: bitbucket
    type: org
    name: scm-backup-testteam
    authName: scm-backup-testuser
    password: not-the-real-password

...and replaced the authName by the actual email from scm-backup-testuser's Bitbucket settings:

sources:

  - title: bitbucket-testorg
    hoster: bitbucket
    type: org
    name: scm-backup-testteam
    authName: not-the-real-email@example.com
    password: not-the-real-password

...and got a different error message (but no exception like you), way earlier, when trying to authenticate with Bitbucket's API:

SCM Backup 0.0.0-DEV https://scm-backup.org/ Backing up config files Getting repositories for source 'bitbucket-testorg' from hoster 'bitbucket' Authentication failed for not-the-real-email@example.com Backup failed! The application will close in 5 seconds!

So I'm wondering what you did different in your configuration.

IT-Yogi commented 4 years ago

Hey Christian, I've done some further investigations. I it is not related to the username as I thought, rather than the password I used. It contains the following characters.

} + , #

I changed it to letters and numbers only and tried it again with the users email address, now I get the error:

URL using bad/illegal format or missing URL

Which seems correct to me, because we need the username. At least, I replaced the email address with the username and the users password. All fine, backup is running.

Small enhancement suggestion: It would be great if you could implement a "Success" or "Failed" Status in the subject of the sent email. So you can see at a glance if your backup was successful or not.

christianspecht commented 4 years ago

I it is not related to the username as I thought, rather than the password I used. It contains the following characters.

} + , #

Hm. Deserializing a password containing all of these characters works on my machine.
(even though I had expected # to cause problems. because comments start with # in YAML)

Still, apparently YAML values should be quoted if they contain certain characters. including some of "your" characters.

So I'll add a notice about quotes to the documentation.