Updated to connect to Danbooru again

[thisismycontributionaccount]

DumpCommand: Changed Download to HTTP client with a user agent because Danbooru requires a user agent. Clarified MD5 not matching error message in the exception.

DanbooruUtility: Moved encoding of username and apikey out of url creation for clarity Added a user agent to the request because Danbooru requires it now.

SQLiteUtility: Added a missing field "media_asset". It is pretty useless but I am not familiar enough with dotnet to know how to remove it from the Post oblect. Possible future enhancement remove fields from post and only keep and insert the fields you really want.

[snaeker58]

Great fix, I'm unable to compile it though :( I have 15 errors:

Severity    Code    Description Project File    Line    Suppression State
Error   CS0234  The type or namespace name 'Data' does not exist in the namespace 'Microsoft' (are you missing an assembly reference?)  DanbooruDownloader  C:\Users\John\Downloads\DanbooruDownloader\DanbooruDownloader\Commands\DumpCommand.cs   2   Active
Error   CS0246  The type or namespace name 'Newtonsoft' could not be found (are you missing a using directive or an assembly reference?)    DanbooruDownloader  C:\Users\John\Downloads\DanbooruDownloader\DanbooruDownloader\Commands\DumpCommand.cs   3   Active
Error   CS0246  The type or namespace name 'NLog' could not be found (are you missing a using directive or an assembly reference?)  DanbooruDownloader  C:\Users\John\Downloads\DanbooruDownloader\DanbooruDownloader\Commands\DumpCommand.cs   4   Active
Error   CS0246  The type or namespace name 'Logger' could not be found (are you missing a using directive or an assembly reference?)    DanbooruDownloader  C:\Users\John\Downloads\DanbooruDownloader\DanbooruDownloader\Commands\DumpCommand.cs   19  Active
Error   CS0246  The type or namespace name 'JObject' could not be found (are you missing a using directive or an assembly reference?)   DanbooruDownloader  C:\Users\John\Downloads\DanbooruDownloader\DanbooruDownloader\Commands\DumpCommand.cs   268 Active
Error   CS0246  The type or namespace name 'JObject' could not be found (are you missing a using directive or an assembly reference?)   DanbooruDownloader  C:\Users\John\Downloads\DanbooruDownloader\DanbooruDownloader\Commands\DumpCommand.cs   371 Active
Error   CS0234  The type or namespace name 'Extensions' does not exist in the namespace 'Microsoft' (are you missing an assembly reference?)    DanbooruDownloader  C:\Users\John\Downloads\DanbooruDownloader\DanbooruDownloader\Program.cs    2   Active
Error   CS0234  The type or namespace name 'Extensions' does not exist in the namespace 'Microsoft' (are you missing an assembly reference?)    DanbooruDownloader  C:\Users\John\Downloads\DanbooruDownloader\DanbooruDownloader\Program.cs    3   Active
Error   CS0246  The type or namespace name 'Newtonsoft' could not be found (are you missing a using directive or an assembly reference?)    DanbooruDownloader  C:\Users\John\Downloads\DanbooruDownloader\DanbooruDownloader\Utilities\DanbooruUtility.cs  1   Active
Error   CS0246  The type or namespace name 'JObject' could not be found (are you missing a using directive or an assembly reference?)   DanbooruDownloader  C:\Users\John\Downloads\DanbooruDownloader\DanbooruDownloader\Utilities\DanbooruUtility.cs  24  Active
Error   CS0234  The type or namespace name 'Data' does not exist in the namespace 'Microsoft' (are you missing an assembly reference?)  DanbooruDownloader  C:\Users\John\Downloads\DanbooruDownloader\DanbooruDownloader\Utilities\SQLiteUtility.cs    1   Active
Error   CS0246  The type or namespace name 'Newtonsoft' could not be found (are you missing a using directive or an assembly reference?)    DanbooruDownloader  C:\Users\John\Downloads\DanbooruDownloader\DanbooruDownloader\Utilities\SQLiteUtility.cs    2   Active
Error   CS0246  The type or namespace name 'SqliteConnection' could not be found (are you missing a using directive or an assembly reference?)  DanbooruDownloader  C:\Users\John\Downloads\DanbooruDownloader\DanbooruDownloader\Utilities\SQLiteUtility.cs    12  Active
Error   CS0246  The type or namespace name 'SqliteConnection' could not be found (are you missing a using directive or an assembly reference?)  DanbooruDownloader  C:\Users\John\Downloads\DanbooruDownloader\DanbooruDownloader\Utilities\SQLiteUtility.cs    84  Active
Error   CS0246  The type or namespace name 'JObject' could not be found (are you missing a using directive or an assembly reference?)   DanbooruDownloader  C:\Users\John\Downloads\DanbooruDownloader\DanbooruDownloader\Utilities\SQLiteUtility.cs    84  Active

[thisismycontributionaccount]

Hello, Let me check I have everything attached. I did not change any of the namespaces. It might take me a day or two as I am away from home. Steve On Friday, June 16, 2023 at 01:38:53 PM CDT, snaeker58 @.***> wrote:

Great fix, I'm unable to compile it though :( I have 15 errors: Severity Code Description Project File Line Suppression State Error CS0234 The type or namespace name 'Data' does not exist in the namespace 'Microsoft' (are you missing an assembly reference?) DanbooruDownloader C:\Users\John\Downloads\DanbooruDownloader\DanbooruDownloader\Commands\DumpCommand.cs 2 Active Error CS0246 The type or namespace name 'Newtonsoft' could not be found (are you missing a using directive or an assembly reference?) DanbooruDownloader C:\Users\John\Downloads\DanbooruDownloader\DanbooruDownloader\Commands\DumpCommand.cs 3 Active Error CS0246 The type or namespace name 'NLog' could not be found (are you missing a using directive or an assembly reference?) DanbooruDownloader C:\Users\John\Downloads\DanbooruDownloader\DanbooruDownloader\Commands\DumpCommand.cs 4 Active Error CS0246 The type or namespace name 'Logger' could not be found (are you missing a using directive or an assembly reference?) DanbooruDownloader C:\Users\John\Downloads\DanbooruDownloader\DanbooruDownloader\Commands\DumpCommand.cs 19 Active Error CS0246 The type or namespace name 'JObject' could not be found (are you missing a using directive or an assembly reference?) DanbooruDownloader C:\Users\John\Downloads\DanbooruDownloader\DanbooruDownloader\Commands\DumpCommand.cs 268 Active Error CS0246 The type or namespace name 'JObject' could not be found (are you missing a using directive or an assembly reference?) DanbooruDownloader C:\Users\John\Downloads\DanbooruDownloader\DanbooruDownloader\Commands\DumpCommand.cs 371 Active Error CS0234 The type or namespace name 'Extensions' does not exist in the namespace 'Microsoft' (are you missing an assembly reference?) DanbooruDownloader C:\Users\John\Downloads\DanbooruDownloader\DanbooruDownloader\Program.cs 2 Active Error CS0234 The type or namespace name 'Extensions' does not exist in the namespace 'Microsoft' (are you missing an assembly reference?) DanbooruDownloader C:\Users\John\Downloads\DanbooruDownloader\DanbooruDownloader\Program.cs 3 Active Error CS0246 The type or namespace name 'Newtonsoft' could not be found (are you missing a using directive or an assembly reference?) DanbooruDownloader C:\Users\John\Downloads\DanbooruDownloader\DanbooruDownloader\Utilities\DanbooruUtility.cs 1 Active Error CS0246 The type or namespace name 'JObject' could not be found (are you missing a using directive or an assembly reference?) DanbooruDownloader C:\Users\John\Downloads\DanbooruDownloader\DanbooruDownloader\Utilities\DanbooruUtility.cs 24 Active Error CS0234 The type or namespace name 'Data' does not exist in the namespace 'Microsoft' (are you missing an assembly reference?) DanbooruDownloader C:\Users\John\Downloads\DanbooruDownloader\DanbooruDownloader\Utilities\SQLiteUtility.cs 1 Active Error CS0246 The type or namespace name 'Newtonsoft' could not be found (are you missing a using directive or an assembly reference?) DanbooruDownloader C:\Users\John\Downloads\DanbooruDownloader\DanbooruDownloader\Utilities\SQLiteUtility.cs 2 Active Error CS0246 The type or namespace name 'SqliteConnection' could not be found (are you missing a using directive or an assembly reference?) DanbooruDownloader C:\Users\John\Downloads\DanbooruDownloader\DanbooruDownloader\Utilities\SQLiteUtility.cs 12 Active Error CS0246 The type or namespace name 'SqliteConnection' could not be found (are you missing a using directive or an assembly reference?) DanbooruDownloader C:\Users\John\Downloads\DanbooruDownloader\DanbooruDownloader\Utilities\SQLiteUtility.cs 84 Active Error CS0246 The type or namespace name 'JObject' could not be found (are you missing a using directive or an assembly reference?) DanbooruDownloader C:\Users\John\Downloads\DanbooruDownloader\DanbooruDownloader\Utilities\SQLiteUtility.cs 84 Active

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

| | Virus-free.www.avast.com |

[snaeker58]

Hello, Let me check I have everything attached. I did not change any of the namespaces. It might take me a day or two as I am away from home. Steve On Friday, June 16, 2023 at 01:38:53 PM CDT, snaeker58 @.> wrote: Great fix, I'm unable to compile it though :( I have 15 errors: Severity Code Description Project File Line Suppression State Error CS0234 The type or namespace name 'Data' does not exist in the namespace 'Microsoft' (are you missing an assembly reference?) DanbooruDownloader C:\Users\John\Downloads\DanbooruDownloader\DanbooruDownloader\Commands\DumpCommand.cs 2 Active Error CS0246 The type or namespace name 'Newtonsoft' could not be found (are you missing a using directive or an assembly reference?) DanbooruDownloader C:\Users\John\Downloads\DanbooruDownloader\DanbooruDownloader\Commands\DumpCommand.cs 3 Active Error CS0246 The type or namespace name 'NLog' could not be found (are you missing a using directive or an assembly reference?) DanbooruDownloader C:\Users\John\Downloads\DanbooruDownloader\DanbooruDownloader\Commands\DumpCommand.cs 4 Active Error CS0246 The type or namespace name 'Logger' could not be found (are you missing a using directive or an assembly reference?) DanbooruDownloader C:\Users\John\Downloads\DanbooruDownloader\DanbooruDownloader\Commands\DumpCommand.cs 19 Active Error CS0246 The type or namespace name 'JObject' could not be found (are you missing a using directive or an assembly reference?) DanbooruDownloader C:\Users\John\Downloads\DanbooruDownloader\DanbooruDownloader\Commands\DumpCommand.cs 268 Active Error CS0246 The type or namespace name 'JObject' could not be found (are you missing a using directive or an assembly reference?) DanbooruDownloader C:\Users\John\Downloads\DanbooruDownloader\DanbooruDownloader\Commands\DumpCommand.cs 371 Active Error CS0234 The type or namespace name 'Extensions' does not exist in the namespace 'Microsoft' (are you missing an assembly reference?) DanbooruDownloader C:\Users\John\Downloads\DanbooruDownloader\DanbooruDownloader\Program.cs 2 Active Error CS0234 The type or namespace name 'Extensions' does not exist in the namespace 'Microsoft' (are you missing an assembly reference?) DanbooruDownloader C:\Users\John\Downloads\DanbooruDownloader\DanbooruDownloader\Program.cs 3 Active Error CS0246 The type or namespace name 'Newtonsoft' could not be found (are you missing a using directive or an assembly reference?) DanbooruDownloader C:\Users\John\Downloads\DanbooruDownloader\DanbooruDownloader\Utilities\DanbooruUtility.cs 1 Active Error CS0246 The type or namespace name 'JObject' could not be found (are you missing a using directive or an assembly reference?) DanbooruDownloader C:\Users\John\Downloads\DanbooruDownloader\DanbooruDownloader\Utilities\DanbooruUtility.cs 24 Active Error CS0234 The type or namespace name 'Data' does not exist in the namespace 'Microsoft' (are you missing an assembly reference?) DanbooruDownloader C:\Users\John\Downloads\DanbooruDownloader\DanbooruDownloader\Utilities\SQLiteUtility.cs 1 Active Error CS0246 The type or namespace name 'Newtonsoft' could not be found (are you missing a using directive or an assembly reference?) DanbooruDownloader C:\Users\John\Downloads\DanbooruDownloader\DanbooruDownloader\Utilities\SQLiteUtility.cs 2 Active Error CS0246 The type or namespace name 'SqliteConnection' could not be found (are you missing a using directive or an assembly reference?) DanbooruDownloader C:\Users\John\Downloads\DanbooruDownloader\DanbooruDownloader\Utilities\SQLiteUtility.cs 12 Active Error CS0246 The type or namespace name 'SqliteConnection' could not be found (are you missing a using directive or an assembly reference?) DanbooruDownloader C:\Users\John\Downloads\DanbooruDownloader\DanbooruDownloader\Utilities\SQLiteUtility.cs 84 Active Error CS0246 The type or namespace name 'JObject' could not be found (are you missing a using directive or an assembly reference?) DanbooruDownloader C:\Users\John\Downloads\DanbooruDownloader\DanbooruDownloader\Utilities\SQLiteUtility.cs 84 Active — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.> | | Virus-free.www.avast.com |

Take as much time as you need you are my lord and savior when it comes to this repo!

[thisismycontributionaccount]

Are you using 5 or 6 to compile, I think I was using 6

[thisismycontributionaccount]

Also, try a clean install. Download the master an then just apply my changes. I did not change any includes or dependencies,.

[snaeker58]

I just installed the missing dependencies by hand. Still getting 403 Forbidden though :(

[thisismycontributionaccount]

So here is how I was using it. If you are getting a 403 error then is likely something messed up with your api key. Try generating a new one an using it. dotnet run dump MyDataset -s 6313423 -e 6314423 --username --api-key

[snaeker58]

So here is how I was using it. If you are getting a 403 error then is likely something messed up with your api key. Try generating a new one an using it. dotnet run dump MyDataset -s 6313423 -e 6314423 --username --api-key

Yep new API key actually solved it! But after downloading a few posts it dies again...

System.AggregateException: One or more errors occurred. (SQLite Error 1: 'table posts has no column named media_asset'.) ---> Microsoft.Data.Sqlite.SqliteException (0x80004005): SQLite Error 1: 'table posts has no column named media_asset'.   at Microsoft.Data.Sqlite.SqliteException.ThrowExceptionForRC(Int32 rc, sqlite3 db)
   at Microsoft.Data.Sqlite.SqliteCommand.PrepareAndEnumerateStatements(Stopwatch timer)+MoveNext()
   at Microsoft.Data.Sqlite.SqliteCommand.ExecuteReader(CommandBehavior behavior)
   at Microsoft.Data.Sqlite.SqliteCommand.ExecuteReader()
   at Microsoft.Data.Sqlite.SqliteCommand.ExecuteNonQuery()
   at DanbooruDownloader.Utilities.SQLiteUtility.InsertOrReplace(SqliteConnection connection, IEnumerable`1 posts) in C:\Users\John\Downloads\Danbooru\DanbooruDownloader\Utilities\SQLiteUtility.cs:line 116
   at DanbooruDownloader.Commands.DumpCommand.Run(String path, Int64 startId, Int64 endId, Boolean ignoreHashCheck, Boolean includeDeleted, String username, String apikey) in C:\Users\John\Downloads\Danbooru\DanbooruDownloader\Commands\DumpCommand.cs:line 249
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
   at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
   at System.Threading.Tasks.Task.Wait()
   at DanbooruDownloader.Program.<>c__DisplayClass0_1.<Main>b__2() in C:\Users\John\Downloads\Danbooru\DanbooruDownloader\Program.cs:line 63
   at Microsoft.Extensions.CommandLineUtils.CommandLineApplication.Execute(String[] args)
   at DanbooruDownloader.Program.Main(String[] args) in C:\Users\John\Downloads\Danbooru\DanbooruDownloader\Program.cs:line 78

[thisismycontributionaccount]

Make sure you grabbed all three files. The fix for that is in SQLiteUtility.

[TimyIsCool]

Seems to work, is there anyway to speed up the download? its very slow

[thisismycontributionaccount]

Seems to work, is there anyway to speed up the download? its very slow

I have no idea on that one. It could probably be threaded or have parallel calls but that is far beyond my ability with this programming language. :-)