search

bodrick / Ember-MM

Ember Media Manager - The all in one media manager for Kodi
http://ember.purplepig.net/
121 stars 32 forks source link

Saved images are always re-compressed #36

Closed chrisallen closed 1 year ago

chrisallen commented 12 years ago

As seen here: http://ember.purplepig.net/issues/103

Every fanart / poster etc. which is scraped will always be recompressed. It's significantly smaller and with noticable artifacts.

I've located the problem and it's in "clsAPIImages.vb" (EmberAPI). The function which creates the new JPG is Public Sub Save(ByVal sPath As String, Optional ByVal iQuality As Long = 0). The quality of the cached images can be altered by iQuality but still always recompresses the image. It can be fixed by recreate the save routines. To scrape all fanarts / posters the function DownloadFile(downloadLink, savePath) can be used to download them and FileCopy(SourceFile, DestinationFile) to copy the selected files from cache to the movie folder. So nothing is recompressed.

msavazzi commented 11 years ago

good idea! But is not trivial, if you read the code the images are stored in memory not on files. When they are in memory they are "uncompressed" on download. The procedure should be changed to download & save images, store the filename, load image from file and then... if the image is selected copy the file and delete the others.

Quite a change.

Other option is to download twice. The first time to show them and then, when an image is selected instead of saving the one in memory it should be downloaded to disk.

I think the chosen option is the best from a performance perspective AND never leaves garbage around.

Additional point is that the current procedure does an implicit conversion from any possible format supported by the Image object (PNG, BMP, GIF, etc) to JPG

Have you tried to raise the quality?

chrisallen commented 11 years ago

I have tired raising the quality in the GUI but no matter what is selected, it is still compressing the JPG's and introducing artifacts. I suggest your second option, where by it downloads the image twice. once to preview and then again to save it to disk if selected.

Cocotus commented 11 years ago

Good ideas guys, I also don't like the fact that Ember recompresses before saving the file, so I added some few modification to download the image again when I click the ok Button and selected an image. I will test it a bit and tell you how it work. It's possible that it requires a bit modification. So far the modified code:

Public Class ScrapeImages, edit Save Method

Public Shared Sub Save(ByVal _image As Image, ByVal sPath As String, Optional ByVal iQuality As Long = 0, Optional ByVal sUrl As String = "") Try If IsNothing(_image) Then Exit Sub

        Dim doesExist As Boolean = File.Exists(sPath)
        Dim fAtt As New FileAttributes
        If Not String.IsNullOrEmpty(sPath) AndAlso (Not doesExist OrElse (Not CBool(File.GetAttributes(sPath) And FileAttributes.ReadOnly))) Then
            If doesExist Then
                'get the current attributes to set them back after writing
                fAtt = File.GetAttributes(sPath)
                'set attributes to none for writing
                File.SetAttributes(sPath, FileAttributes.Normal)

                'pm
                Try
                    If Not sUrl = "" Then
                        Dim stroriginalurl As String = sUrl

                        'Image Download from tmdb is special, need original size
                        If Not sUrl.Contains("impawards") AndAlso Not sUrl.Contains("movieposterdb") Then
                            'Always get original image...
                            'links to images (tmdb) have following structure:  'example: http://d3gtl9l2a4fn1j.cloudfront.net/t/p/w92/x65b4vsFKYuA878pLN1mJiAsgIP.jpg
                            Dim temp As String = sUrl
                            Dim stringArray() As String = Split(temp, "/")
                            If stringArray.Length > 4 Then
                                ' stringArray(5) contains values like "w185","original", "w154"...-->size -> we want original!
                                stroriginalurl = sUrl.Replace(stringArray(5), "original")
                            End If
                        End If
                        Dim webclient As New Net.WebClient
                        webclient.DownloadFile(stroriginalurl, sPath)
                        If doesExist Then File.SetAttributes(sPath, fAtt)
                        Exit Sub
                    End If
                Catch ex As Exception
                    Master.eLog.WriteToErrorLog(ex.Message, ex.StackTrace, "Error")
                End Try
                'pm

            End If

            Using msSave As New MemoryStream
                Dim retSave() As Byte
                Dim ICI As ImageCodecInfo = GetEncoderInfo(ImageFormat.Jpeg)
                Dim EncPars As EncoderParameters = New EncoderParameters(If(iQuality > 0, 2, 1))

                EncPars.Param(0) = New EncoderParameter(Encoder.RenderMethod, EncoderValue.RenderNonProgressive)

                If iQuality > 0 Then
                    EncPars.Param(1) = New EncoderParameter(Encoder.Quality, iQuality)
                End If

                _image.Save(msSave, ICI, EncPars)

                retSave = msSave.ToArray

                Using fs As New FileStream(sPath, FileMode.Create, FileAccess.Write)
                    fs.Write(retSave, 0, retSave.Length)
                    fs.Flush()
                End Using
                msSave.Flush()
            End Using

            If doesExist Then File.SetAttributes(sPath, fAtt)
        End If
    Catch ex As Exception
        Master.eLog.WriteToErrorLog(ex.Message, ex.StackTrace, "Error")
    End Try
End Sub

Public Class Images, edit Save Method

Public Sub Save(ByVal sPath As String, Optional ByVal iQuality As Long = 0, Optional ByVal sUrl As String = "") Try If IsNothing(_image) Then Exit Sub

        Dim doesExist As Boolean = File.Exists(sPath)
        Dim fAtt As New FileAttributes
        Dim fAttWritable As Boolean = True

        If Not String.IsNullOrEmpty(sPath) AndAlso (Not doesExist OrElse (Not CBool(File.GetAttributes(sPath) And FileAttributes.ReadOnly))) Then
            If doesExist Then
                'get the current attributes to set them back after writing
                fAtt = File.GetAttributes(sPath)
                'set attributes to none for writing
                Try
                    File.SetAttributes(sPath, FileAttributes.Normal)
                Catch ex As Exception
                    fAttWritable = False
                End Try
            End If

            'pm
            Try
                If Not sUrl = "" Then
                    'TODO V3 API implementation to get ALL posters! http://docs.themoviedb.apiary.io/#configuration
                    '  GetsImagesFromTMDBv3("URL/MOVIEDID")

                    Dim stroriginalurl As String = sUrl
                    'Image Download from tmdb is special, need original size
                    If Not sUrl.Contains("impawards") AndAlso Not sUrl.Contains("movieposterdb") Then
                        'Always get original image...
                        'links to images (tmdb) have following structure:  'example: http://d3gtl9l2a4fn1j.cloudfront.net/t/p/w92/x65b4vsFKYuA878pLN1mJiAsgIP.jpg
                        Dim temp As String = sUrl
                        Dim stringArray() As String = Split(temp, "/")
                        If stringArray.Length > 4 Then
                            ' stringArray(5) contains values like "w185","original", "w154"...-->size -> we want original!
                            stroriginalurl = sUrl.Replace(stringArray(5), "original")
                        End If
                    End If

                    Dim webclient As New Net.WebClient
                    'Download image!
                    webclient.DownloadFile(stroriginalurl, sPath)

                    If doesExist And fAttWritable Then File.SetAttributes(sPath, fAtt)
                    Exit Sub

                End If
            Catch ex As Exception
                Master.eLog.WriteToErrorLog(ex.Message, ex.StackTrace, "Error")
            End Try
            'pm

            Using msSave As New MemoryStream
                Dim retSave() As Byte
                Dim ICI As ImageCodecInfo = GetEncoderInfo(ImageFormat.Jpeg)
                Dim EncPars As EncoderParameters = New EncoderParameters(If(iQuality > 0, 2, 1))

                EncPars.Param(0) = New EncoderParameter(Encoder.RenderMethod, EncoderValue.RenderNonProgressive)

                If iQuality > 0 Then
                    EncPars.Param(1) = New EncoderParameter(Encoder.Quality, iQuality)
                End If

                _image.Save(msSave, ICI, EncPars)

                retSave = msSave.ToArray

                'make sure directory exists
                Directory.CreateDirectory(Directory.GetParent(sPath).FullName)
                If sPath.Length <= 260 Then
                    Using fs As New FileStream(sPath, FileMode.Create, FileAccess.Write)
                        fs.Write(retSave, 0, retSave.Length)
                        fs.Flush()
                    End Using
                End If
                msSave.Flush()
            End Using

            If doesExist And fAttWritable Then File.SetAttributes(sPath, fAtt)
        End If
    Catch ex As Exception
        Master.eLog.WriteToErrorLog(ex.Message, ex.StackTrace, "Error")
    End Try
End Sub

Public Class dlgImgSelect

Add in declaration: Private selURL As String = ""

edit Private Sub DoSelect, Add under Me.selIndex = iIndex: Me.selURL = poster.URL

edit Private Sub OK_Button_Click, edit Save 2xtimes in this Sub: Me.tmpImage.Save(tmpPathPlus, 100, selURL)

msavazzi commented 11 years ago

Wonderful

Continue to test this week (I will be traveling) and when I’m back I can incorporate the changes.

From: Cocotus [mailto:notifications@github.com] Sent: Sunday, 27 January, 2013 17:32 To: bodrick/Ember-MM Cc: msavazzi Subject: Re: [Ember-MM] Saved images are always re-compressed (#36)

Good ideas guys, I also doesn't like the fact that Ember recompresses before saving the file, so I added some few modification to download the image again when I click the ok Button and selected an image. I will test it a bit and tell you how it work. It's possible that it requires a bit modification. So far the modified code:

Public Class ScrapeImages, edit Save Method

Public Shared Sub Save(ByVal _image As Image, ByVal sPath As String, Optional ByVal iQuality As Long = 0, Optional ByVal sUrl As String = "") Try If IsNothing(_image) Then Exit Sub

    Dim doesExist As Boolean = File.Exists(sPath)
    Dim fAtt As New FileAttributes
    If Not String.IsNullOrEmpty(sPath) AndAlso (Not doesExist OrElse (Not CBool(File.GetAttributes(sPath) And FileAttributes.ReadOnly))) Then
        If doesExist Then
            'get the current attributes to set them back after writing
            fAtt = File.GetAttributes(sPath)
            'set attributes to none for writing
            File.SetAttributes(sPath, FileAttributes.Normal)

            'pm
            Try
                If Not sUrl = "" Then
                    Dim stroriginalurl As String = sUrl

                    'Image Download from tmdb is special, need original size
                    If Not sUrl.Contains("impawards") AndAlso Not sUrl.Contains("movieposterdb") Then
                        'Always get original image...
                        'links to images (tmdb) have following structure:  'example: http://d3gtl9l2a4fn1j.cloudfront.net/t/p/w92/x65b4vsFKYuA878pLN1mJiAsgIP.jpg
                        Dim temp As String = sUrl
                        Dim stringArray() As String = Split(temp, "/")
                        If stringArray.Length > 4 Then
                            ' stringArray(5) contains values like "w185","original", "w154"...-->size -> we want original!
                            stroriginalurl = sUrl.Replace(stringArray(5), "original")
                        End If
                    End If
                    Dim webclient As New Net.WebClient
                    webclient.DownloadFile(stroriginalurl, sPath)
                    If doesExist Then File.SetAttributes(sPath, fAtt)
                    Exit Sub
                End If
            Catch ex As Exception
                Master.eLog.WriteToErrorLog(ex.Message, ex.StackTrace, "Error")
            End Try
            'pm

        End If

        Using msSave As New MemoryStream
            Dim retSave() As Byte
            Dim ICI As ImageCodecInfo = GetEncoderInfo(ImageFormat.Jpeg)
            Dim EncPars As EncoderParameters = New EncoderParameters(If(iQuality > 0, 2, 1))

            EncPars.Param(0) = New EncoderParameter(Encoder.RenderMethod, EncoderValue.RenderNonProgressive)

            If iQuality > 0 Then
                EncPars.Param(1) = New EncoderParameter(Encoder.Quality, iQuality)
            End If

            _image.Save(msSave, ICI, EncPars)

            retSave = msSave.ToArray

            Using fs As New FileStream(sPath, FileMode.Create, FileAccess.Write)
                fs.Write(retSave, 0, retSave.Length)
                fs.Flush()
            End Using
            msSave.Flush()
        End Using

        If doesExist Then File.SetAttributes(sPath, fAtt)
    End If
Catch ex As Exception
    Master.eLog.WriteToErrorLog(ex.Message, ex.StackTrace, "Error")
End Try

End Sub

Public Class Images, edit Save Method

Public Sub Save(ByVal sPath As String, Optional ByVal iQuality As Long = 0, Optional ByVal sUrl As String = "") Try If IsNothing(_image) Then Exit Sub

    Dim doesExist As Boolean = File.Exists(sPath)
    Dim fAtt As New FileAttributes
    Dim fAttWritable As Boolean = True

    If Not String.IsNullOrEmpty(sPath) AndAlso (Not doesExist OrElse (Not CBool(File.GetAttributes(sPath) And FileAttributes.ReadOnly))) Then
        If doesExist Then
            'get the current attributes to set them back after writing
            fAtt = File.GetAttributes(sPath)
            'set attributes to none for writing
            Try
                File.SetAttributes(sPath, FileAttributes.Normal)
            Catch ex As Exception
                fAttWritable = False
            End Try
        End If

        'pm
        Try
            If Not sUrl = "" Then
                'TODO V3 API implementation to get ALL posters! http://docs.themoviedb.apiary.io/#configuration
                '  GetsImagesFromTMDBv3("URL/MOVIEDID")

                Dim stroriginalurl As String = sUrl
                'Image Download from tmdb is special, need original size
                If Not sUrl.Contains("impawards") AndAlso Not sUrl.Contains("movieposterdb") Then
                    'Always get original image...
                    'links to images (tmdb) have following structure:  'example: http://d3gtl9l2a4fn1j.cloudfront.net/t/p/w92/x65b4vsFKYuA878pLN1mJiAsgIP.jpg
                    Dim temp As String = sUrl
                    Dim stringArray() As String = Split(temp, "/")
                    If stringArray.Length > 4 Then
                        ' stringArray(5) contains values like "w185","original", "w154"...-->size -> we want original!
                        stroriginalurl = sUrl.Replace(stringArray(5), "original")
                    End If
                End If

                Dim webclient As New Net.WebClient
                'Download image!
                webclient.DownloadFile(stroriginalurl, sPath)

                If doesExist And fAttWritable Then File.SetAttributes(sPath, fAtt)
                Exit Sub

            End If
        Catch ex As Exception
            Master.eLog.WriteToErrorLog(ex.Message, ex.StackTrace, "Error")
        End Try
        'pm

        Using msSave As New MemoryStream
            Dim retSave() As Byte
            Dim ICI As ImageCodecInfo = GetEncoderInfo(ImageFormat.Jpeg)
            Dim EncPars As EncoderParameters = New EncoderParameters(If(iQuality > 0, 2, 1))

            EncPars.Param(0) = New EncoderParameter(Encoder.RenderMethod, EncoderValue.RenderNonProgressive)

            If iQuality > 0 Then
                EncPars.Param(1) = New EncoderParameter(Encoder.Quality, iQuality)
            End If

            _image.Save(msSave, ICI, EncPars)

            retSave = msSave.ToArray

            'make sure directory exists
            Directory.CreateDirectory(Directory.GetParent(sPath).FullName)
            If sPath.Length <= 260 Then
                Using fs As New FileStream(sPath, FileMode.Create, FileAccess.Write)
                    fs.Write(retSave, 0, retSave.Length)
                    fs.Flush()
                End Using
            End If
            msSave.Flush()
        End Using

        If doesExist And fAttWritable Then File.SetAttributes(sPath, fAtt)
    End If
Catch ex As Exception
    Master.eLog.WriteToErrorLog(ex.Message, ex.StackTrace, "Error")
End Try

End Sub

Public Class dlgImgSelect

Add in declaration: Private selURL As String = ""

edit Private Sub DoSelect, Add under Me.selIndex = iIndex: Me.selURL = poster.URL

edit Private Sub OK_Button_Click, edit Save 2xtimes in this Sub: Me.tmpImage.Save(tmpPathPlus, 100, selURL)

— Reply to this email directly or view it on GitHub https://github.com/bodrick/Ember-MM/issues/36#issuecomment-12756749 .

https://github.com/notifications/beacon/8XM1_O3IS--NYVZ2mdhRlY_0QDH_NkOEexHH4rFB_tP4Yaw9WC9tm86UrYBqnuK6.gif

Cocotus commented 11 years ago

Another problem is, that someone should definately rewrite the TMDB API-Calls in Ember. It uses an depracated API which sometimes delivers wrong results! (Not all possible posters are displayed in Ember - definately not cool)

I asked here: http://www.themoviedb.org/talk/510558fb19c2952146367a60

msavazzi commented 11 years ago

Working on it

M

From: Cocotus [mailto:notifications@github.com] Sent: Sunday, 27 January, 2013 20:32 To: bodrick/Ember-MM Cc: msavazzi Subject: Re: [Ember-MM] Saved images are always re-compressed (#36)

Another problem is, that someone should definately rewrite the TMDB API-Calls in Ember. It uses an depracated API which sometimes delivers wrong results! (Not all possible posters are displayed in Ember - definately not cool)

I asked here: http://www.themoviedb.org/talk/510558fb19c2952146367a60

— Reply to this email directly or view it on GitHub https://github.com/bodrick/Ember-MM/issues/36#issuecomment-12759742 .

https://github.com/notifications/beacon/8XM1_O3IS--NYVZ2mdhRlY_0QDH_NkOEexHH4rFB_tP4Yaw9WC9tm86UrYBqnuK6.gif

Cocotus commented 11 years ago

Great msavazzi! BTW, the nocompressing poster and fanart I coded (code above) seems to work, so far no problems during scraping at least 30 movies.

Oh and I just found an explanation to sporadic crashes of Ember . Stacktrace of crash:

bei System.String.InternalSubStringWithChecks(Int32 startIndex, Int32 length, Boolean fAlwaysCopy) bei generic.EmberCore.WebServer.clsServer.HandleConnection() in D:\Dropbox\BACKUP\Modding\Programmierung\bodrick-Ember-MM-be61071\cocotus-Ember-MM\Addons\generic.EmberCore.WebServer\Server\clsServer.vb:Zeile 71. bei System.Threading.ThreadHelper.ThreadStart_Context(Object state) bei System.Threading.ExecutionContext.runTryCode(Object userData) bei System.Runtime.CompilerServices.RuntimeHelpers.ExecuteCodeWithGuaranteedCleanup(TryCode code, CleanupCode backoutCode, Object userData) bei System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state) bei System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state) bei System.Threading.ThreadHelper.ThreadStart()

It's this line of code in Addons\generic.EmberCore.WebServer\Server\clsServer.vb , Line 71:

iStartPos = sRequest.LastIndexOf("/") + 1 sRequestedFile = sRequest.Substring(iStartPos)

This code will crash Ember If Not found (-1)... happenend to me twice!

So I replaced it with: If (iStartPos > 0) Then sRequestedFile = sRequest.Substring(iStartPos) Else sRequestedFile = sRequest End If

So far no problems anymore :)

msavazzi commented 11 years ago

Will fix that too once back at home

I've been traveling for work and did not get any free minute :(

Please excuse brevity. Sent from my Nokia Windows Phone

----- Messaggio originale ----- Da: Cocotus Inviato: 01/02/2013 11:44 A: bodrick/Ember-MM Cc: msavazzi Oggetto: Re: [Ember-MM] Saved images are always re-compressed (#36)

Great msavazzi! BTW, the nocompressing poster and fanart I coded (code above) seems to work, so far no problems during scraping at least 30 movies.

Oh and I just found an explanation to sporadic crashes of Ember . Stacktrace of crash:

bei System.String.InternalSubStringWithChecks(Int32 startIndex, Int32 length, Boolean fAlwaysCopy) bei generic.EmberCore.WebServer.clsServer.HandleConnection() in D:\Dropbox\BACKUP\Modding\Programmierung\bodrick-Ember-MM-be61071\cocotus-Ember-MM\Addons\generic.EmberCore.WebServer\Server\clsServer.vb:Zeile 71. bei System.Threading.ThreadHelper.ThreadStart_Context(Object state) bei System.Threading.ExecutionContext.runTryCode(Object userData) bei System.Runtime.CompilerServices.RuntimeHelpers.ExecuteCodeWithGuaranteedCleanup(TryCode code, CleanupCode backoutCode, Object userData) bei System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state) bei System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state) bei System.Threading.ThreadHelper.ThreadStart()

It's this line of code in Addons\generic.EmberCore.WebServer\Server\clsServer.vb , Line 71:

iStartPos = sRequest.LastIndexOf("/") + 1 sRequestedFile = sRequest.Substring(iStartPos)

This code will crash Ember If Not found (-1)... happenend to me twice!

So I replaced it with: If (iStartPos > 0) Then sRequestedFile = sRequest.Substring(iStartPos) Else sRequestedFile = sRequest End If

So far no problems anymore :)

Reply to this email directly or view it on GitHub: https://github.com/bodrick/Ember-MM/issues/36#issuecomment-13010589

Cocotus commented 11 years ago

Hey no problem, btw in earlier post I wrote:

If (iStartPos > 0) Then sRequestedFile = sRequest.Substring(iStartPos) Else sRequestedFile = sRequest End If

As the fix - but I think it's better to leave the funciton right away, cause going further with "wrong" sRequestedFile" seems to produce other problems. So I modified it to:

If (iStartPos > 0) Then sRequestedFile = sRequest.Substring(iStartPos) Else mySocket.Close() Return End If

Cheers!

msavazzi commented 11 years ago

This line is not correct

                            stroriginalurl = sUrl.Replace(stringArray(5), "original")

as if the image has the same sequence the resulting URL is invalid.

I’ve replaced the code with the correct one

                                                   stringArray(5) = "original"

                                                   stroriginalurl = Join(stringArray, "/")

Also temp was not needed to perform the split – join so I removed it to save memory

The use of Exit Sub is not appropriate as it leads to issues when following the code, is hard to find :)

This is a better way

   Public Shared Sub Save(ByVal _image As Image, ByVal sPath As String, Optional ByVal iQuality As Long = 0, Optional ByVal sUrl As String = "")

          Try

                 If IsNothing(_image) Then Exit Sub

                 Dim doesExist As Boolean = File.Exists(sPath)

                 Dim fAtt As New FileAttributes

                 If Not String.IsNullOrEmpty(sPath) AndAlso (Not doesExist OrElse (Not CBool(File.GetAttributes(sPath) And FileAttributes.ReadOnly))) Then

                       If doesExist Then

                              'get the current attributes to set them back after writing

                              fAtt = File.GetAttributes(sPath)

                              'set attributes to none for writing

                              File.SetAttributes(sPath, FileAttributes.Normal)

                       End If

                       If Not sUrl = "" Then

                              Dim stroriginalurl As String = sUrl

                              'Image Download from tmdb is special, need original size

                              If Not sUrl.Contains("impawards") AndAlso Not sUrl.Contains("movieposterdb") Then

                                     'Always get original image...

                                     'links to images (tmdb) have following structure:  'example: http://d3gtl9l2a4fn1j.cloudfront.net/t/p/w92/x65b4vsFKYuA878pLN1mJiAsgIP.jpg

                                     Dim stringArray() As String = Split(stroriginalurl, "/")

                                     If stringArray.Length > 4 Then

                                            ' stringArray(5) contains values like "w185","original", "w154"...-->size -> we want original!

                                            stringArray(5) = "original"

                                            stroriginalurl = Join(stringArray, "/")

                                     End If

                              End If

                              Dim webclient As New Net.WebClient

                              webclient.DownloadFile(stroriginalurl, sPath)

                       Else

                              Using msSave As New MemoryStream

                                     Dim retSave() As Byte

                                     Dim ICI As ImageCodecInfo = GetEncoderInfo(ImageFormat.Jpeg)

                                     Dim EncPars As EncoderParameters = New EncoderParameters(If(iQuality > 0, 2, 1))

                                     EncPars.Param(0) = New EncoderParameter(Encoder.RenderMethod, EncoderValue.RenderNonProgressive)

                                     If iQuality > 0 Then

                                            EncPars.Param(1) = New EncoderParameter(Encoder.Quality, iQuality)

                                     End If

                                     _image.Save(msSave, ICI, EncPars)

                                     retSave = msSave.ToArray

                                     Using fs As New FileStream(sPath, FileMode.Create, FileAccess.Write)

                                            fs.Write(retSave, 0, retSave.Length)

                                            fs.Flush()

                                     End Using

                                     msSave.Flush()

                              End Using

                       End If

                       If doesExist Then File.SetAttributes(sPath, fAtt)

                 End If

          Catch ex As Exception

                 Master.eLog.WriteToErrorLog(ex.Message, ex.StackTrace, "Error")

          End Try

   End Sub

I’ve added all your mods but why you have not changed SaveAsFanart etc?

M

From: Cocotus [mailto:notifications@github.com] Sent: Sunday, 27 January, 2013 17:32 To: bodrick/Ember-MM Cc: msavazzi Subject: Re: [Ember-MM] Saved images are always re-compressed (#36)

Good ideas guys, I also doesn't like the fact that Ember recompresses before saving the file, so I added some few modification to download the image again when I click the ok Button and selected an image. I will test it a bit and tell you how it work. It's possible that it requires a bit modification. So far the modified code:

Public Class ScrapeImages, edit Save Method

Public Shared Sub Save(ByVal _image As Image, ByVal sPath As String, Optional ByVal iQuality As Long = 0, Optional ByVal sUrl As String = "") Try If IsNothing(_image) Then Exit Sub

    Dim doesExist As Boolean = File.Exists(sPath)
    Dim fAtt As New FileAttributes
    If Not String.IsNullOrEmpty(sPath) AndAlso (Not doesExist OrElse (Not CBool(File.GetAttributes(sPath) And FileAttributes.ReadOnly))) Then
        If doesExist Then
            'get the current attributes to set them back after writing
            fAtt = File.GetAttributes(sPath)
            'set attributes to none for writing
            File.SetAttributes(sPath, FileAttributes.Normal)

            'pm
            Try
                If Not sUrl = "" Then
                    Dim stroriginalurl As String = sUrl

                    'Image Download from tmdb is special, need original size
                    If Not sUrl.Contains("impawards") AndAlso Not sUrl.Contains("movieposterdb") Then
                        'Always get original image...
                        'links to images (tmdb) have following structure:  'example: http://d3gtl9l2a4fn1j.cloudfront.net/t/p/w92/x65b4vsFKYuA878pLN1mJiAsgIP.jpg
                        Dim temp As String = sUrl
                        Dim stringArray() As String = Split(temp, "/")
                        If stringArray.Length > 4 Then
                            ' stringArray(5) contains values like "w185","original", "w154"...-->size -> we want original!
                            stroriginalurl = sUrl.Replace(stringArray(5), "original")
                        End If
                    End If
                    Dim webclient As New Net.WebClient
                    webclient.DownloadFile(stroriginalurl, sPath)
                    If doesExist Then File.SetAttributes(sPath, fAtt)
                    Exit Sub
                End If
            Catch ex As Exception
                Master.eLog.WriteToErrorLog(ex.Message, ex.StackTrace, "Error")
            End Try
            'pm

        End If

        Using msSave As New MemoryStream
            Dim retSave() As Byte
            Dim ICI As ImageCodecInfo = GetEncoderInfo(ImageFormat.Jpeg)
            Dim EncPars As EncoderParameters = New EncoderParameters(If(iQuality > 0, 2, 1))

            EncPars.Param(0) = New EncoderParameter(Encoder.RenderMethod, EncoderValue.RenderNonProgressive)

            If iQuality > 0 Then
                EncPars.Param(1) = New EncoderParameter(Encoder.Quality, iQuality)
            End If

            _image.Save(msSave, ICI, EncPars)

            retSave = msSave.ToArray

            Using fs As New FileStream(sPath, FileMode.Create, FileAccess.Write)
                fs.Write(retSave, 0, retSave.Length)
                fs.Flush()
            End Using
            msSave.Flush()
        End Using

        If doesExist Then File.SetAttributes(sPath, fAtt)
    End If
Catch ex As Exception
    Master.eLog.WriteToErrorLog(ex.Message, ex.StackTrace, "Error")
End Try

End Sub

Public Class Images, edit Save Method

Public Sub Save(ByVal sPath As String, Optional ByVal iQuality As Long = 0, Optional ByVal sUrl As String = "") Try If IsNothing(_image) Then Exit Sub

    Dim doesExist As Boolean = File.Exists(sPath)
    Dim fAtt As New FileAttributes
    Dim fAttWritable As Boolean = True

    If Not String.IsNullOrEmpty(sPath) AndAlso (Not doesExist OrElse (Not CBool(File.GetAttributes(sPath) And FileAttributes.ReadOnly))) Then
        If doesExist Then
            'get the current attributes to set them back after writing
            fAtt = File.GetAttributes(sPath)
            'set attributes to none for writing
            Try
                File.SetAttributes(sPath, FileAttributes.Normal)
            Catch ex As Exception
                fAttWritable = False
            End Try
        End If

        'pm
        Try
            If Not sUrl = "" Then
                'TODO V3 API implementation to get ALL posters! http://docs.themoviedb.apiary.io/#configuration
                '  GetsImagesFromTMDBv3("URL/MOVIEDID")

                Dim stroriginalurl As String = sUrl
                'Image Download from tmdb is special, need original size
                If Not sUrl.Contains("impawards") AndAlso Not sUrl.Contains("movieposterdb") Then
                    'Always get original image...
                    'links to images (tmdb) have following structure:  'example: http://d3gtl9l2a4fn1j.cloudfront.net/t/p/w92/x65b4vsFKYuA878pLN1mJiAsgIP.jpg
                    Dim temp As String = sUrl
                    Dim stringArray() As String = Split(temp, "/")
                    If stringArray.Length > 4 Then
                        ' stringArray(5) contains values like "w185","original", "w154"...-->size -> we want original!
                        stroriginalurl = sUrl.Replace(stringArray(5), "original")
                    End If
                End If

                Dim webclient As New Net.WebClient
                'Download image!
                webclient.DownloadFile(stroriginalurl, sPath)

                If doesExist And fAttWritable Then File.SetAttributes(sPath, fAtt)
                Exit Sub

            End If
        Catch ex As Exception
            Master.eLog.WriteToErrorLog(ex.Message, ex.StackTrace, "Error")
        End Try
        'pm

        Using msSave As New MemoryStream
            Dim retSave() As Byte
            Dim ICI As ImageCodecInfo = GetEncoderInfo(ImageFormat.Jpeg)
            Dim EncPars As EncoderParameters = New EncoderParameters(If(iQuality > 0, 2, 1))

            EncPars.Param(0) = New EncoderParameter(Encoder.RenderMethod, EncoderValue.RenderNonProgressive)

            If iQuality > 0 Then
                EncPars.Param(1) = New EncoderParameter(Encoder.Quality, iQuality)
            End If

            _image.Save(msSave, ICI, EncPars)

            retSave = msSave.ToArray

            'make sure directory exists
            Directory.CreateDirectory(Directory.GetParent(sPath).FullName)
            If sPath.Length <= 260 Then
                Using fs As New FileStream(sPath, FileMode.Create, FileAccess.Write)
                    fs.Write(retSave, 0, retSave.Length)
                    fs.Flush()
                End Using
            End If
            msSave.Flush()
        End Using

        If doesExist And fAttWritable Then File.SetAttributes(sPath, fAtt)
    End If
Catch ex As Exception
    Master.eLog.WriteToErrorLog(ex.Message, ex.StackTrace, "Error")
End Try

End Sub

Public Class dlgImgSelect

Add in declaration: Private selURL As String = ""

edit Private Sub DoSelect, Add under Me.selIndex = iIndex: Me.selURL = poster.URL

edit Private Sub OK_Button_Click, edit Save 2xtimes in this Sub: Me.tmpImage.Save(tmpPathPlus, 100, selURL)

— Reply to this email directly or view it on GitHub https://github.com/bodrick/Ember-MM/issues/36#issuecomment-12756749 .

https://github.com/notifications/beacon/8XM1_O3IS--NYVZ2mdhRlY_0QDH_NkOEexHH4rFB_tP4Yaw9WC9tm86UrYBqnuK6.gif

msavazzi commented 11 years ago

Coctus,

I’ve added you changes but I think they are partial as the SaveFanart, SavePoster and the other saves are still using the quality.

Can you check what happens with different options?

From: Cocotus [mailto:notifications@github.com] Sent: Friday, 1 February, 2013 20:44 To: bodrick/Ember-MM Cc: msavazzi Subject: Re: [Ember-MM] Saved images are always re-compressed (#36)

Great msavazzi! BTW, the nocompressing poster and fanart I coded (code above) seems to work, so far no problems during scraping at least 30 movies.

Oh and I just found an explanation to sporadic crashes of Ember . Stacktrace of crash:

bei System.String.InternalSubStringWithChecks(Int32 startIndex, Int32 length, Boolean fAlwaysCopy) bei generic.EmberCore.WebServer.clsServer.HandleConnection() in D:\Dropbox\BACKUP\Modding\Programmierung\bodrick-Ember-MM-be61071\cocotus-Ember-MM\Addons\generic.EmberCore.WebServer\Server\clsServer.vb:Zeile 71. bei System.Threading.ThreadHelper.ThreadStart_Context(Object state) bei System.Threading.ExecutionContext.runTryCode(Object userData) bei System.Runtime.CompilerServices.RuntimeHelpers.ExecuteCodeWithGuaranteedCleanup(TryCode code, CleanupCode backoutCode, Object userData) bei System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state) bei System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state) bei System.Threading.ThreadHelper.ThreadStart()

It's this line of code in Addons\generic.EmberCore.WebServer\Server\clsServer.vb , Line 71:

iStartPos = sRequest.LastIndexOf("/") + 1 sRequestedFile = sRequest.Substring(iStartPos)

This code will crash Ember If Not found (-1)... happenend to me twice!

So I replaced it with: If (iStartPos > 0) Then sRequestedFile = sRequest.Substring(iStartPos) Else sRequestedFile = sRequest End If

So far no problems anymore :)

— Reply to this email directly or view it on GitHub https://github.com/bodrick/Ember-MM/issues/36#issuecomment-13010589 .

https://github.com/notifications/beacon/Jshd8sI44GVrKZBvymxqKFPoNwzUzxZH563rTp7UvZTBJJShpAfzL_RNxjhZPUOH.gif

msavazzi commented 11 years ago

I’ve put this code in but can you provide us the example where it was happening?

M

From: Cocotus [mailto:notifications@github.com] Sent: Sunday, 3 February, 2013 09:15 To: bodrick/Ember-MM Cc: msavazzi Subject: Re: [Ember-MM] Saved images are always re-compressed (#36)

Hey no problem, btw in earlier post I wrote:

If (iStartPos > 0) Then sRequestedFile = sRequest.Substring(iStartPos) Else sRequestedFile = sRequest End If

As the fix - but I think it's better to leave he funciton right away, cause going further with "wrong" sRequestedFile" seems to produce other problems. So I modified it to:

If (iStartPos > 0) Then sRequestedFile = sRequest.Substring(iStartPos) Else mySocket.Close() Return End If

Cheers!

— Reply to this email directly or view it on GitHub https://github.com/bodrick/Ember-MM/issues/36#issuecomment-13043915 .

https://github.com/notifications/beacon/8XM1_O3IS--NYVZ2mdhRlY_0QDH_NkOEexHH4rFB_tP4Yaw9WC9tm86UrYBqnuK6.gif

Cocotus commented 11 years ago

"’ve added you changes but I think they are partial as the SaveFanart, SavePoster and the other saves are still using the quality. Can you check what happens with different options?"

Hmm do you mean the methods SaveAsFanart, SaveAsPoster in clsAPImages? I think they all using the Save method which I edited or I'm missing something?

EDIT: Btw I'm not using Ember with Series, so I can't check it out if thats the problem :)

msavazzi commented 11 years ago

Correct but they invoke the save method WITHOUT the filename so they always recompress.

That is correct IF the final filesize is different from the downloaded size but if they are equal they are not correct.

I think you have to do some more tests and cross check.

M

From: Cocotus [mailto:notifications@github.com] Sent: Thursday, 7 February, 2013 23:28 To: bodrick/Ember-MM Cc: msavazzi Subject: Re: [Ember-MM] Saved images are always re-compressed (#36)

"’ve added you changes but I think they are partial as the SaveFanart, SavePoster and the other saves are still using the quality. Can you check what happens with different options?"

Hmm do you mean the methods SaveAsFanart, SaveAsPoster in clsAPImages? I think they all using the Save method which I edited or I'm missing something?

— Reply to this email directly or view it on GitHub https://github.com/bodrick/Ember-MM/issues/36#issuecomment-13265409 .

https://github.com/notifications/beacon/J6T91GIPIyhU-8ti4GCGP1qCR1AZSgT-5mw4hCHRlRdwAZCCSqJM1AwRAJk0uaWq.gif

Cocotus commented 11 years ago

I see now thanks! I looked into it and so far thats how it works in the moment. You go in Edit Movie -> Poster Tab -> Scrape Poster --> Select the one you like in the preview window --> And now my modification will make sure to download that file....but only in the LOCAL Temp-folder of Ember!

Now to save the poster/fanart to the movie folder you have to hit "OK" Button in the tab and it will call "SavePoster" or "SaveFanart" method. But here's still recrompressing which you stated above.

Now since we have already downloaded the image to the TEMP Directory of Ember all there's need to do is copy that image (its always called poster/fanart.jpg) to the moviepath folder.

What is strange, that I thought that my code I posted earlier was already working because I compared the filesize/DPI of the orignal image on tmdb with my created fanart/poster in movie-folder and so far the files were identical! But theoretically there should be differences since it's still doing recrompressing thing - or is there something else going on like copying from TEMP-directory to MovieFolder - thats the only logical explanation for this.

msavazzi commented 11 years ago

Cocotus,

In theory a recompress with 100% quality should be identical to original, if you look at filesize or DPI they could not change.

You have to do a graphic diff or use CRC32 to check if the files are the same or not.

I do not think there is any copy as the original EMM was keeping all the images in memory :)

If you can continue to look into it a propose me a solution it would be great.

I’m working on the new scraper for TMDB…

From: Cocotus [mailto:notifications@github.com] Sent: Friday, 8 February, 2013 18:09 To: bodrick/Ember-MM Cc: msavazzi Subject: Re: [Ember-MM] Saved images are always re-compressed (#36)

I see now thanks! I looked into it and so far thats how it works in the moment. You go in Edit Movie -> Poster Tab -> Scrape Poster --> Select the one you like in the preview window --> And now my modification will make sure to download that file....but only in the LOCAL Temp-folder of Ember!

Now to save the poster/fanart to the movie folder you have to hit "OK" Button in the tab and it will call "SavePoster" or "SaveFanart" method. But here's still recrompressing which you stated above.

Now since we have already downloaded the image to the TEMP Directory of Ember all there's need to do is copy that image (its always called poster/fanart.jpg) to the moviepath folder.

What is strange, that I thought that my code I posted earlier was already working because I compared the filesize/DPI of the orignal image on tmdb with my created fanart/poster in movie-folder and so far the files were identical! But theoretically there should be differences since it's still doing recrompressing thing - or is there something else going on like copying from TEMP-directory to MovieFolder - thats the only logical explanation for this.

— Reply to this email directly or view it on GitHub https://github.com/bodrick/Ember-MM/issues/36#issuecomment-13300337 .

https://github.com/notifications/beacon/J6T91GIPIyhU-8ti4GCGP1qCR1AZSgT-5mw4hCHRlRdwAZCCSqJM1AwRAJk0uaWq.gif

Cocotus commented 11 years ago

I have rescraped about 50 movies the last days and I will check/compare if there are differences. Thanks for the hints. Also good job on working on the new scraper, I think it's definately a good idea :)