search

Sitefinity / amazon-s3-provider

Amazon S3 provider for Sitefinity
8 stars 10 forks source link

Sitefinity 5.4 support #1

Closed scleaver closed 10 years ago

scleaver commented 11 years ago

Hi,

I installed this into 5.4 with no joy. Should it be compatible?

I got an error on the advanced settings where it has a pop up with "test connection" and also an error in image libraries when I tried to move a library to s3... it said "failed" but with no details.

Thanks,

Seth

devkev2403 commented 11 years ago

I have installed this successfully on Sitefinity 5.4 and it works fine. Be sure to follow the instructions carefully, specifically when it comes to the parameter names.

I have experienced an issue with the "Move library" instructions though and I have raised it with Sitefinity support. Here is what I wrote:

I have debugged this and the code that throws the error in the plug-in is:

public override long Upload(IBlobContent content, Stream source, int bufferSize)
{
            var request = new TransferUtilityUploadRequest()
                .WithBucketName(this.bucketName)
                .WithKey(content.FilePath)
                .WithPartSize(bufferSize)
                .WithContentType(content.MimeType);

            //set the item's accessibility as public
            request.AddHeader("x-amz-acl", "public-read");

            //get it before the upload, because afterwards the stream is closed already
            long sourceLength = source.Length;
            request.InputStream = source;

            try
            {
                this.transferUtility.Upload(request);  // THIS THROWS AN EXCEPTION WHEN MOVING A LIBRARY BUT NOT WHEN UPLOADING A NEW IMAGE
            }
            catch (Exception ex)
            {
                throw (ex);
            }

However, the same code is called when uploading a new image to an S3 library and that works.

The following is written to the Sitefinity error log.

Timestamp: 01/03/2013 15:04:51

Message: HandlingInstanceID: 990b9a4b-0517-4a17-aa28-6cf6d58e07f4
An exception of type 'Telerik.Sitefinity.Scheduling.TaskFailedException' occurred and was caught.

03/01/2013 15:04:51
Type : Telerik.Sitefinity.Scheduling.TaskFailedException, Telerik.Sitefinity, Version=5.4.4010.0, Culture=neutral, PublicKeyToken=b28c218413bdf563
Message : Task '00d22a58-28ac-416e-b43b-4c89c0b388b0' (aka 'TransferLibrary') has FAILED: Specified method is not supported.
Source :
Help link :
Data : System.Collections.ListDictionaryInternal
TargetSite :
HResult : -2146233088
Stack Trace : The stack trace is unavailable.
Additional Info:

MachineName : PC0801
TimeStamp : 01/03/2013 15:04:51
FullName : Telerik.Sitefinity.Utilities, Version=5.4.4010.0, Culture=neutral, PublicKeyToken=b28c218413bdf563
AppDomainName : /LM/W3SVC/2/ROOT-3-130066230177596842
ThreadIdentity :
WindowsIdentity : IIS APPPOOL\ASP.NET v4.0
Requested URL : http://localhost/
    Inner Exception

    Type : System.NotSupportedException, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089
    Message : Specified method is not supported.
    Source : Telerik.Sitefinity
    Help link :
    Data : System.Collections.ListDictionaryInternal
    TargetSite : Int64 get_Position()
    HResult : -2146233067
    Stack Trace :    at Telerik.Sitefinity.Modules.Libraries.LibraryRelocationTask.ExecuteTask()
       at Telerik.Sitefinity.Scheduling.Scheduler.ExecuteTask(ScheduledTask task, String transactionName)

Category: ErrorLog
Priority: 0
EventId: 9010
Severity: Warning
Title:Enterprise Library Exception Handling
Machine: PC0801
App Domain: /LM/W3SVC/2/ROOT-3-130066230177596842
ProcessId: 6460
Process Name: c:\windows\system32\inetsrv\w3wp.exe
Thread Name:
Win32 ThreadId:9572
Extended Properties:

Digging deeper, when moving a library, the stack trace from the exception is:

at Telerik.Sitefinity.Modules.Libraries.BlobStorage.BlobDownloadStream.get_Position() at Amazon.S3.AmazonS3Client.invoke[T](S3AsyncResult s3AsyncResult, Boolean isRedirect) at Amazon.S3.AmazonS3Client.invokePutObject(PutObjectRequest request, AsyncCallback callback, Object state, Boolean synchronized) at Amazon.S3.AmazonS3Client.PutObject(PutObjectRequest request) at Telerik.Sitefinity.Amazon.BlobStorage.AmazonBlobStorageProvider.Upload(IBlobContent content, Stream source, Int32 bufferSize) in c:\githome\amazon-s3-provider-master\BlobStorage\AmazonBlobStorageProvider.cs:line 130

and interestingly, looking at the source for Telerik.Sitefinity.Modules.Libraries.BlobStorage.BlobDownloadStream.get_Position() it is clear why the exception is thrown.

public override long Position
{
    get
    {
        throw new NotSupportedException();
    }
    set
    {
        if (value != (long)0)
        {
            throw new NotSupportedException("The only valid value for setting the position of the stream is '0'.");
        }
        else
        {
            this.currentChunk = null;
            this.currentChunkOrdinal = 0;
            return;
        }
    }
}

Moving a library must be doing something different to uploading an image.

scleaver commented 11 years ago

Hi,

Thanks for getting back to me. Maybe it is a setting issue on the aws end?

Do I need to turn on static files serving or anything like that?

Sent from my iPhone

On 02/03/2013, at 1:29 AM, devkev2403 notifications@github.com wrote:

I have installed this successfully on Sitefinity 5.4 and it works fine. Be sure to follow the instructions carefully, specifically when it comes to the parameter names.

I have experienced an issue with the "Move library" instructions though and I have raised it with Sitefinity support. Here is what I wrote:

I have debugged this and the code that throws the error in the plug-in is:

public override long Upload(IBlobContent content, Stream source, int bufferSize) { var request = new TransferUtilityUploadRequest() .WithBucketName(this.bucketName) .WithKey(content.FilePath) .WithPartSize(bufferSize) .WithContentType(content.MimeType);

    //set the item's accessibility as public
    request.AddHeader("x-amz-acl", "public-read");

    //get it before the upload, because afterwards the stream is

closed already long sourceLength = source.Length; request.InputStream = source;

    try
    {
        this.transferUtility.Upload(request);  // THIS THROWS AN

EXCEPTION WHEN MOVING A LIBRARY BUT NOT WHEN UPLOADING A NEW IMAGE } catch (Exception ex) { throw (ex); }

However, the same code is called when uploading a new image to an S3 library and that works.

The following is written to the Sitefinity error log.

Timestamp: 01/03/2013 15:04:51

Message: HandlingInstanceID: 990b9a4b-0517-4a17-aa28-6cf6d58e07f4 An exception of type 'Telerik.Sitefinity.Scheduling.TaskFailedException' occurred and was caught.

03/01/2013 15:04:51 Type : Telerik.Sitefinity.Scheduling.TaskFailedException, Telerik.Sitefinity, Version=5.4.4010.0, Culture=neutral, PublicKeyToken=b28c218413bdf563 Message : Task '00d22a58-28ac-416e-b43b-4c89c0b388b0' (aka 'TransferLibrary') has FAILED: Specified method is not supported. Source : Help link : Data : System.Collections.ListDictionaryInternal TargetSite : HResult : -2146233088 Stack Trace : The stack trace is unavailable. Additional Info:

MachineName : PC0801 TimeStamp : 01/03/2013 15:04:51 FullName : Telerik.Sitefinity.Utilities, Version=5.4.4010.0, Culture=neutral, PublicKeyToken=b28c218413bdf563 AppDomainName : /LM/W3SVC/2/ROOT-3-130066230177596842 ThreadIdentity : WindowsIdentity : IIS APPPOOL\ASP.NET v4.0 Requested URL : http://localhost/ Inner Exception

Type : System.NotSupportedException, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089 Message : Specified method is not supported. Source : Telerik.Sitefinity Help link : Data : System.Collections.ListDictionaryInternal TargetSite : Int64 get_Position() HResult : -2146233067 Stack Trace : at Telerik.Sitefinity.Modules.Libraries.LibraryRelocationTask.ExecuteTask() at Telerik.Sitefinity.Scheduling.Scheduler.ExecuteTask(ScheduledTask task, String transactionName)

Category: ErrorLog Priority: 0 EventId: 9010 Severity: Warning Title:Enterprise Library Exception Handling Machine: PC0801 App Domain: /LM/W3SVC/2/ROOT-3-130066230177596842 ProcessId: 6460 Process Name: c:\windows\system32\inetsrv\w3wp.exe Thread Name: Win32 ThreadId:9572 Extended Properties:

Digging deeper, when moving a library, the stack trace from the exception is:

at Telerik.Sitefinity.Modules.Libraries.BlobStorage.BlobDownloadStream.get_Position() at Amazon.S3.AmazonS3Client.invokeT<S3AsyncResult%20s3AsyncResult,%20Boolean%20isRedirect> at Amazon.S3.AmazonS3Client.invokePutObject(PutObjectRequest request, AsyncCallback callback, Object state, Boolean synchronized) at Amazon.S3.AmazonS3Client.PutObject(PutObjectRequest request) at Telerik.Sitefinity.Amazon.BlobStorage.AmazonBlobStorageProvider.Upload(IBlobContent content, Stream source, Int32 bufferSize) in c:\githome\amazon-s3-provider-master\BlobStorage\AmazonBlobStorageProvider.cs:line 130

and interestingly, looking at the source for Telerik.Sitefinity.Modules.Libraries.BlobStorage.BlobDownloadStream.get_Position() it is clear why the exception is thrown.

''' public override long Position { get { throw new NotSupportedException(); } set { if (value != (long)0) { throw new NotSupportedException("The only valid value for setting the position of the stream is '0'."); } else { this.currentChunk = null; this.currentChunkOrdinal = 0; return; } } } '''

Moving a library must be doing something different to uploading an image.

— Reply to this email directly or view it on GitHubhttps://github.com/Sitefinity/amazon-s3-provider/issues/1#issuecomment-14294385 .

devkev2403 commented 11 years ago

AWS settings are beyond me. I simply created a new account and followed the plug in instructions. You could try that to see if it is an AWS settings issue.

scleaver commented 11 years ago

Hi,

Thanks for the response. This is looking good.

I got it working. It was because I tried to move a library and that just doesn't work as you pointed out. I created a new library and it worked when uploading a 150 Kb file.

There are a few issues though:

  1. Large files won't upload. I tried to upload a 22Mb file and I got an error once the file was uploaded. It seems to upload it to the server but then the move from server to S3 appears to fail. Any ideas what the go is there? This is a pity because this was actually the main reason for my interest.
  2. Can't move a library from another provider eg. Database to the S3 provider - it fails.
  3. Can't move a file/image from or to a S3 library to a non-S3 library eg. move a image from Database library to S3 library - it says "not supported"
  4. Files seem to be uploaded to S3 as non-public and you have to go and manually set it as public in the AWS console. This may be a setting in AWS somewhere.

The Sitefinity team have blogged about this provider and it is great but there seems to be a few issues and it is not production ready yet. Do you have any contacts in Telerik that we could get some help from Telerik on these items resolved? I have a lot of libraries that I would like to move to S3.

Actually I just found out how to resolve point 4. You need to generate an access policy for getObject:

http://docs.aws.amazon.com/AmazonS3/latest/dev/AccessPolicyLanguage_UseCases_s3_a.html

Cheers,

Seth

On Mon, Mar 4, 2013 at 7:41 PM, devkev2403 notifications@github.com wrote:

AWS settings are beyond me. I simply created a new account and followed the plug in instructions. You could try that to see if it is an AWS settings issue.

— Reply to this email directly or view it on GitHubhttps://github.com/Sitefinity/amazon-s3-provider/issues/1#issuecomment-14371982 .

taosd commented 11 years ago

Hey,

First off, I'm glad you wrote this provider. Thank you! Gave us a good jump on things. As Seth noted above, the plugin, as is, does not allow you to upload large files. The error which gets thrown is that the file you attempted to upload was too small. I was able to fix this by modifying the partSize when uploading. By default, it's set to what ever the buffer size is (as passed to the Upload function). I increased this value to 6 MB, and I no longer get the error, and successfully uploaded a 47.5 MB file.

When instantiating the TransferUtilityUploadRequest object, when calling the "WithPartSize" bit, instead of passing bufferSize, I passed "6291456" (as an int). Then it worked, as far as uploading large files..