Multipart upload leads to Rails PG error

[alfmatos]

Hi, we're integrating the ruby client 2.0.1, in our Rails 5.1. project. We have our own frontend uploader, and we just need to upload the file. We do a regular upload and then save the FilestackFilelink handle to a local model (Attachment) using the code below:

security = FilestackSecurity.new(app_secret, options: {})
client = FilestackClient.new(apikey, security: security)
filelink_handle = client.upload(filepath: path)
Attachment.create!(handle: filelink_handle.handle)

However, this throws an exception on the ActiveRecord create.

ActiveRecord::StatementInvalid (PG::ConnectionBad: PQconsumeInput() server closed the connection unexpectedly
    This probably means the server terminated abnormally
    before or while processing the request.

I've narrowed it down to the multipart operation. Somewhere along the line something is throwing an exception that gets propagated into the Rails internals. A workaround is to force multipart to false:

filelink_handle = client.upload(filepath: path, multipart: false)

Any ideas on what could be causing this exception?

[staturecrane]

@alfmatos There is nothing that I can see based on your description. Multipart sends requests to more endpoints, but that's the only real difference between the two. If one works, so should the other. Could you provide me with more details about your Rails app, your platform, how you are serving, etc?

[alfmatos]

@staturecrane sorry for not getting back to you sooner. Here is the info:

filestack 2.1.0 rails 5.1.1 pg 0.20.0 (postgres 9.6.4)

It happens in development mode, which I'm serving through Puma 3.8.2. It also throws the exception when running the commands in the console. Debugging didn't yield any results in finding out if a silent exception was being thrown somewhere along the call stack.

I guess the next step it so setup an empty rails project and test?

[alfmatos]

Well, just gave a quick spin to a rails project with file stack: https://github.com/alfmatos/rails-filestack

And this works, with and without multipart, so it's another gem or step in the environment. I'll upgrade the test to work with a tempfile, and comeback with more info.

[staturecrane]

@alfmatos Thanks for the update and doing some extra testing. Please update me with what you find, because even if it's not our SDK, it would great to know if there are widely-used Gems or processes we have conflicts with for some reason.

[alfmatos]

@staturecrane added the tempfile and bumped into an error when multipart is on:

{"error"=>"Invalid size", "c"=>"78a31f45-6cce-4e87-8184-852688c12504", "i"=>"UPL-EU-W-1-99", "t"=>1503679096}

For some reason this pops as a Runtime Error, so it might get aliased in our main app.

If you want to test it on your side, just clone the project, do the migration and go to : localhost:3000/attachments

I've coded all the magic inside the attachments_controller.rb to make it simpler.

[staturecrane]

@alfmatos I am going to start looking into this and see why that error might be happening with the tempfile. In the meantime, would you mind updating to the newest Gem and seeing if that fixes anything? We made a small fix to the upload functionality and I want to see if that was affecting you at all.

[alfmatos]

@staturecrane this test project is using version 2.2.0. Just tested the differences to 2.1.0, and the only change is that the error is more cryptic, and maybe part of the problem why I don't get any output in our main project.

undefined method `Exception' for #<FilestackClient:0x007fee5fb2f7c8>

[bjorntrondsen]

I was seeing the same issue today with the latest version of filestack-rails, which still depends on filestack v2.1.0

v.2.1.0 seems to cause issues for the pg gem (postgres adapter) since there are no issues with the sqlite gem for instance.

v.2.2.1 does not have this issues. More specifically, it looks like switching the Parallel gem to using threads with this line is what solved it: Parallel.map(batch, in_threads: 4) do |part| https://github.com/filestack/filestack-ruby/blob/master/lib/filestack/utils/multipart_upload_utils.rb#L163

I've disabled multipart uploading for now until a new version of filestack-rails is released. Pull request https://github.com/filestack/filestack-rails/pull/179

[tmaconnelly]

Hello, curious if there is any new developments on this issue, I ran into the same issue today.

[acerdan92]

Thank you @bjorntrondsen, it works for me.

[gabifija]

I'm going to close the issue, but please reopen if the newest version of filestack-ruby also causes the problem.