Concerns with backends

[asiniy]

I can't get the logic of cache & store backends. In refile file is placed into cache backend until record is stored, but it's working slightly different here.

I'm implementing shot processor now, which is about taking frames of the videofiles due to specific time of video. I haven't extracted its logic from model yet.

The overall code looks like that (I'm inserting process_shot method to changeset):

  defp process_shot(changeset, file) do
    thumbnail_time = Ecto.Changeset.get_change(changeset, :thumbnail_time, "0")
    video_path     = Exfile.Phoenix.Helpers.exfile_url(MyReelty.Endpoint, file) # the problem here
    temp_path      = "/tmp/#{Ecto.UUID.generate}"

    System.cmd("ffmpeg", ["-i", video_path, "-ss", thumbnail_time, "-vframes", "1", "-f", "image2", temp_path])

    shot = %Exfile.LocalFile{
      path: temp_path
    }

    cast(changeset, %{ shot: shot }, [:shot])
  end

Here is the problem. I have a exfile-b2 installed, and there are configs:

  backends: %{
    "store" => {ExfileB2.Backend,
      hasher:           Exfile.Hasher.Random,
      account_id:       ":P",
      application_key:  ":P",
      bucket:           ":P"
    },
    "cache" => {Exfile.Backend.FileSystem,
      directory: Path.expand("tmp/attachments")
    }
  }

In the changeset I'm supposing file to be stored in the local system according to cache backend system, but it doesn't. That's why my code makes roundtrip by downloading video from b2 and then uploading its shot back. This is definitely big price in the terms of both time and price for the traffic. What's wrong with cache backend? How can I deal with my situation?

[keichan34]

Right now, when you upload a file via the multipart method, the file will be directly uploaded to the store backend. To upload the file to the cache backend, you have to use the PUT /attachments/cache endpoint to upload the file outside the context of an ecto transaction.

In your case, you may be able to upload the file using that endpoint, set the file ID in to the changeset, then when you're done processing, you can upload it to the store backend. It may be a bit easier to write an Exfile.Processor and put it in the list of preprocessor, though -- those will run before the file is uploaded.

I agree, this isn't ideal -- I made a mistake by uploading the file immediately when the file is assigned to the struct using Module.change/3. I'll deal with this in #31 by automatically uploading to the cache backend first, then uploading to the store backend after the save to the DB has succeeded.