search

jbrukh / octopus

Project Octopus
0 stars 0 forks source link

When reading OBF files use timestamps to drive duration #23

Closed jonnii closed 11 years ago

jonnii commented 11 years ago

sample_rate isn't accurate enough.

jbrukh commented 11 years ago

Also, are we sure we want to duplicate this logic in Ruby?

I have eventual plans for the format and it is going to change. We will need to maintain two implementations.

One option is having backend Go services that serves CSV to the front-end, very easy. Another options is trying to do some sort of binding with the Go OBF library. Not sure the best way here.

jonnii commented 11 years ago

It's not ideal, but I want to get something working. We can always change it to GO later.

jbrukh commented 11 years ago

kk

On Thu, May 9, 2013 at 12:38 PM, Jonathan Goldman notifications@github.com wrote:

It's not ideal, but I want to get something working. We can always change it to GO later.

Reply to this email directly or view it on GitHub: https://github.com/jbrukh/octopus/issues/23#issuecomment-17674571

jonnii commented 11 years ago

One of the potential pitfalls, on heroku at least, would be that you can't have two apps running on the same dyno both bound to port 80. The solution to this would be to either:

  1. Run a different app with access to the db, which would run on a different subdomain.
  2. Run a different app on the same dyno with a non-80 port and proxy from rails to the app.
  3. Don't use heroku, proxy upstream using nginx to the right app using a pattern.

There may be other options, it might be worth exploring them.

jbrukh commented 11 years ago

On whether we want to expose the backend service directly or whether it's just something that the rails app uses for processing behind the scenes.

Trying to think of use-cases (for my own benefit)...

Assumptions: (a) the recording has a resource_id; (b) the resource_id is stored in the db along with metadata about the trial; (c) the OBF file for the recording lives in S3 under the resource_id.

  1. User wants to retrieve a recording and look at it. So we need the data for a recording in a format JS will understand (JSON, CSV, etc). We hit /api/recordings/[resource_id]. At this point we could go to S3 directly from rails, parse the file in ruby, and provide the output. This use-case alone does not need a service.
  2. User wants to apply an algorithm to a recording. We would want to specify a resource_id and an algorithm_id. Processing should be clearly done in a fast language. We send these to a service, service retrieves resource, processes it, and creates a new resource that is the output. So the procedure to view it is the same as in (1) and this service need not (should not) be front-facing.

The generalization to (1) is if were to make the conversion between binary and CSV happen in Go, in a service. Service goes to S3 (or just gets the binary itself as a parameter), does the processing in the OBF library, etc. In this case, proxying seems correct.

So it seems to me:

At least in the short-run this seems okay.

On Thu, May 9, 2013 at 12:49 PM, Jonathan Goldman notifications@github.comwrote:

One of the potential pitfalls, on heroku at least, would be that you can't have two apps running on the same dyno both bound to port 80. The solution to this would be to either:

  1. Run a different app with access to the db, which would run on a different subdomain.
  2. Run a different app on the same dyno with a non-80 port and proxy from rails to the app.
  3. Don't use heroku, proxy upstream using nginx to the right app using a pattern.

There may be other options, it might be worth exploring them.

— Reply to this email directly or view it on GitHubhttps://github.com/jbrukh/octopus/issues/23#issuecomment-17675218 .

jonnii commented 11 years ago

This isn't an issue now, we're not going to read the duration.