Netflix-Skunkworks / commuter

🚎 Notebook sharing hub
BSD 3-Clause "New" or "Revised" License
1 stars 1 forks source link

Merge S3 Commuter Into Commuter, following bookstore organization #1

Open rgbkrk opened 5 years ago

rgbkrk commented 5 years ago

We now have a few discrete prefixes associated with one common bucket, granted via bookstore:

We currently run two commuters. One for S3 (coded to work against one bucket and prefix at a time) and one for EFS. Given that we're going to take commuter off of EFS we need to have a transitiion over to the S3 backed version (what's now going to /workspace). Here's what I propose (and what @stormpython, @mpacer talked about yesterday):

mpacer commented 5 years ago

In order to make the frontend pages know what prefix will actually be used, we will probably want to expose configuration at a higher level in the package.

Right now, there's an implicit coordination happening via the COMMUTER_PORT env variable, which has been the only place where configuration would matter before. That's why this hasn't been a pressing need up to this point.