feat: better way of interacting with databases

[sg-s]

problem description

a typical user flow is

• get some data from a database. data is then exstant as a pandas dataframe
• do some computation on it
• maybe write results to a new column in the dataframe
• write those results back to the DO database (sync back)

right now, to do so, we have to worry about API calls where we have to laboriously pass database ID, column IDs, etc.

proposed solution -- ux

1. user creates a df using pandas-like syntax

2. user modifies some data

the dataframe warns user that there are unsynced changes

3. user uses pandas-like to_deeporigin df method to write changes back to DO

• only updated data is sent back
• the df intelligently keeps track of modified data
• all operations resulting in modifying DO databases trigger a print statement

4. user views df again

• now we see that there is no warning about unsynced data/local changes

5. user turns on auto_sync to live dangerously

• a message is shown indicating that changes will be automatically written when viewing df

6. user makes some changes

• changes automatically written
• this triggers a print statement
• no warning about local unsaved data

technical implementation

we create a new class that subclasses a pandas dataframe, and write special methods to talk to the data hub

changes

• [x] api.get_dataframe() stores database info in dataframe attrs
• [x] dataframe contains information about columns
• [x] dataframe.sync works
• [x] support for intelligently creating new columns won't do this PR
• [x] updating a column writes only those values to that DB
• [x] updating a single cell writes only that value to the DB
• [x] documentation describing how to use this
• [x] pretty printing of databases includes a link to the database on the web UI
• [x] auto_sync should be turned off when running in the lambda to preserve existing behavior

changes post review

• [x] when auto_sync is False, df keeps track of modified columns so that we can selectively update those
• [x] auto_sync is False by default
• [x] printing df in jupyter shows last modified row and by whom
• [x] printing df in jupyter tells me if there are modifications that need to be written back to DO
• [x] printing df in jupyter tells me if auto_sync is enabled
• [x] docs updated with new behavior
• [x] docs updated with screenshots
• [x] every operation of to_deeporigin will print a message to console