What happens right now all tbls become stale, src becomes stale and one needs to refresh by hand.
Possible steps:
src stores creation call, uses it to refresh itself
tbl can temporary (TT) or persistent (PT). TT are just a backing for an in-memory object, PT have a life of their own. The table behind a TT should be destroyed at gc time, the one behind PT should outlive it. When a TT is saved, meaning it has a chance to be loaded in another session it should become a PT. The latter is problematic because the risk of leaving behind, but it mimicks the behavior for regular objects.
> z = data.frame(col1, col2)
> quit()
save? N
$ R
> #z is gone
> z = data.frame(col1, col2)
> quit()
save? Y
$ R
> print(z)
> #z is still there
> rm(z) #z gone
There's a third possibility where you can save z to a file and load it later. Now in an ideal world we can go through the same use case with a tbl_SparkSQL and never have z in some butchered state where it exists but is invalid. This may be a pipe-dream because I don't think save is a generic so that we can do something with it, but I thought I'd write it down.
What happens right now all tbls become stale, src becomes stale and one needs to refresh by hand. Possible steps:
There's a third possibility where you can save z to a file and load it later. Now in an ideal world we can go through the same use case with a tbl_SparkSQL and never have z in some butchered state where it exists but is invalid. This may be a pipe-dream because I don't think save is a generic so that we can do something with it, but I thought I'd write it down.