The previous version of this uses Enums with a switch cases, also it had hard-coded file extensions baked into the file type (which was undesired - for reasons). While speaking with @nlordell about this, he mentioned that "I manually implemented Dynamic Dispatch" and gave some very good suggestions similar to what we see here.
Now, there are three structures here:
FileRWInterface and abstract class declaring load, write and append. In some cases there is a base implementation provided.
JSONFile, CSVFile & NDJSONFile which implement the FileRWInterface
FileIO which is the "user interface". This class allows the user to to specify and encoding and path to do all the file operations by any of the three file implementations. It provides convenience methods write_XXX, load_XXX and append_XXX for each of the existing filetypes.
There is still a bit of awkwardness in this code that I would love to get suggestions on
load_singleton still attempts to parse a file extension and we still have this switch case. It also results not needing to provide a valid file name if you have your hands already on an instance implementation of FileRWInterface. It is the intention that users of this package won't actually import FileInterfaces directly, but rather use the FileIO with its convenience methods.
both of Write and Load methods actually take TextIO as a function parameter, while append is the only method that "opens" the file. That is to say, two of the methods take an already open file as input, while the third doesn't follow the same pattern. I suspect that could be changed and it might make things a bit more consistent.
Test Plan
All the existing tests were adapted to work with this new setup (but logically they are the same). A few additional tests were added to demonstrate the new capabilities (specifically that you can save any file format with whatever extension you may want).
Follow Up Ideas
Implement FileRWInterface for SQLLite database. The awkward thing about this is that no types are provided when querying dune. It is always a file of type list[dict[str, str]]. I have requested from Dune that they return column types along with the query results (this would make things even better).
The previous version of this uses Enums with a switch cases, also it had hard-coded file extensions baked into the file type (which was undesired - for reasons). While speaking with @nlordell about this, he mentioned that "I manually implemented Dynamic Dispatch" and gave some very good suggestions similar to what we see here.
Now, there are three structures here:
encodingandpathto do all the file operations by any of the three file implementations. It provides convenience methodswrite_XXX,load_XXXandappend_XXXfor each of the existing filetypes.There is still a bit of awkwardness in this code that I would love to get suggestions on
load_singletonstill attempts to parse a file extension and we still have this switch case. It also results not needing to provide a valid file name if you have your hands already on an instance implementation ofFileRWInterface. It is the intention that users of this package won't actually importFileInterfacesdirectly, but rather use theFileIOwith its convenience methods.both of Write and Load methods actually take
TextIOas a function parameter, whileappendis the only method that "opens" the file. That is to say, two of the methods take an already open file as input, while the third doesn't follow the same pattern. I suspect that could be changed and it might make things a bit more consistent.Test Plan
All the existing tests were adapted to work with this new setup (but logically they are the same). A few additional tests were added to demonstrate the new capabilities (specifically that you can save any file format with whatever extension you may want).
Follow Up Ideas
FileRWInterfaceforSQLLitedatabase. The awkward thing about this is that no types are provided when querying dune. It is always a file of typelist[dict[str, str]]. I have requested from Dune that they return column types along with the query results (this would make things even better).