fohrloop / dash-uploader

The alternative upload component for python Dash applications.
MIT License
141 stars 29 forks source link

Change folder dynamically #17

Open VitalySvyatyuk opened 3 years ago

VitalySvyatyuk commented 3 years ago

Is it possible to change the folder dynamically? In my application user can change the working directory, so the Upload folder should be changed simultaneously. But when I'm trying to du.configure_upload() again, I'm getting an error: AssertionError: A setup function was called after the first request was handled. This usually indicates a bug in the application where a module was not imported and decorators or other functionality was called too late. To fix this make sure to import all your view modules, database models and everything related at a central place before the application starts serving requests.

fohrloop commented 3 years ago

I assume your application is used by running Dash on local computer and users use the app via browser. Changing the upload root folder dynamically would, in general, break every upload process that might be ongoing, and there is a possibility that it would crash the app altogether. Offhand I would say that it would require pretty clever tricks to make it work, if it is possible.

What would be the use case? The upload_id argument can be used to define a subfolder for the upload. Would that help? Another possibility is to create your own callback logic to copy/move the uploaded files where ever you wish.

sorenwacker commented 3 years ago

I have the same question. The dcc.Upload component stores the data in memory first and then I can decide how to parse it and where to store it. I have a similar setup that users can setup different workspaces and upload different file types which need to be stored in different folders for the app to find them.

fohrloop commented 3 years ago

Could callbacks be used to move the data into the correct directories?

I think data flow like this:

1) User uploads data file (newdata.csv) to server. File goes into C:\tmp\Uploads\some-unique-id\newdata.csv 2) Callback initiates validation of file, if needed 3) Callback initiates some calculation, and maybe updates some graph. 4) If certain criteria is met, and data is needed later callback moves the file into permanent storage that can depend for example on the username, or some selected settings. For example: C:\dashdata\user123\prices\newdata.csv

I also assume that it would be beneficial to empty the temporary folder every now and then, so only the important files are stored for longer time. For example, by using a script that runs every 24 hours and removes all files from C:\tmp\Uploads\ that are older than 24 hours.

What do you think? Could this work for you, or would your use case require changes to dash-uploader?

sorenwacker commented 3 years ago

I think it could be done with callbacks. When the upload finishes the next callback moves the files to a better location. How do you get a username though? Are you using the enterprise version? I would like to be able to upload files and folders though, but that is a different issue.

sorenwacker commented 3 years ago

I now upload the file and then move it into a different folder.