improve file uploads

[kwheelan]

Goal

Improve the process for uploading files on the app. We want to allow the user to upload files from multiple folders (right now they must all be in the same folder). We can either allow multiple upload rounds or allow the user to select files from different folders in the dialog box.

Background

• app/templates/site/home.html includes the html code for the file upload form (lines 22-40)
• app/static/js/spinner.js is badly named, but it contains all the javascript code that controls the draggable file list (and that submits the POST request).
• The upload_files() function in routes.py processes the POST request and receives the files[] array

[lucakato]

@kwheelan I don't think it's possible to select multiple folders according to this. Instead, it might make sense to add a delete button next to each file. This way if the user accidentally uploads 1 extra file they don't have to go through the re-selection process of all files again, but just delete the single file.

[kwheelan]

@lucakato That makes sense -- we'll just have to make sure the file request still includes all the files in the correct order.

[lucakato]

@kwheelan Should be done but please check whenever possible! I also changed the saving location of viewer.html to templates/site because I realized saving it to sessions specific folder was breaking the online viewer.

I added a delete button while keeping the ability to move around files. Also if you delete a file it should change the number of files displayed on the message "Convert X files"

[kwheelan]

@lucakato Thanks -- I'll take a look this weekend.

[kwheelan]

@lucakato

Thanks for your work on this. A few comments:

• Re moving the location of viewer.html: we ultimately need that file to be written to a session-specific file. If we write to templates/ one person using the app will change the viewer for all users. Writing to a session-specific file should allow multiple simultaneous users. What error were you getting? We might just need to move the javascript it relies on to a different location. I haven't looked closely at the actual script, but if it's different for each viewer instances, we'll need to move it into the sessions folders as well. See lines 78-88 of app/utils/xbrl_processing.py where we move the js script and change the reference in the viewer html.

• I like the delete button, but the update on the number of files displayed on the message "Convert X files" is not always accurate. When I add 5 files and delete some, it doesn't always correctly decrease the file count. We can probably just get rid of the counter completely and have it say "convert your files." But it seems like there is something else going on because I've also had some other unexpected behavior when testing. When I upload several files and delete all but the Excel file, I get the error "please upload exactly 1 Excel file."

• I'll do a bit of research as well to see if we can find a way to get file uploads from multiple locations to work. I'll leave this issue open until then.

[lucakato]

@kwheelan thanks for the review. I'll try to re-create that error I got by reverting it to saving to the session specific folder.

I'll also just keep it to 'Convert X files' and then test it more.

[kwheelan]

@lucakato Let me know if/when you want me to review this

[lucakato]

@kwheelan thanks, I will let you know. I fixed the error message thing. I was working on this other bug where files weren't being shown when everything was deleted and then files were re-selected to be uploaded but now that's been fixed too. Then I'll probably try moving the saving location of viewer.html back to the session and see if I break anything again.

[kwheelan]

@lucakato Sounds good

[lucakato]

@kwheelan okay so here's the error message:

I think render_template only works on something stored in the /templates folder.

Do you think modifying the folder/code structure like this is okay?

project_root/
    app/
        templates/
            session_templates/
                sessions_data/
                    <session_id>/
                        viewer.html
            base.html
            other_template.html
        static/
            css/
            js/
        routes.py
        ...