search

NAVADMC / ADSM

A simulation of disease spread in livestock populations. Includes detection and containment simulation.
Other
10 stars 5 forks source link

Download Files To Workspace #1005

Closed BryanHurst closed 4 years ago

BryanHurst commented 4 years ago

The new Viewer doesn't have a download handler, so all "downloads" should automatically save to the appropriate folder in the workspace and notify the User where to find it.

missyschoenbaum commented 4 years ago

Here's the list of downloads that will need to be tested.

Note: regular supplemental output files are created in appropriate directory.

ConradSelig commented 4 years ago

If we wan't to change where they are saving and/or add messages we can do that.

Other than that exports are not working, I'm going to start getting them sorted out now.

@BryanHurst You've worked on the SQL Panel, maybe you know where that is saving files?

@missyschoenbaum I'm not sure what "download error" you are seeing by the Result's file names, but those supplemental output downloads are also broken. I've added them to the list to fix. Maybe you have a screenshot of the error?

missyschoenbaum commented 4 years ago

@ConradSelig the problem on Results Supp Outputs isn't that an error shows up. The problem is that we offer a button to download, and then nothing happens. In this case, they really don't need to download, because the files are already written to the correct spot. While this may be the same problem as downloading the image, in this case we have already downloaded the files.

One option would be to remove the action buttons that show up at the end of the file list. This might be a quick. resolution on this. A slightly more robust action might be to change the title from plain "Supplemental Output Files" to something like ""Supplemental Output Files - present in scenario diretory"

SuppFileDownload

missyschoenbaum commented 4 years ago

@BryanHurst I tested again on the SQL output (save to csv or excel). I saved the query with a specific name, ran it. I can see output on the screen, but it didn't write to a file that I am able to find by searching the file system.

missyschoenbaum commented 4 years ago

I also searched the whole C drive, in case it was putting it back somewhere in the install files.

ConradSelig commented 4 years ago

OK I did some research on the sql explorer not saving results problem and figured out whats going on.

First off, clicking "Save & Run" (rather non-intuitively I should add) does not download the results. Instead, you can follow these steps to actually download the results:

  1. Write your query and then click "Save & Run".
  2. On the top navigation bar, select "Logs".
  3. Press "Open" next to the query you want to download in the list on this page.
  4. Click "Download" below the query writing box, or press the drop-down to select the file format.
  5. Select a place to download the file.

Not sure why it's like this, but we didn't write the django-sql-explorer. We might be able to move that download button to the first page, or maybe redirect the users automatically? Otherwise the easiest option is just to write some documentation on the problem.

missyschoenbaum commented 4 years ago

@ConradSelig I will get documentation done.

BryanHurst commented 4 years ago

"Save & Run" means it is saving the Query that is written in the text field. The dropdown next to it is what allows you to download files. Note: You don't have to go into the Logs or Playground sections to download query results.

However, this still doesn't account for the fact that the Viewer application doesn't have a download handler built in.
You Cannot download the files from the SQL Explorer in the Viewer.

We'll leave this issue open to see if we have time to fork SQL Explorer and have the download options work with ADSM's Workspace paradigm before our next build.

missyschoenbaum commented 4 years ago

@ConradSelig I can follow your instructions, but I still don't seem to be getting a file. I am selected CSV. Back on the SQL explorer main page, I can see my query has a CSV, and what looks like a download arrow. Are you actually getting a file somewhere?

ConradSelig commented 4 years ago

I should have spotted it right away but this cant work because our current viewer doesn't have a download handler (that's literally this ticket!). Here is a screenshot of what I see for reference.

image

But you won't get that download popup when you hit download.

Not sure what to do about this.

ConradSelig commented 4 years ago

OK I see three possible solutions for the PDF White Paper and the Data Dictionary.

Option number 1: We ship these two documents with ADSM itself. These documents could then only be updated if ADSM is updated. We change the UI to allow the users to re-save the files in their ADSM workspace.

Option number 2: We upload these documents to Google Drive, and link ADSM to there so that when users go to view these documents in ADSM they don't have to download them. This biggest disadvantage here is that I believe Missy has problems accessing Google Drive while on-site, and I'm not sure if these problems would carry over or not. We could also use a different online service like Zoho, which is even more like Word and Excel then Google Drive is. Again not sure how this would actually show up in ADSM, or what accessibility would be for users in a similar situation to Missy.

Option number 3: We translate these two documents into markdown and upload them to the wiki, which will then be viewable from within ADSM. This might sound really painful, but I don't actually think it would be that bad. What would be painful is editing later on - so we would want to be certain the documents are fully up-to-date before doing this.

missyschoenbaum commented 4 years ago

@ConradSelig let's go with option 3. If you will give me an example markdown table, I can fill in. It looks like I did table and field level definitions.

ConradSelig commented 4 years ago

@missyschoenbaum I went ahead and just did the first page as an example. Here it is:

https://github.com/NAVADMC/ADSM/wiki/Data-Dictionary---Database-Overview

I recommend using this online tool for making markdown tables. There is going to be a bit of a learning curve with using this tool - but it should still be easier than trying to make it work out yourself.

ConradSelig commented 4 years ago

I went ahead and just did the PDF page. It can be found here: https://github.com/NAVADMC/ADSM/wiki/Probability-Density-Functions-Overview

ConradSelig commented 4 years ago

That is everything except the django-sql-explorer problems.

If I really had to make a call on what to do about that problem - I would say that we should remove the download button and tell users that the build-in sql explorer is for "quick reference only" and that if they want to really dive into the sql database they should use sqllite studio. I say this because removing the download button should not be too difficult, and then we won't have to add another known bug either.

At the end of the day however, its Missy's call. Maybe we should just discuss this at out next meeting.

missyschoenbaum commented 4 years ago

I will leave this open until I get Wiki pages complete.

BryanHurst commented 4 years ago

Just a note that the above commit modifies the SQL Explorer plugin to properly save the files into the ADSM Workspace.

missyschoenbaum commented 4 years ago

Yeah Team! I figured I was taking the "L" on that one.

ConradSelig commented 4 years ago

I'm taking over the rest of translating the data dictionary spreadsheet over to markdown.

missyschoenbaum commented 4 years ago

Reference to fail test above - it appears I am getting the delimiter in the rel and pdf export. ExportRel

Population CSV didn't show this action.

BryanHurst commented 4 years ago

I'm going to hold off on building the next beta release until we can look into this issue and the two problems around #1001

BryanHurst commented 4 years ago

@ConradSelig the problem that @missyschoenbaum is seeing is due to the quotechar and quoting=csv.QUOTE_ALL parameters.

What was the logic behind using '¿' as a special character and to always quote?

I've pushed a different approach to this, but not knowing the original intent, it might cause issues. Conrad, take a look at the commit below and let me know if that breaks any of your reasoning.

ConradSelig commented 4 years ago

@BryanHurst I believe '¿' was implemented before we started using python's csv writer/reader. Why we didn't switch to a different delimiter and why csv.QUOTE_ALL was used I'm not sure. Too bad I can't go back in time and ask myself!

Everything looks good - that should work fine.

ConradSelig commented 4 years ago

Translating the data dictionary into the wiki is done. @missyschoenbaum It might be smart to go through these and make sure you like how everything ended up getting formatted.

missyschoenbaum commented 4 years ago

OK, looks good. I will double check documentation when I have time.