data download and access control

[scfulford]

Hi I've been having an issue when trying to download data. After using the workflow designer to create inferences from the model and requesting the data on the "Data Download" page, a date comes up under the "Downloads" section but the corresponding file link doesn't show up with it. We also had some problems with getting the Access Control page to load, it seems to get stuck on the loading screen.

Thank you!

[bkellenb]

Hello,

Thank you for the report. A missing download link could have multiple causes: a. The Celery worker is not running. Preparing files for data download is dispatched by the main server, but handled through a Celery task by the file server machine. If you don't use Docker, you need to install and launch the appropriate Celery consumer in addition to the server:

export AIDE_CONFIG_PATH=/path/to/config.ini
export AIDE_MODULES=FileServer      # add "LabelUI,AIController,AIWorker" if only one machine is being used
./launch_celery.sh

b. There may also be a bug in the data preparation implementation. For this I would need to know more about the annotation and prediction types you use, the number of images, number of annotations and/or predictions, as well as the settings made when requesting a download link. Also, a potential error message on the console (F12 in the browser) may help. I will in any case try to reproduce the error and will let you know about success and following bug fixes.

As for the access control page: this is fixed now from version 1.9.201009.

Details: I don't want to speak out too loudly, but this looks to me like a bug in the HTML- or JavaScript parsing engines of Chrome as well as Webkit. When using this command (file "accessControl.html", line 142):

let isAdmin = data['users'][i]['isadmin'] ? '<img src="/static/general/img/success.svg"></img>' : '<img src="/static/general/img/error.svg"></img>';

it works fine (implemented for safety in the latest version of AIDE, ver. 1.9.201009). However, this version (note the valid <img src="" /> shorthand):

let isAdmin = data['users'][i]['isadmin'] ? '<img src="/static/general/img/success.svg" />' : '<img src="/static/general/img/error.svg" />';

causes Chromium to garble up the code definition, and Webkit to throw an "unexpected identifier" error. I honestly don't know why this happens, as I have used this exact code snippet in other files, too (e.g., "userAccounts.html").

[ctorney]

Hi @bkellenb This seems to be an issue with the temp directory - the file is being created ok but when saving it checks for the FileServer temp directory parameter: https://github.com/microsoft/aerial_wildlife_detection/blob/1253cae4c5a55e2bc4799b083f6f6018ed9e011a/modules/DataAdministration/backend/dataWorker.py#L54 then when retrieving it only seems to use /tmp/ https://github.com/microsoft/aerial_wildlife_detection/blob/1253cae4c5a55e2bc4799b083f6f6018ed9e011a/modules/DataAdministration/app.py#L394

For larger files it also appears to timeout before the file is ready so the file is created but the link doesn't get added. If I remove this line https://github.com/microsoft/aerial_wildlife_detection/blob/4718bfb519c53f88eacfd4c31e0969999997a2ff/modules/DataAdministration/static/js/taskPolling.js#L44 then it keeps polling until the file is ready and adds to the list box

[bkellenb]

Thank you @ctorney! The temp dir is fixed in version 1.9.201015. As for the download of large files: that code line mentioned occurs in an error case; I first have to check what causes the error and will post updates here once the culprit is found.

[scfulford]

Hi The the model prediction download file that I was having trouble with initially is all working now. Thank you! However, when we try to select predictions from within a certain date range, file that's produced in empty (only the column headings are present). I'm certain I'm picking dates that I'd made predictions between. Do you have any idea what would cause that to happen? Thank you again!