search

nextml / NEXT

NEXT is a machine learning system that runs in the cloud and makes it easy to develop, evaluate, and apply active learning in the real-world. Ask better questions. Get better results. Faster. Automated.
http://nextml.org
Apache License 2.0
160 stars 54 forks source link

hidden directories in apps folder break stuff #157

Closed dconathan closed 7 years ago

dconathan commented 7 years ago 
nextbackenddocker_1  |   File "/next_backend/next/dashboard/dashboard.py", line 50, in experiment_list
nextbackenddocker_1  |     for exp_uid in rm.get_app_exp_uids(app_id):
nextbackenddocker_1  |   File "/next_backend/next/api/resource_manager.py", line 101, in get_app_exp_uids
nextbackenddocker_1  |     for doc in docs:
nextbackenddocker_1  | TypeError: 'NoneType' object is not iterable

I'm not sure if this is related to failed experiments or not, but I get an error when I try to launch the NEXT /home page or any of the dashboards (query pages still load fine)

If I add a if docs: to that statement, everything loads fine.

stsievert commented 7 years ago

I've tried reproducing this issue but can't. Can you give more detail and include print statements from utils.get_app_exp_uids?

dconathan commented 7 years ago

Started looking into it... put a debug_print statement in get_app_exp_uids... If I try to load the experiment dashboard page, get_app_exp_uids gets called 7 times. The first 5 times it returns [], True, '', the 6th time it returns the expected result (list of dictionaries containing exp info), the 7th time it returns None, False, MongoDB.getDocs Failed with unknown exception

I will keep digging to see what's happening that last time... At first I thought it was because of failed experiments (as in, experiments that "partially launched" or something), but I tried rebuilding images/with fresh containers and this still happens.

dconathan commented 7 years ago

nextbackenddocker_1 | InvalidName: collection names must not start or end with '.': '.ropeproject:experiments'

Know what that's about?

dconathan commented 7 years ago

Haha. It was a hidden directory in my apps folder...

Should we maybe protect against this? Like, when looking for apps, ignore hidden directories since they cause database errors and break everything?

stsievert commented 7 years ago

Should we maybe protect against this?

Absolutely – I've recently implemented a similar line in utils.get_supported_apps#L25. Feel free to put in a PR for this.

dconathan commented 7 years ago

Interesting. That does seem to be the only time a os.walk is used... it's possible I was using an old .pyc version of utils.