dandi / dandi-api-webshots-tools

0 stars 0 forks source link

selenium.common.exceptions.TimeoutException crashing the process #2

Closed yarikoptic closed 3 years ago

yarikoptic commented 3 years ago

Just received an email from cron on dandi (not staging) which stated

2021-08-26T12:10:44-0400 [INFO    ] 11433 000013 view-data                                                                                                                                                                                                                                                    
2021-08-26T12:10:52-0400 [INFO    ] 11433 000015 landing                                                                                                                                                                                                                                                      
concurrent.futures.process._RemoteTraceback:                                                                                                                                                                                                                                                                  
"""                                                                                                                                                                                                                                                                                                           
Traceback (most recent call last):                                                                                                                                                                                                                                                                            
  File "/home/dandi/miniconda3/lib/python3.8/concurrent/futures/process.py", line 239, in _process_worker                                                                                                                                                                                                     
    r = call_item.fn(*call_item.args, **call_item.kwargs)                                                                                                                                                                                                                                                     
  File "/home/dandi/miniconda3/lib/python3.8/concurrent/futures/process.py", line 198, in _process_chunk                                                                                                                                                                                                      
    return [fn(*args) for args in chunk]                                                                                                                                                                                                                                                                      
  File "/home/dandi/miniconda3/lib/python3.8/concurrent/futures/process.py", line 198, in <listcomp>                                                                                                                                                                                                          
    return [fn(*args) for args in chunk]                                                                                                                                                                                                                                                                      
  File "tools/make_webshots.py", line 275, in snapshot_page                                                                                                                                                                                                                                                   
    with Webshotter(dandi_instance) as ws:                                                                                                                                                                                                                                                                    
  File "tools/make_webshots.py", line 69, in __init__                                                                                                                                                                                                                                                         
    self.set_driver()                                                                                                                                                                                                                                                                                         
  File "tools/make_webshots.py", line 93, in set_driver                                                                                                                                                                                                                                                       
    self.login(os.environ["DANDI_USERNAME"], os.environ["DANDI_PASSWORD"])                                                                                                                                                                                                                                    
  File "tools/make_webshots.py", line 121, in login                                                                                                                                                                                                                                                           
    el = WebDriverWait(self.driver, 300).until(                                                                                                                                                                                                                                                               
  File "/home/dandi/cronlib/dandi-api-webshots/venv/lib/python3.8/site-packages/selenium/webdriver/support/wait.py", line 80, in until                                                                                                                                                                        
    raise TimeoutException(message, screen, stacktrace)                                                                                                                                                                                                                                                       
selenium.common.exceptions.TimeoutException: Message:                                                                                                                                                                                                                                                         

"""                                                                                                                                                                                                                                                                                                           

The above exception was the direct cause of the following exception:                                                                                                                                                                                                                                          

Traceback (most recent call last):                                                                                                                                                                                                                                                                            
  File "tools/make_webshots.py", line 365, in <module>                                                                                                                                                                                                                                                        
    main()                                                                                                                                                                                                                                                                                                    
  File "/home/dandi/cronlib/dandi-api-webshots/venv/lib/python3.8/site-packages/click/core.py", line 829, in __call__                                                                                                                                                                                         
    return self.main(*args, **kwargs)                                                                                                                                                                                                                                                                         
  File "/home/dandi/cronlib/dandi-api-webshots/venv/lib/python3.8/site-packages/click/core.py", line 782, in main                                                                                                                                                                                             
    rv = self.invoke(ctx)                                                                                                                                                                                                                                                                                     
  File "/home/dandi/cronlib/dandi-api-webshots/venv/lib/python3.8/site-packages/click/core.py", line 1066, in invoke                                                                                                                                                                                          
    return ctx.invoke(self.callback, **ctx.params)                                                                                                                                                                                                                                                            
  File "/home/dandi/cronlib/dandi-api-webshots/venv/lib/python3.8/site-packages/click/core.py", line 610, in invoke                                                                                                                                                                                           
    return callback(*args, **kwargs)                                                                                                                                                                                                                                                                          
  File "tools/make_webshots.py", line 310, in main                                                                                                                                                                                                                                                            
    for stat in executor.map(                                                                                                                                                                                                                                                                                 
  File "/home/dandi/miniconda3/lib/python3.8/concurrent/futures/process.py", line 484, in _chain_from_iterable_of_lists                                                                                                                                                                                       
    for element in iterable:                                                                                                                                                                                                                                                                                  
  File "/home/dandi/miniconda3/lib/python3.8/concurrent/futures/_base.py", line 611, in result_iterator                                                                                                                                                                                                       
    yield fs.pop().result()                                                                                                                                                                                                                                                                                   
  File "/home/dandi/miniconda3/lib/python3.8/concurrent/futures/_base.py", line 439, in result                                                                                                                                                                                                                
    return self.__get_result()                                                                                                                                                                                                                                                                                
  File "/home/dandi/miniconda3/lib/python3.8/concurrent/futures/_base.py", line 388, in __get_result                                                                                                                                                                                                          
    raise self._exception                                                                                                                                                                                                                                                                                     
selenium.common.exceptions.TimeoutException: Message:         

so it seems we aren't restarting the process and re-doing upon time out?