Petastorm library enables single machine or distributed training and evaluation of deep learning models from datasets in Apache Parquet format. It supports ML frameworks such as Tensorflow, Pytorch, and PySpark and can be used from pure Python code.
Apache License 2.0
1.78k
stars
285
forks
source link
'WorkerThread' object has no attribute 'isAlive' #692
After updating to Python 3.9.5 and using Petastorm '0.11.1', I am getting an error 'WorkerThread' object has no attribute 'isAlive'. My code mostly follows the databricks example. Based off some other posts such as this, it looks like
It seems that thread's method isAlive() was renamed to is_alive() in python verson 3.9
Is there a temporary workaround?
Thanks
Full error:
AttributeError Traceback (most recent call last)
D:\Data\BayesNN.py in train_and_evaluate_SVI(self, svi, criterion, bs, ne, lr)
141
--> 142 train_loss = self.train_one_epoch_SVI(svi, train_dataloader_iter, steps_per_epoch, epoch, self.device)
143
d:\data\venv\lib\site-packages\petastorm\spark\spark_dataset_converter.py in __exit__(self, exc_type, exc_value, exc_traceback)
403 self.reader.stop()
--> 404 self.reader.join()
405
d:\data\venv\lib\site-packages\petastorm\reader.py in join(self)
658 """Joins all worker threads/processes. Will block until all worker workers have been fully terminated."""
--> 659 self._workers_pool.join()
660
d:\data\venv\lib\site-packages\petastorm\workers_pool\thread_pool.py in join(self)
186 for w in self._workers:
--> 187 if w.isAlive():
188 w.join()
AttributeError: 'WorkerThread' object has no attribute 'isAlive'
During handling of the above exception, another exception occurred:
AttributeError Traceback (most recent call last)
<timed eval> in <module>
D:\Data\BayesNN.py in train_and_evaluate_SVI(self, svi, criterion, bs, ne, lr)
140 if (epoch + 1) % 10 == 0: print('-' * 10)
141
--> 142 train_loss = self.train_one_epoch_SVI(svi, train_dataloader_iter, steps_per_epoch, epoch, self.device)
143
144 #if (epoch + 1) % 10 == 0: val_loss = evaluate(model, criterion, val_dataloader_iter, validation_steps, device) #val_loss =
d:\data\venv\lib\site-packages\petastorm\spark\spark_dataset_converter.py in __exit__(self, exc_type, exc_value, exc_traceback)
402 def __exit__(self, exc_type, exc_value, exc_traceback):
403 self.reader.stop()
--> 404 self.reader.join()
405
406
d:\data\venv\lib\site-packages\petastorm\reader.py in join(self)
657 def join(self):
658 """Joins all worker threads/processes. Will block until all worker workers have been fully terminated."""
--> 659 self._workers_pool.join()
660
661 @property
d:\data\venv\lib\site-packages\petastorm\workers_pool\thread_pool.py in join(self)
185 """Block until all workers are terminated."""
186 for w in self._workers:
--> 187 if w.isAlive():
188 w.join()
189
AttributeError: 'WorkerThread' object has no attribute 'isAlive'
After updating to Python 3.9.5 and using Petastorm '0.11.1', I am getting an error
'WorkerThread' object has no attribute 'isAlive'
. My code mostly follows the databricks example. Based off some other posts such as this, it looks likeIs there a temporary workaround?
Thanks
Full error: